io-bound-data-processing
pproenca/dot-skillsProcessing, transforming, or moving datasets that may exceed RAM on a single low-compute box — covers memory discipline (streaming, generators, dtype…
Scores out of 100 · grade B+
2026-08-22Works
40% of the score100/100
- Loads cleanly: valid frontmatter, required fields present, no dangling references.
Maintained
25% of the score100/100
- no commits in the last 12 weeks
Adopted
20% of the score31/100
- 196 stars on the source repo.
Documented
15% of the score64/100
- No usage example or code block.
- 1,184-word body.
- Ships 46 bundled files.
Install
npx skills add pproenca/dot-skills/io-bound-data-processingWhat it says it does
Processing, transforming, or moving datasets that may exceed RAM on a single low-compute box — covers memory discipline (streaming, generators, dtype shrinkage), I/O access patterns (sequential vs random, mmap, async), data formats (Parquet vs CSV vs JSON, predicate pushdown), chunking & batching, spill-to-disk (external merge sort, DuckDB/Polars), pipelining (bounded queues, backpressure, checkpointing), codec selection (zstd/lz4/gzip), concurrency for I/O-bound workloads (asyncio, threads, prefetch), and observability (iowait vs CPU%, rows/sec, py-spy/strace). Trigger on "process a large file", "stream this", "out-of-core", "OOM kill", "this is slow", or code with `pd.read_csv` of multi-GB files, `requests.get(...).content` on big bodies, `BytesIO` on unbounded inputs, per-row INSERTs, sequential `requests.get` loops, falling `tqdm` rates — even if I/O or memory isn't mentioned. Complement to computer-science-algorithms.
Also in pproenca/dot-skills
| Artifact | Score | What the check found | Type | Reach | Last commit |
|---|---|---|---|---|---|
| bug-reviewpproenca/dot-skills | clean | Skill | 196 stars | 6 days ago | |
| ios-tastepproenca/dot-skills | clean | Skill | 196 stars | 6 days ago | |
| web-tastepproenca/dot-skills | clean | Skill | 196 stars | 6 days ago | |
| build-mcp-serverpproenca/dot-skills | clean | Skill | 196 stars | 6 days ago | |
| cli-review-runnerpproenca/dot-skills | clean | Skill | 196 stars | 6 days ago | |
| complexity-optimizerpproenca/dot-skills | clean | Skill | 196 stars | 6 days ago |
Other office & files skills
Browse all| Artifact | Score | What the check found | Category | Reach | Last commit |
|---|---|---|---|---|---|
| pdfanthropics/skills | No license | Office & files | 182,719 installs | today | |
| pptxanthropics/skills | No license | Office & files | 206,139 installs | today | |
| xlsxanthropics/skills | No license | Office & files | 156,275 installs | today | |
| instrument-data-to-allotropeanthropics/knowledge-work-plugins | clean | Office & files | 1,950 installs | today | |
| pitch-deckanthropics/financial-services | clean | Office & files | 1,942 installs | today | |
| pptx-html-fidelity-auditnexu-io/open-design | clean | Office & files | 2,516 installs | today |
Put this measurement in your README
A badge carrying how many listings this index holds from the repository and how many pass every static structural check. It reads from this index every time somebody loads your page, so it changes when the measurement changes and there is nothing to keep up to date. Free, no account, and the value is not something you or we can set by hand.
[](https://skillworks.kynth.studio/?q=pproenca%2Fdot-skills)Would rather not hotlink us? Every badge is also served in shields.io’s endpoint schema, so shields renders the image and your readers never talk to our domain:
Published by Toolproof, the masthead over this index and eight others. The method behind the number is at toolproof.kynth.studio/methodology, and the whole thing is readable as JSON with no key at /api.
