Home

qsv Wiki

qsv is a blazing-fast, composable data-wrangling toolkit for CSV, TSV, SSV, Excel, JSON/JSONL, Parquet and more. 80 commands across five binary variants, and a single binary that fits in your toolbox.

Latest release: 28.0.0 — the "DCAT-US v3.0 neuro-symbolic FAIRification" release — ~58 commits since 27.0.1, and the banner feature is profile writing much more of a DCAT-US v3.0 record for you, the metadata standard data.gov is moving to. The symbolic half measures properties from the data: temporalResolution, accrualPeriodicity (only for a truly regular series), a spatial bounding box that drops out-of-range and "null island" rows, language, DQV quality measurements, and CSVW column constraints and a primary key. --locate names the places a dataset covers (offline GeoNames, or the US Census Geocoder, which also reads state and county columns), and a WKT POINT column now counts as coordinates. The neural half, --describegpt or --infer, adds an LLM description, tags (optionally held to a --tag-vocab) and a data dictionary, checked before use. Every value records where it came from in a new projection_provenance block. Also in this release: describegpt's answer cache works as documented (--fresh, --forget, multi-phase JSON/TOON runs, Redis); Data Schematics size themselves when embedded in an iframe; benchmarks on hyperfine 2.0 with peak memory; and least-privilege CI. See the full changelog.

Important

28.0.0 breaking changes (only if you consume profile's DCAT-US v3 output). The CSVW column datatype is now an object ({"base": "integer", "minimum": ...}) instead of a string, so read datatype.base (#4967; dcat-ap-v3 keeps the bare string). A theme built from CKAN groups is now a list of Concepts instead of group-name strings (#4972). Details in the changelog's Breaking Changes. (Since 24.0.0, the x86_64 prebuilts require AVX2, all variants except qsvdp — check first before qsv --update on an older CPU.)

Welcome! This wiki is the workflow + discoverability layer for qsv. Use it to learn how the commands fit together and find recipes for real-world tasks. For per-command flag references, follow the deep-links into /docs/help/ — that's the canonical reference.

Three ways to start

Path	When
Try it in your browser	You want to evaluate qsv before installing anything
Install qsv	You're ready to run it locally
Read the Whirlwind Tour	You want a guided 2.7M-row walkthrough

Then jump to Getting Started for a hands-on tutorial, or Why qsv? for the elevator pitch.

Where things live

qsv documentation lives on several surfaces. Each has a different job:

Surface	Job
README.md	Project overview, command catalog with legend, install options, feature flags
/docs/ (canonical reference)	The source of truth: PERFORMANCE.md, BENCHMARKS.md, ENVIRONMENT_VARIABLES.md, STATS_DEFINITIONS.md, Validate.md, Fetch.md, Describegpt.md, INTERPRETERS.md, plus the whirlwind tour
/docs/help/	Per-command help — flags, arguments, examples
This wiki	Workflow & discoverability: getting started, cookbook recipes, command-reference category guides, tuning, integrations, troubleshooting
100.dathere.com	Interactive lessons & exercises (WIP)
Discussions	Q&A, FAQ, announcements, ideas
	LLM-generated Wiki that crawls the codebase once a week, that you can chat with!

Browse by tier

Every wiki page has a Tier badge — Beginner, Intermediate, or Advanced. See the Tier Legend for what each tier assumes.

Beginner

Installation · Getting Started · Why qsv? · Command Reference (index) · Cookbook · Troubleshooting · FAQ · Glossary · External Resources · qsv pro

Intermediate

Binary Variants · Transform & Reshape · Aggregation & Statistics · Joins & Set Ops · Validation & Schema · Conversion & I/O · Geospatial · HTTP & Web · Indexing, Compression & Diff · AI & Documentation · Comparison vs others · Integrations · US Census Integration · Claude Cowork Plugin · MCP Server

Advanced

SQL & Polars · Scripting (Luau / Python) · Performance Tuning · Environment Variables · Stats Cache & Caching · Lookup Tables · Contributing to the Wiki

Anchor datasets used across this wiki

Examples in this wiki reach for a small set of datasets repeatedly, so you build mental models:

• wcp.csv — World Cities Population, 2.7M rows / 124 MB. Used in Getting Started and many command-reference examples. Available in files/wcp.zip.
• NYC 311 Service Requests — public dataset, available as a 1M-row sample and a full 16 GB / 27M-row export. Used in Validation, SQL & Polars, Geographic Enrichment, Larger-than-RAM.
• Boston 311 — small (100 rows) and in many compression formats (.csv, .gz, .zst, .parquet). Used in Clean & Normalize and conversion demos.
• Allegheny County property sales — mid-size, already indexed. Used in Build a Data Pipeline and Stats → Insights.
• NOAA GHCN-Daily — weather time-series, fetched via qsv fetch. Used in Fetch & Cache and Multi-table Joins (asof joins).
• GitHub stargazers — qsv's own stars, fetched via qsv fetch. Used in HTTP & Web demos.

Try it in your browser

No install required: visit qsv.dathere.com to try qsv against a sample CSV right now.

Talks & blog series

• "Have we achieved ACI?" series: 1 · 2 · 3 · 4
• NYC SOD 2026 · csv,conf,v9 · PyCon US 2025 · csv,conf,v8 · NYC SOD 2022
• Full list: External Resources

Legacy pages (preserved during wiki rewrite)

These pages predate the wiki rewrite. Their content has been integrated into the new structure but the originals are kept for backwards-compatibility with external links:

• Cookbook (legacy) — original recipes; new Cookbook expands them
• qsv cheat sheet — pre-rewrite infographic & quick reference (WIP)
• Luau Development — folded into Scripting (Luau / Python)
• Luau Helper Functions Examples — referenced from Scripting (Luau / Python)
• describegpt model notes — folded into AI & Documentation
• xsv 0.13.0 stats vs qsv 1.0.0 stats — referenced from Comparison vs others

Contributing to this wiki

This wiki is itself a git repo at https://github.com/dathere/qsv.wiki.git. See Contributing to the Wiki for clone/edit/push instructions and the verification checklist.