csvpeek

A fast, zero-dependency CSV profiler for the terminal. Point it at a file and get column types, null counts, numeric statistics, and top values, in one glance.

Pure Python standard library. Deterministic: the same file always produces the same profile. No pandas, no install-time downloads, nothing leaves your machine.

$ csvpeek examples/people.csv

  6 rows × 5 columns

  column  type      nulls  unique  summary
  ────────────────────────────────────────
  name    string        0       6  Ann (1), Bob (1), Cara (1), Dan (1), Eve (1)
  age     int     1 (16.7%)       4  min 22 · p25 25.5 · median 29 · p75 37.5 · max 41 · mean 31 · sd 6.2929 · hist ▅▁▁█▁▁▅▁▁▅
  city    string        0       3  Santa Cruz (3), Brentwood (2), Oakland (1)
  active  bool          0       2  true (4), false (2)
  score   float   1 (16.7%)       5  min 6.5 · p25 6.8 · median 8.8 · p75 9.2 · max 9.4 · mean 8.16 · sd 1.143 · hist █▁█▁▁▁▁███

Install

Requires Python 3.9+.

pip install csvpeek-cli

The distribution is csvpeek-cli because csvpeek on PyPI belongs to an unrelated project. The command, the import and this repository are all still csvpeek; only the string after pip install differs.

Or run it straight from a clone, no install needed:

python -m csvpeek data.csv

Usage

csvpeek data.csv                 # profile a CSV
cat data.csv | csvpeek -         # read from standard input
csvpeek data.tsv -d $'\t'        # tab-separated
csvpeek data.csv --top 10        # more top values for text columns
csvpeek big.csv -n 10000         # sample the first 10k rows of a large file
csvpeek data.csv -c age,city     # profile only these columns, in this order
csvpeek data.csv --format md     # a Markdown table to paste into a PR
csvpeek data.csv --json          # machine-readable output
csvpeek data.csv --no-color      # plain text

What it computes

• Type inference per column: int, float, bool, date, string, or empty. A column takes a type only if every non-null value fits; one stray label keeps it a string (no silent coercion). Dates require a separator (2024-01-01, 01/02/2024), so bare years and ids stay numeric.
• Nulls: empty, NA, N/A, null, none, nan, nil (case-insensitive), with a percentage.
• Numeric columns: min, 25th/50th/75th percentiles, max, mean, and population standard deviation, plus a small histogram of the values bucketed into ten bins, shown as a sparkline in the table and as edges/counts in the JSON.
• Text/bool columns: unique count and the most common values (ties broken alphabetically, so runs are reproducible).
• Near misses: a string column that is mostly one type says so, and names the values stopping it, mostly int, 2 not: "twelve", "x". This is the question a profile usually raises and rarely answers: the column was supposed to be a number, and finding the two rows that ruined it otherwise means grepping.

Library

Everything the CLI does is available as a small API:

from csvpeek import profile_file

profile = profile_file("data.csv")
print(profile.rows)
for col in profile.columns:
    print(col.name, col.dtype, col.nulls, col.mean)

profile.to_dict()   # JSON-ready

Documentation

Development

pip install pytest .
pytest -q

Related

Four small tools that each do one thing to a table of data:

• csvpeek profiles a file: column types, null counts, distributions.
• sift queries one: filter, sort, aggregate, streaming.
• drift diffs two of them, in Rust.
• quarry is the long way round, a hand-written SQL parser and executor meant to be read.

License

MIT © Martin Muskov