sortdx

A sorting library and command line tool for Python that handles CSV, TSV, JSONL, and text files, including gzip and zstandard compressed files. Large files are sorted on disk through chunking and a k-way merge, so memory usage stays bounded. The core library has no required dependencies and ships full type information.

Installation

pip install sortdx

The core library has no required dependencies. The command line interface and extra features (encoding detection, flexible date parsing) need the optional dependencies:

pip install "sortdx[full]"

Quick start

import sortdx

# Sort a CSV by a numeric column
sortdx.sort_file("data.csv", "sorted.csv", keys=[sortdx.key("age", "num")])

# Sort a JSONL file by timestamp
sortdx.sort_file("logs.jsonl", "sorted.jsonl", keys=[sortdx.key("timestamp", "date")])

# Sort in-memory data
data = [{"name": "Bob", "age": 30}, {"name": "Alice", "age": 25}]
result = list(sortdx.sort_iter(data, keys=[sortdx.key("age", "num")]))

Sort keys

Create sort keys with sortdx.key():

# String sorting (default, case-insensitive)
sortdx.key("name")

# Numeric sorting
sortdx.key("price", "num")

# Date sorting
sortdx.key("created_at", "date")

# Natural sorting: file2 before file10
sortdx.key("filename", "nat")

# Descending order for one key
sortdx.key("salary", "num", desc=True)

# Locale-aware string collation
sortdx.key("name", "str", locale_name="fr_FR.UTF-8")

Columns are addressed by name for dict rows (CSV, JSONL), by integer index for list rows, and None sorts plain text lines as whole items.

Data types

Empty and unparseable values sort first within their type.

API

sortdx.sort_file(input_path, output_path, keys, ...)

Sort a file into another file.

stats = sortdx.sort_file(
    "huge.jsonl.gz",
    "sorted.jsonl.gz",
    keys=[sortdx.key("timestamp", "date")],
    memory_limit="512M",
    stats=True,
)
print(f"Processed {stats.lines_processed} lines in {stats.processing_time:.2f}s")

sortdx.sort_iter(data, keys, ...)

Sort an iterable in memory and return an iterator. Accepts the same reverse and unique arguments.

Multi-key sorting

sortdx.sort_file("employees.csv", "sorted.csv", keys=[
    sortdx.key("department"),
    sortdx.key("salary", "num", desc=True),
])

Command line

# Sort a CSV by price, then name; write to a file
sortdx data.csv -o sorted.csv -k price:num -k name:str

# Sort a compressed JSONL file with a memory budget
sortdx logs.jsonl.gz -o sorted.jsonl.gz -k timestamp:date --memory-limit=512M

# Natural sort of a text file to stdout
sortdx filenames.txt --natural

# Keep one row per user id, newest first
sortdx users.jsonl -o unique.jsonl -k created_at:date --reverse --unique=user_id

Key specifications use the format column:type[:option=value], for example price:num, name:str:locale=fr_FR.UTF-8, or rank:num:desc=true.

Supported formats

Development

git clone https://github.com/Okymi-X/sortdx.git
cd sortdx
pip install -e ".[full,dev]"
pytest

See CONTRIBUTING.md for the full set of checks (formatting, lint, type checking) and the contribution guidelines.

Changelog

Release history is documented in CHANGELOG.md.

License

MIT License. See the LICENSE file for details.

Contributing

Contributions are welcome. Please open an issue or submit a pull request on GitHub.