Disk CSV Sort

Description

Sort huge CSV files using disk space and RAM together.

For now support only CSV files with header.

Usage

For example

CSV file with movies

Sort this CSV file that stored in movies.csv by year and name.

Note: order of columns is matter during sorting.

Using diskcsvsort package

from pathlib import Path
from diskcsvsort import CSVSort

csvsort = CSVSort(
    src=Path('movies.csv'),
    key=lambda row: (int(row['year']), row['name']),
)
csvsort.apply()

Using diskcsvsort CLI

python -m diskcsvsort movies.csv --by year:int --by name:str

Note: columns year and name will be converted to int and str, respectively.

Available types:

• str
• int
• float
• datetime
• date
• time

Types usage:

• str: column:str
• int: column:int
• float: column:float
• datetime: column:datetime(%Y-%m-%d %H:%M:%S)
• date: column:datetime(%Y-%m-%d)
• time: column:datetime(%H:%M:%S)

Algorithm

TODO

Metrics

TODO