csv_pg

A Python module to import/export CSV files efficiently to/from a Postgresql database.

Key features:

• Detect CSV file dialect automatically
• Create target database table automatically
• Handle localized date/time and decimal formats
• Filter CSV file rows during import

Usage examples

Installation:

pip install csv_pg

Import CSV file tests/samples/misc.csv to table misc (recreate table if already exists):

python -m csv_pg from_csv tests/samples/misc.csv --recreate

Export table misc to CSV file misc.local.csv (recreate file if already exists):

python -m csv_pg to_csv misc.local.csv misc --recreate

Contributors guidelines

Prepare environment:

# Debian: sudo apt install libpq-dev
python -m venv .venv       # Debian: python3 -m venv .venv
.\.venv\Scripts\activate   # Linux: source .venv/bin/activate
pip install .[dev]

Create test database:

sudo -u postgres createdb -E UTF8 -l fr_FR.UTF8 -T template0 test_csv_pg

Run tests:

python -m unittest

Alternatives / inspirations

• Pandas:

  • Have to manually call Postgresql COPY command?

• Django Postgres Copy

  • Depends on Django, and force import to a Django model (cannot simply load to a temporary table)
  • No filter during import?