Skip to main content

sqlalchemy-csv-normalise

https://img.shields.io/pypi/v/sqlalchemy-csv-normalise.svg https://travis-ci.com/mohawk2/sqlalchemy-csv-normalise.svg?branch=master Documentation Status Updates

SQLAlchemy utilities for normalising / denormalising table data, useful for CSV

Where a table is normalised to have “lookup tables” of values referred to by e.g. a numeric foreign-key ID, these functions will enable extraction of the data (or conversely, loading from such) with the looked-up values substituted in. Among other things, this allows more human-friendly data editing in e.g. a spreadsheet.

Example:

from sqlalchemy_csv_normalise import denormalise_prepare
q, col_names = denormalise_prepare(db.session, table)
filename = table_to_filename(table)
with open(filename, 'w', newline='') as csv_file:
    csv_file_writer = csv.writer(csv_file)
    csv_file_writer.writerow(col_names)
    csv_file_writer.writerows(q.all())

from sqlalchemy_csv_normalise import renormalise_prepare, empty_deleter,\
    type_coercer
row_maker = renormalise_prepare(db.session, table)
row_cleaner = empty_deleter(table)
row_coercer = type_coercer(table)
filename = table_to_filename(table)
with open(filename, newline='') as csv_file:
    for d in csv.DictReader(csv_file):
        row = row_coercer(row_cleaner(row_maker(d)))
        db.session.add(table(**row))
db.session.commit()

Features

  • denormalise_prepare(session, table, colname_tidier)

Returns SQLAlchemy query, and the column-names it will return. The query will denormalise any foreign keys (FKs) if they refer to a table with a unique column that is not its primary key.

The names of any FK columns will have _id taken off the end as a simple heuristic. Override this by providing a colname_tidier.

  • empty_deleter(table)

Returns function that returns given dict minus empty strings for nullable columns. Useful because CSV has no way to record NULL.

  • type_coercer(table)

Returns function that given a row dict will coerce values. Works on dates and booleans. Will only operate on strings, so if you have pass in a row that has already got non-string values, they will not be affected.

  • renormalise_prepare(session, table, colname_tidier)

Returns function that will renormalise given dictionary Does the inverse of denormalise_prepare.

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.1.1 (2020-02-01)

  • First release on PyPI.

0.1.2 (2020-02-01)

  • make type_coercer only affect string values

Release files for sqlalchemy-csv-normalise 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sqlalchemy-csv-normalise 0.1.2
File Size Uploaded
sqlalchemy-csv-normalise-0.1.2.tar.gz 11.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for sqlalchemy-csv-normalise 0.1.2
File Interpreter ABI Platform
sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl Python 3, Python 2 none any Details

Total release size: 17.4 kB

Release files / sqlalchemy-csv-normalise-0.1.2.tar.gz

Download URL sqlalchemy-csv-normalise-0.1.2.tar.gz
Size 11.2 kB
Tags Source
SHA-256 checksum
How to use checksums
b6b90762ec8d0f7a2a0be714c7b0ec37bccd7f9288951c3a3b98531adddc80a6
BLAKE2b-256 checksum
How to use checksums
4dae13a73d0c0f09482e27432a933599c77f73446665244da62819ed8c2dd069
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.9

Release files / sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl

Download URL sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl
Size 6.2 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
3c479a22ca5a4149c7f8606ef16a71d40948bc63d136441fa9575f40b00ae11b
BLAKE2b-256 checksum
How to use checksums
0bc2df068dd89c2eb336a8beca39bb6c5bf2ff86d8ede59905a8f81fd2e9667e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.9

Release history Release notifications | RSS feed

This release

0.1.2 This release

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page