sqlalchemy-csv-normalise
SQLAlchemy utilities for normalising / denormalising table data, useful for CSV
Where a table is normalised to have “lookup tables” of values referred to by e.g. a numeric foreign-key ID, these functions will enable extraction of the data (or conversely, loading from such) with the looked-up values substituted in. Among other things, this allows more human-friendly data editing in e.g. a spreadsheet.
Example:
from sqlalchemy_csv_normalise import denormalise_prepare
q, col_names = denormalise_prepare(db.session, table)
filename = table_to_filename(table)
with open(filename, 'w', newline='') as csv_file:
csv_file_writer = csv.writer(csv_file)
csv_file_writer.writerow(col_names)
csv_file_writer.writerows(q.all())
from sqlalchemy_csv_normalise import renormalise_prepare, empty_deleter,\
type_coercer
row_maker = renormalise_prepare(db.session, table)
row_cleaner = empty_deleter(table)
row_coercer = type_coercer(table)
filename = table_to_filename(table)
with open(filename, newline='') as csv_file:
for d in csv.DictReader(csv_file):
row = row_coercer(row_cleaner(row_maker(d)))
db.session.add(table(**row))
db.session.commit()
Free software: MIT license
Documentation: https://sqlalchemy-csv-normalise.readthedocs.io.
Features
denormalise_prepare(session, table, colname_tidier)
Returns SQLAlchemy query, and the column-names it will return. The query will denormalise any foreign keys (FKs) if they refer to a table with a unique column that is not its primary key.
The names of any FK columns will have _id taken off the end as a simple heuristic. Override this by providing a colname_tidier.
empty_deleter(table)
Returns function that returns given dict minus empty strings for nullable columns. Useful because CSV has no way to record NULL.
type_coercer(table)
Returns function that given a row dict will coerce values. Works on dates and booleans. Will only operate on strings, so if you have pass in a row that has already got non-string values, they will not be affected.
renormalise_prepare(session, table, colname_tidier)
Returns function that will renormalise given dictionary Does the inverse of denormalise_prepare.
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.1 (2020-02-01)
First release on PyPI.
0.1.2 (2020-02-01)
make type_coercer only affect string values
Release files for sqlalchemy-csv-normalise 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sqlalchemy-csv-normalise-0.1.2.tar.gz | 11.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Total release size: 17.4 kB
Release files / sqlalchemy-csv-normalise-0.1.2.tar.gz
| Download URL | sqlalchemy-csv-normalise-0.1.2.tar.gz |
|---|---|
| Size | 11.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b6b90762ec8d0f7a2a0be714c7b0ec37bccd7f9288951c3a3b98531adddc80a6
|
|
BLAKE2b-256 checksum How to use checksums |
4dae13a73d0c0f09482e27432a933599c77f73446665244da62819ed8c2dd069
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.9
|
Release files / sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl
| Download URL | sqlalchemy_csv_normalise-0.1.2-py2.py3-none-any.whl |
|---|---|
| Size | 6.2 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
3c479a22ca5a4149c7f8606ef16a71d40948bc63d136441fa9575f40b00ae11b
|
|
BLAKE2b-256 checksum How to use checksums |
0bc2df068dd89c2eb336a8beca39bb6c5bf2ff86d8ede59905a8f81fd2e9667e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.9
|