Supports importing files to a database using a mapping configuration.
Project description
Documentation
Installation
$ pip install csvdata-importer
CSV Data Importer
A configurable file to database table importer. It can be used to import data from a csv file to a postgres or sqlite database.
You build a mapping json file and then use that to create or update tables on the database.
Mapping file Example
{
"DATABASE_URL": "postgresql://user:pwd@host:5432/db",
"TARGET_TABLE": "quote",
"IF_EXISTS": "append",
"COLUMN_MAPPING": [
{
"COLUMN_NAME": "source",
"FILE_COLUMN_NAME": "source",
"MAPPING_TYPE": "CONSTANT",
"CONSTANT_VALUE": "test-data"
},
{
"COLUMN_NAME": "quote_value",
"FILE_COLUMN_NAME": "close",
}
]
}
Importing data using the mapping file
The overall mapping configuration
Column Name |
Example |
Description |
---|---|---|
DATABASE_URL |
postgresql://user:pass@host:5432/db |
The database we are connecting to |
TARGET_TABLE |
quote |
The table in that database we are updating or creating |
FILE_PATH |
/test.csv |
The file we are importing |
IF_EXISTS |
append |
The action taken on the table if it is non empty. Can be one of fail, append. |
COLUMN_MAPPING |
array of columns see next table |
The columns and how they are mapped |
The column mappings
Column Name |
Example |
Description |
---|---|---|
COLUMN_NAME |
quote_value |
The table column we are updating |
FILE_COLUMN_NAME |
quote |
The column in the csv we are reading from |
MAPPING_TYPE |
DIRECT |
The type of mapping we are doing. See below |
CONSTANT_VALUE |
File Import |
For use when we are evaluating or mapping a constant to the table. |
Different types of mappings
DIRECT In this case we read the value form the file and write it directly to the table row column.
CONSTANT In this case it is a constant value and we are just updating the table with the value we find in the CONSTANT_VALUE mapping.
EVALUATION Here we are evaluating some code to create the data.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for csvddata_importer-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1feba29fae835804ca448bd4cd2e19c7684c2aca8077ffbf84ae615ea7fa26c |
|
MD5 | 92b1383b6b902a1757bd0cacce14473f |
|
BLAKE2b-256 | 192067a292e37e7cc846ab441befd31949a08954f65901f97aa4158c741373a5 |