Skip to main content

A data model based on in-memory sqlite to fetch, manipulate and push data to and from multiple sources

Project description

SQLDataModel

SQLDataModel is a fast & lightweight data model with no additional dependencies for quickly fetching and storing your tabular data to and from the most commonly used databases & data sources in a couple lines of code. It's as easy as ETL:

from SQLDataModel import SQLDataModel as sdm

# do the E part:
my_table = sdm.from_sql("your_table", cx_Oracle.Connection)

# take care of your T business:
for row in my_table.iter_rows():
    print(row)

# finish the L and be done:
my_table.to_sql("new_table", psycopg2.Connection)

Made for those times when you need to move data around but the full pandas, numpy, sqlalchemy installation is just overkill. SQLDataModel includes all the most commonly used features, including additional ones like pretty printing your table, at 1/1000 the size, 0.03MB vs 30MB

Installation

Use the package manager pip to install SQLDataModel.

pip install SQLDataModel

Usage

from SQLDataModel import SQLDataModel

# create a SQLDataModel object from any valid source, whether csv:
sdm = SQLDataModel.from_csv('region_data.csv')

# or any DB-API 2.0 connection like psycopg2, cx-oracle, pyodbc, sqlite3:
sdm = SQLDataModel.from_sql('region_data', psycopg2.Connection) 

# or from objects like dicts, lists, tuples, iterables:
sdm = SQLDataModel.from_data(data=region_data, headers=region_headers)

# manipulate it
sdm_sliced = sdm[2:7]

# view it as a table
print(sdm)
sdm_colorful_table
# group by columns:
sdm_group = sdm.group_by('region','check')
print(sdm_group)
sdm_grouped_table
# loop through it:
for row in sdm.iter_rows():
    print(row)

# or save it for later as csv:
sdm.to_csv('region_data.csv')

# or to sqlite database:
sdm.to_sql('data', sqlite3.Connection)

# and get it back again as a new model:
sdm_new = SQLDataModel.from_sql('select * from data', sqlite3.Connection)

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

Thank you!
Ante Tonkovic-Capin

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

SQLDataModel-0.1.3.tar.gz (12.1 kB view hashes)

Uploaded Source

Built Distribution

SQLDataModel-0.1.3-py3-none-any.whl (18.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page