Skip to main content

auto_sql is a memory aware csv to sqlite converter.

Project description

auto_sql

Build Status

==Work in progress==

auto_sql is very early in development, more features to come

Description

auto_sql is a memory aware csv to sqlite converter capable of converting multi-gigabyte tabular files to sqlite databases on low memory machines. auto_sql focuses on speed by enabling multi-processing on multi-core machines.

Installation

$pip install auto_sql

Usage Case

from auto_sql import AutoSql

tab_obj = AutoSql(file='file.csv',
                        db_name='database',
                        sep='\t',
                        out_dir=".")

if __name__ == "__main__":
    tab_obj.run()

Currently auto_sql only supports csv's with headers

Dependencies

  • Python 3.4, 3.5, or 3.6

  • pandas==0.21.1

  • psutil==5.4.7

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

auto_sql-0.0.2.tar.gz (4.3 kB view details)

Uploaded Source

File details

Details for the file auto_sql-0.0.2.tar.gz.

File metadata

  • Download URL: auto_sql-0.0.2.tar.gz
  • Upload date:
  • Size: 4.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.5

File hashes

Hashes for auto_sql-0.0.2.tar.gz
Algorithm Hash digest
SHA256 5621b07eb49aa1ca92d883d57f59ede0f14b79f089489b4326b1ea7e7980bb6c
MD5 8c4226482dd710c2b066df3efae13690
BLAKE2b-256 fd1e82317566d4187c1ab52243b74b609b7b74ce71be59886c1515be6c21eff9

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page