ingestr is a command-line application that ingests data from various sources and stores them in any database.
Project description
Ingest & copy data from any source to any destination without any code
Ingestr is a command-line application that allows you to ingest data from any source into any destination using simple command-line flags, no code necessary.
- ✨ copy data from your Postges / Mongo / BigQuery or any other source into any destination
- ➕ incremental loading
- 🐍 single-command installation
- 💅 Docker image for easy installation & usage
ingestr takes away the complexity of managing any backend or writing any code for ingesting data, simply run the command and watch the magic.
Installation
pip install ingestr
Quickstart
ingestr \
--source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
--source-table 'public.some_data' \
--dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
--dest-table 'ingestr.some_data'
That's it.
This command will:
- get the table
public.some_data
from the Postgres instance. - upload this data to your BigQuery warehouse under the schema
ingestr
and tablesome_data
.
Supported Sources & Destinations
Database | Source | Destination |
---|---|---|
Postgres | ✅ | ✅ |
BigQuery | ✅ | ✅ |
Snowflake | ✅ | ✅ |
Redshift | ✅ | ✅ |
Databricks | ✅ | ✅ |
DuckDB | ✅ | ✅ |
Microsoft SQL Server | ✅ | ✅ |
SQLite | ✅ | ❌ |
MySQL | ✅ | ❌ |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ingestr-0.0.3.tar.gz
(236.5 kB
view hashes)
Built Distribution
ingestr-0.0.3-py3-none-any.whl
(14.1 kB
view hashes)