ingestr 1.0.16

ingestr is a command-line application that ingests data from various sources and stores them in any database.

Copy data from any source to any destination without any code

---

ingestr is a command-line app that allows you to ingest data from any source into any destination using simple command-line flags, no code necessary.

• ✨ copy data from your database into any destination
• ➕ incremental loading: append, merge or delete+insert
• 🐍 single-command installation

ingestr takes away the complexity of managing any backend or writing any code for ingesting data, simply run the command and watch the data land on its destination.

Installation

You can install ingestr using the install script:

curl -LsSf https://getbruin.com/install/ingestr | sh

Alternatively, you can install it with pip:

pip install ingestr

Quickstart

ingestr ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'ingestr.some_data'

That's it.

This command:

• gets the table public.some_data from the Postgres instance.
• uploads this data to your BigQuery warehouse under the schema ingestr and table some_data.

Documentation

You can see the full documentation here.

Community

Join our Slack community here.

Contributing

Pull requests are welcome. However, please open an issue first to discuss what you would like to change. We maybe able to offer you help and feedback regarding any changes you would like to make.

[!NOTE] After cloning ingestr make sure to run make setup to install githooks.

Supported sources & destinations

	Source	Destination
Databases
BigQuery	✅	✅
ClickHouse	✅	✅
CrateDB	✅	✅
Databricks	✅	✅
IBM Db2	✅	-
DuckDB	✅	✅
DynamoDB	✅	-
Elasticsearch	✅	✅
GCP Spanner	✅	-
Local CSV file	✅	✅
Azure SQL	✅	-
Microsoft Fabric	✅	✅
Microsoft OneLake	❌	✅
Microsoft SQL Server	✅	✅
MongoDB	✅	✅
MotherDuck	✅	✅
MySQL	✅	❌
Oracle	✅	❌
Postgres	✅	✅
Redshift	✅	✅
SAP Hana	✅	❌
Snowflake	✅	✅
SQLite	✅	✅
Trino	✅	✅
Platforms
Adjust	✅	-
Airtable	✅	-
Amazon Kinesis	✅	-
Apache Kafka	✅	-
AppsFlyer	✅	-
App Store	✅	-
Asana	✅	-
Attio	✅	-
Chess.com	✅	-
Facebook Ads	✅	-
GitHub	✅	-
Gorgias	✅	-
Google Sheets	✅	-
Google Ads	✅	-
Google Analytics	✅	-
Granola	✅	-
Klaviyo	✅	-
LinkedIn Ads	✅	-
Notion	✅	-
Personio	✅	-
Phantombuster	✅	-
Pipedrive	✅	-
Azure Data Lake Storage Gen2	✅	✅
S3	✅	-
Salesforce	✅	-
Shopify	✅	-
Slack	✅	-
Smartsheets	✅	-
Solidgate	✅	-
Stripe	✅	-
TikTok Ads	✅	-
Zendesk	✅	-

Feel free to create an issue if you'd like to see support for another source or destination.

License

ingestr is source-available under the Functional Source License 1.1, with Apache 2.0 as the future license. You can use ingestr freely for internal production use, development, testing, education, research, and professional services. You cannot use ingestr to offer a competing commercial ingestion, ELT, connector, or managed data pipeline product/service.

Each version becomes Apache 2.0 two years after release.