Singer tap / Meltano extractor for CrateDB
About
A Singer tap for CrateDB, built with the Meltano SDK for custom extractors and loaders, and based on the Meltano PostgreSQL tap.
In Singer ELT jargon, a "tap" conceptually wraps a data source, where you "extract" data from.
In order to learn more about Singer, Meltano, and friends, please navigate to the Singer Intro.
Install
Usually, you will not install this package directly, but rather on behalf of a Meltano project. A corresponding snippet is outlined in the next section.
After adding it to your meltano.yml project definition file, you can install
all defined components and their dependencies with a single command.
meltano install
Usage
You can run the CrateDB Singer tap tap-cratedb by itself, or
in a pipeline using Meltano.
Meltano
Using the meltano add subcommand, you can add the plugin to your
Meltano project.
meltano add extractor tap-cratedb
NB: It will only work like this when released and registered on Meltano Hub. In the meanwhile, please add the configuration snippet manually.
CrateDB Cloud
In order to connect to CrateDB Cloud, configure the sqlalchemy_url setting
within your meltano.yml configuration file like this.
- name: tap-cratedb
namespace: cratedb
variant: cratedb
pip_url: meltano-tap-cratedb
config:
sqlalchemy_url: "crate://admin:password@example.aks1.westeurope.azure.cratedb.net:4200?ssl=true"
On localhost
In order to connect to a standalone or on-premise instance of CrateDB, configure
the sqlalchemy_url setting within your meltano.yml configuration file like this.
- name: tap-cratedb
namespace: cratedb
variant: cratedb
pip_url: meltano-tap-cratedb
config:
sqlalchemy_url: crate://crate@localhost/
Then, invoke the pipeline by using meltano run, similar like this.
meltano run tap-cratedb target-csv
Standalone
You can also invoke the adapter standalone by using the tap-cratedb program.
This example demonstrates how to export data from the database into a file.
Define the database connection string including credentials in SQLAlchemy format.
export TAP_CRATEDB_SQLALCHEMY_URL='crate://admin:password@example.aks1.westeurope.azure.cratedb.net:4200?ssl=true'
Discover all available database streams.
tap-cratedb --config ENV --discover
Export CrateDB's sys.summits table.
tap-cratedb --config ENV --catalog tests/resources/cratedb-summits.json
Development
In order to work on this adapter dialect on behalf of a real pipeline definition,
link your sandbox to a development installation of meltano-tap-cratedb, and
configure the pip_url of the component to point to a different location than the
vanilla package on PyPI.
Use this URL to directly point to a specific Git repository reference.
pip_url: git+https://github.com/crate/meltano-tap-cratedb.git@main
Use a pip-like notation to link the CrateDB Singer tap in development mode,
so you can work on it at the same time while running the pipeline, and iterating
on its definition.
pip_url: --editable=/path/to/sources/meltano-tap-cratedb
Release files for meltano-tap-cratedb 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| meltano_tap_cratedb-0.0.1.tar.gz | 16.8 kB | Without provenance | Details for meltano_tap_cratedb-0.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| meltano_tap_cratedb-0.0.1-py3-none-any.whl | Python 3 | none | any | Details for meltano_tap_cratedb-0.0.1-py3-none-any.whl |
Total release size: 23.9 kB
Release files / meltano_tap_cratedb-0.0.1.tar.gz
| Download URL | meltano_tap_cratedb-0.0.1.tar.gz |
|---|---|
| Size | 16.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
23fa555fe935ea90cdc493cd0c4018ccabf892f58ea5d32ae84f754c49b5a997
|
|
BLAKE2b-256 checksum How to use checksums |
5088544b35c5ae82283a4fc827cf2520a511c1557a7d141037a05760d42d618b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.11
|
Release files / meltano_tap_cratedb-0.0.1-py3-none-any.whl
| Download URL | meltano_tap_cratedb-0.0.1-py3-none-any.whl |
|---|---|
| Size | 7.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8b266e3685fbbf7d02ce2608d5d12eac67f313cd95f64f9b7a70ce849e3c579f
|
|
BLAKE2b-256 checksum How to use checksums |
1960e52b68845b929a49b44019b5780a0df34de7ba07c4402cea493f522ef7ff
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.11
|
Security for meltano-tap-cratedb 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
