An library used within Acryl Agents to execute tasks
Project description
Acryl Executor
Remote execution agent used for running DataHub tasks, such as ingestion powered through the UI.
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip3 install .
Notes
By default, this library comes with a set of default task implementations:
RUN_INGEST Task
-
SubprocessProcessIngestionTask - Executes a metadata ingestion run by spinning off a subprocess. Supports ingesting from a particular version, and with a specific plugin (based on the platform type requested)
-
InMemoryIngestionTask - Executes a metadata ingestion run using the datahub library in the same process. Not great for production where we can see strange dependency conflicts when certain packages are executed together. Use this for testing, as it has no ability to check out a specific DataHub Metadata Ingestion plugin.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for acryl_executor-0.0.3.14-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea946cced9d1a9dd3d10f89f5b90498b82f948f25c7866eefd2a144b2c534f6f |
|
MD5 | 0f82f84061f3b751fdf7643c62f6f0bb |
|
BLAKE2b-256 | 3d8f0167258d5299a5ccdf02ec1ae000ab130d6cf70574c7d49c23e2ba9ffaf0 |