DaggerML
Project description
Dagger-ML Python Library
Prerequisites
pipx
If pipx is not installed, first do that.
hatch
Then install hatch via: pipx install hatch
.
Configuration
# help
python -m daggerml --help
# configure global API endpoint
python -m daggerml configure \
--global \
--profile ${DML_PROFILE} \ # optional
--api-endpoint https://api.${DML_ZONE}-${AWS_REGION}.${DOMAIN}
# configure API key
python -m daggerml login \
--profile ${DML_PROFILE} \ # optional
--username ${USERNAME}
# configure group ID for local project
python -m daggerml configure --group-id ${DML_GROUP_ID}
# bash command completion
pip install argcomplete
# enable completion for dml in this shell
eval "$(register-python-argcomplete dml)"
# enable completion for dml via bashrc
cat <<'EOT' >> ~/.bashrc
eval "$(register-python-argcomplete dml)"
EOT
# or see: https://kislyuk.github.io/argcomplete/#activating-global-completion
Usage
You currently need AWS_DEFAULT_REGION
and DML_ZONE
environment variables
set. Then you can run python bootstrap-docker.py
, for instance.
bootstrap-docker.py
sets up the docker-build func, so you can now run docker
stuff in your dags (e.g. as we do in the docs/examples/ directory).
Run Locally
# Start local postgres:
sudo systemctl start postgresql
# Connect to local postgres:
psql -h localhost postgres postgres
# Start local DML API server:
python infra/lib/api/server.py
# Run dag locally
DML_LOCAL_DB=1 python mydag.py
Docs
To build the docs, first make sure bootstrap-docker.py
has been run, then
run: hatch run docs:build
To serve the docs: hatch run docs:serve
Tests
To run the tests: hatch run test:cov
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
daggerml-0.0.10.tar.gz
(10.4 kB
view hashes)
Built Distribution
daggerml-0.0.10-py3-none-any.whl
(10.5 kB
view hashes)
Close
Hashes for daggerml-0.0.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f15df3064603db844365cc417c87f143aa784dd12f1054b2a5b20843ee74ec2 |
|
MD5 | 38a98eb456554e63c3a9a1b803e918c6 |
|
BLAKE2b-256 | 30c9f3dbd96ea7cf2625389cbb071704e9f472de2f1423b447ca164e265cf40c |