Airflow Toloka Provider
This library allows you to run crowdsourcing Toloka processes in Apache Airflow - a widely used workflow management system
Here you can find a collection of ready-made Airflow tasks for the most frequently used actions in Toloka-Kit.
Getting started
$ pip install airflow-provider-toloka
A good way to start is to follow the example in this repo.
TolokaHook
TolokaHook is used for getting toloka OAuth token and creating TolokaClient with it.
You can get TolokaClient from TolokaHook by calling get_conn() method.
To make an appropriate Airflow Connection you need to create it in the Airflow Connections UI with following parameters:
Conn ID:toloka_defaultConn Type:TolokaToken: enter your OAuth token for Toloka. You can learn more about how to get it here.Environment: enterproductionorsandbox
Tasks use the toloka_default connection id by default, but
if needed, you can create additional Airflow Connections and reference them
as the function toloka_conn_id argument.
Tasks and Sensors
There are several tasks and sensors that give you easy way to interact with Toloka from Airflow DAGs.
Creating a project and a pool, adding tasks and getting assignments are among them.
You can easily create your own task using TolokaHook if it is beyond the scope of implemented ones.
And it would be nice to have your pull request with updates.
Check out our example to see tasks and sensors in the battlefield.
Useful Links
Questions and bug reports
- For reporting bugs please use the Toloka/bugreport page.
- Join our English-speaking slack community for both tech and abstract questions.
License
© YANDEX LLC, 2022. Licensed under the Apache License, Version 2.0. See LICENSE file for more details.
Metadata
Release files for airflow-provider-toloka 0.0.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| airflow-provider-toloka-0.0.8.tar.gz | 9.0 kB | Without provenance | Details for airflow-provider-toloka-0.0.8.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| airflow_provider_toloka-0.0.8-py3-none-any.whl | Python 3 | none | any | Details for airflow_provider_toloka-0.0.8-py3-none-any.whl |
Total release size: 19.6 kB
Release files / airflow-provider-toloka-0.0.8.tar.gz
| Download URL | airflow-provider-toloka-0.0.8.tar.gz |
|---|---|
| Size | 9.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a22e5c159fe7f4411f5ed358e2a9c3acb930ceeb63d0c3955c1199d3603a2b4b
|
|
BLAKE2b-256 checksum How to use checksums |
66c22aa5469803870b07c8bcd80047eab12e64967e9cf6e3cfe21c361b8012f8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.8.13
|
Release files / airflow_provider_toloka-0.0.8-py3-none-any.whl
| Download URL | airflow_provider_toloka-0.0.8-py3-none-any.whl |
|---|---|
| Size | 10.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c9337eb7d991b77fb101bc25ce1025d3619492ef1a0418ece579eaf35c077d0e
|
|
BLAKE2b-256 checksum How to use checksums |
66a8ad09b84ad78b08b5c886272eaef48bc55340f82a9921cd025f4da8284a82
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.8.13
|
Security for airflow-provider-toloka 0.0.8
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.