Archived
This project has been archived by its maintainers, and is no longer receiving any updates.
DQM-ML Pipeline
This package provides the orchestration engine for DQM-ML V2. It handles the lifecycle of data processing, from loading to metric computation and output writing.
Key Components
DatasetPipeline
The main orchestrator that:
- Loads the configuration.
- Discovers plugins via entry points.
- Executes the streaming loop.
- Manages memory and I/O efficiency.
Protocols
DataLoader: A factory for creating data selections (e.g., Parquet, CSV loaders).DataSelection: Represents a specific subset of data and provides an iterator over batches.OutputWriter: Persists computed features or metrics to disk.
Built-in Loaders
parquet: Optimized loading using PyArrow.csv: Flexible loading using Pandas.
1 maintainer
Metadata
1 maintainer
Release files for dqm-ml-pipeline 1.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dqm_ml_pipeline-1.1.5.tar.gz | 11.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dqm_ml_pipeline-1.1.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 24.9 kB
Release files / dqm_ml_pipeline-1.1.5.tar.gz
| Download URL | dqm_ml_pipeline-1.1.5.tar.gz |
|---|---|
| Size | 11.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f6e4fc499b3efb3350ee17328e75c65ff74e242556c8e3a79f2e8efa3a223e98
|
|
BLAKE2b-256 checksum How to use checksums |
8629cc63ed29954d8fa7790eaa813f38cd6d31e8105143bfcfd7095924219225
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.28 {"installer":{"name":"uv","version":"0.9.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / dqm_ml_pipeline-1.1.5-py3-none-any.whl
| Download URL | dqm_ml_pipeline-1.1.5-py3-none-any.whl |
|---|---|
| Size | 13.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b08b42043d6200bcac75b43fc2a18e99621dc9eb4d8db31267532b25fad38d5d
|
|
BLAKE2b-256 checksum How to use checksums |
fd47eaf47017b9ba47a1de5454e85c6bc4856c0bbd3bc3db912d3bdcfba574c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.28 {"installer":{"name":"uv","version":"0.9.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
