HyFI: Hydra Fast Interface
HyFI, short for Hydra Fast Interface, is a powerful Python framework built atop the foundations of Hydra and Pydantic. Its main objective is to provide a streamlined interface for configuring, structuring, executing, and scaling Python applications and workflows.
Overview
In the modern world of software and data science, creating scalable, reproducible, and modular Python workflows is vital. HyFI is here to make that process not just feasible but also efficient and straightforward. It has been meticulously crafted to help developers and researchers build pipeline-oriented projects and promote shareable and reproducible workflows.
Core Features
1. Dynamic Configuration Management
- Modular Configuration: Harness the power of Hydra and Pydantic to manage your application configurations dynamically and modularity.
- Seamless Command Line Overrides: Experiment swiftly without the clutter of multiple similar configuration files.
- Jupyter Notebook Integration: The HyFI class allows for easy composition of configurations even in a jupyter notebook setting.
2. Zero Boilerplate
Stay focused on your core problems. HyFI takes care of the usual overheads such as command line flags, loading configuration files, and logging.
3. Structured Workflows
-
Configurable Automated Processes: Divide your research into unit jobs or tasks, and then bundle them into workflows.
-
Versatility: Have the liberty to create multiple workflows, with each performing varied sets of tasks.
4. Reproducibility and Sharing
-
Sharing Capability: Share your datasets, models, and configurations effortlessly.
-
Reproducibility: Sharing configurations alongside datasets and models ensures that every piece of research can be replicated.
-
Granular or Holistic: Choose to share individual unit jobs or an entire workflow, as per your need.
5. Support for Plugins
- Integration of HyFI-based Applications: Enhance your project by plugging in several HyFI-based applications. This allows you to leverage the functionalities of plugins along with their specific configuration files.
- Modular Design Enablement: The plugin support fosters a truly modular design approach, enabling seamless integration of various components and features, thereby promoting greater flexibility and extensibility in your projects.
Other Noteworthy Features:
- Workspace Management: Automatic workspace creation and environment variable management.
- Task Pipelining: Chain and parallelize jobs and steps seamlessly.
- Optimized Parallelism: Efficient job batching with the help of Joblib.
- Dotenv Integration: Manage configurations via .env files with ease.
- Built-in Commands: Simplify processes like initializing projects, running pipelines, and copying files.
- Extensibility: Easily extend HyFI's capabilities by crafting new Config classes and utilizing the building blocks it offers.
- Command Line Interface (CLI): Access a suite of commands and workflows directly from the command line.
- Advanced Logging: Make use of the integrated logging configurations, complemented by Hydra logging.
- Robust Caching: Cache both files and data efficiently.
Citation
@software{lee_2023_8247719,
author = {Young Joon Lee},
title = {HyFI: Hydra Fast Interface},
month = aug,
year = 2023,
publisher = {Zenodo},
doi = {10.5281/zenodo.8247719},
url = {https://doi.org/10.5281/zenodo.8247719}
}
@software{lee_2023_hyfi,
author = {Young Joon Lee},
title = {HyFI: Hydra Fast Interface},
year = 2023,
publisher = {GitHub},
url = {https://github.com/entelecheia/hyfi}
}
Changelog
See the CHANGELOG for more information.
Contributing
Contributions are welcome! Please see the contributing guidelines for more information.
License
This project is released under the MIT License.
1 maintainer
Metadata
1 maintainer
Release files for hyfi 1.37.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hyfi-1.37.0.tar.gz | 111.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| hyfi-1.37.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 271.9 kB
Release files / hyfi-1.37.0.tar.gz
| Download URL | hyfi-1.37.0.tar.gz |
|---|---|
| Size | 111.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3a66244690945d20110a40542c957d2f9694560f427ab04aea64c97b3d6bdc08
|
|
BLAKE2b-256 checksum How to use checksums |
4fedf7684b33a5fde57b3246843fc68724975e272660314dbc79947ab58abc01
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.11.2 readme-renderer/44.0 requests/2.32.3 requests-toolbelt/1.0.0 urllib3/2.2.3 tqdm/4.66.5 importlib-metadata/8.5.0 keyring/25.4.1 rfc3986/2.0.0 colorama/0.4.6 CPython/3.10.12
|
Release files / hyfi-1.37.0-py3-none-any.whl
| Download URL | hyfi-1.37.0-py3-none-any.whl |
|---|---|
| Size | 160.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fe509d7fd4144346af0e1f516850b0f4e06c0ad05cef57fa22e0c912bb6e6994
|
|
BLAKE2b-256 checksum How to use checksums |
b11022642695f0b714759739d048aeb1360e8940dcfbeacbf9861f896ad9a187
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.11.2 readme-renderer/44.0 requests/2.32.3 requests-toolbelt/1.0.0 urllib3/2.2.3 tqdm/4.66.5 importlib-metadata/8.5.0 keyring/25.4.1 rfc3986/2.0.0 colorama/0.4.6 CPython/3.10.12
|
