Spock is a framework designed to help manage complex parameter configurations for Python applications
Project description
Managing complex configurations any other way would be highly illogical...
About
spock
is a framework that helps manage complex parameter configurations during research and development of Python
applications. spock
lets you focus on the code you need to write instead of re-implementing boilerplate code like
creating ArgParsers, reading configuration files, implementing traceability etc.
In short, spock
configurations are defined by simple and familiar class-based structures. This allows spock
to
support inheritance, read from multiple markdown formats, automatically generate cmd-line arguments, and allow
hierarchical configuration by composition.
Key Features
- Simple Declaration: Type checked parameters are
defined within a
@spock
decorated class. Supports required/optional and automatic defaults. - Easily Managed Parameter Groups: Each class automatically generates its own object within a single namespace.
- Parameter Inheritance: Classes support inheritance allowing for complex configurations derived from a common base set of parameters.
- Complex Types: Nested Lists/Tuples,
List/Tuples of Enum of
@spock
classes, List of repeated@spock
classes - Multiple Configuration File Types: Configurations are specified from YAML, TOML, or JSON files.
- Hierarchical Configuration: Compose from multiple configuration files via simple include statements.
- Command-Line Overrides: Quickly experiment by overriding a value with automatically generated command line arguments.
- Immutable: All classes are frozen preventing any misuse or accidental overwrites (to the extent they can be in Python).
- Tractability and Reproducibility: Save runtime parameter configuration to YAML, TOML, or JSON with a single chained command (with extra runtime info such as Git info, Python version, machine FQDN, etc). The saved markdown file can be used as the configuration input to reproduce prior runtime configurations.
- Hyper-Parameter Tuner Addon: Provides a unified
interface for defining hyper-parameters (via
@spockTuner
decorator) that supports various tuning/algorithm backends (currently: Optuna, Ax) - S3 Addon: Automatically detects
s3://
URI(s) and handles loading and savingspock
configuration files when an activeboto3.Session
is passed in (plus any additionalS3Transfer
configurations)
Quick Install
The basic install and [s3]
extension require Python 3.6+ while the [tune]
extension requires Python 3.7+
Base | w/ S3 Extension | w/ Hyper-Parameter Tuner |
---|---|---|
pip install spock-config |
pip install spock-config[s3] |
pip install spock-config[tune] |
Quick Start & Documentation
Refer to the quick-start guide here.
Current documentation and more information can be found here.
Example spock
usage is located here.
News/Releases
See Releases for more information.
December 14, 2021
- Refactored the backend to better handle nested dependencies (and for clarity)
- Refactored the docs to use Docusaurus
August 17, 2021
- Added hyper-parameter tuning backend support for Ax via Service API
July 21, 2021
- Added hyper-parameter tuning support with
pip install spock-config[tune]
- Hyper-parameter tuning backend support for Optuna define-and-run API (WIP for Ax)
Original Implementation
Nicholas Cilfone, Siddharth Narayanan
spock
was originally developed by the Artificial Intelligence Center of Excellence at Fidelity Investments.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for spock_config-2.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d553642f184859b86532952f09d8faf8720caf6990b78bab81ea82c2c2172f9 |
|
MD5 | c5c3cf7e27d2385384af2e4f7d34f19a |
|
BLAKE2b-256 | 20ae1c542d701533c6a7335543e93b33303324179d457c5f01600770dc2c245e |