Skip to main content

Flytekit OmegaConf Plugin

Flytekit python natively supports serialization of many data types for exchanging information between tasks. The Flytekit OmegaConf Plugin extends these by the DictConfig type from the OmegaConf package as well as related types that are being used by the hydra package for configuration management.

Task example

from dataclasses import dataclass
import flytekitplugins.omegaconf  # noqa F401
from flytekit import task, workflow
from omegaconf import DictConfig

@dataclass
class MySimpleConf:
    _target_: str = "lightning_module.MyEncoderModule"
    learning_rate: float = 0.0001

@task
def my_task(cfg: DictConfig) -> None:
    print(f"Doing things with {cfg.learning_rate=}")


@workflow
def pipeline(cfg: DictConfig) -> None:
    my_task(cfg=cfg)


if __name__ == "__main__":
    from omegaconf import OmegaConf

    cfg = OmegaConf.structured(MySimpleConf)
    pipeline(cfg=cfg)

Transformer configuration

The transformer can be set to one of three modes:

Dataclass - This mode should be used with a StructuredConfig and will reconstruct the config from the matching dataclass during deserialisation in order to make typing information from the dataclass and continued validation thereof available. This requires the dataclass definition to be available via python import in the Flyte execution environment in which objects are (de-)serialised.

DictConfig - This mode will deserialize the config into a DictConfig object. In particular, dataclasses are translated into DictConfig objects and only primitive types are being checked. The definition of underlying dataclasses for structured configs is only required during the initial serialization for this mode.

Auto - This mode will try to deserialize according to the Dataclass mode and fall back to the DictConfig mode if the dataclass definition is not available. This is the default mode.

You can set the transformer mode globally or for the current context only the following ways:

from flytekitplugins.omegaconf import set_transformer_mode, set_local_transformer_mode, OmegaConfTransformerMode

# Set the global transformer mode using the new function
set_transformer_mode(OmegaConfTransformerMode.DictConfig)

# You can also the mode for the current context only
with set_local_transformer_mode(OmegaConfTransformerMode.Dataclass):
    # This will use the Dataclass mode
    pass
Since the DictConfig is flattened and keys transformed into dot notation, the keys of the DictConfig must not contain
dots.

Metadata

Release files for flytekitplugins-omegaconf 1.16.31

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flytekitplugins-omegaconf 1.16.31
File Size Uploaded Provenance Actions
flytekitplugins_omegaconf-1.16.31.tar.gz 12.3 kB Without provenance Details for flytekitplugins_omegaconf-1.16.31.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for flytekitplugins-omegaconf 1.16.31
File Interpreter ABI Platform Actions
flytekitplugins_omegaconf-1.16.31-py3-none-any.whl Python 3 none any Details for flytekitplugins_omegaconf-1.16.31-py3-none-any.whl

Total release size: 23.1 kB

Release files / flytekitplugins_omegaconf-1.16.31.tar.gz

Download URL flytekitplugins_omegaconf-1.16.31.tar.gz
Size 12.3 kB
Tags Source
SHA-256 checksum
How to use checksums
129f9a0e76bd7f599680a0abfb67b156c98532af597cdd8ee523c9e3b1700ba5
BLAKE2b-256 checksum
How to use checksums
ff4377d8a66d9cfec5e6f3e20ac2cf85d9aab9e72bc906abd4a16e2c744c62bc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.15

Release files / flytekitplugins_omegaconf-1.16.31-py3-none-any.whl

Download URL flytekitplugins_omegaconf-1.16.31-py3-none-any.whl
Size 10.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
fca320cd631bcba80223d1a84981e0562fea3610d6cd119a50540c42b41f1ae7
BLAKE2b-256 checksum
How to use checksums
ab9d93fb640ca756fc1f3230c8866e809d55996ee7d57600e4c1808e31250de8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.15

Security for flytekitplugins-omegaconf 1.16.31

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.16.31 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page