Skip to main content



openfs

An S3 feature store client for data pipelines.

Python PyPi License: MIT Downloads

Openfs provides a simple api to boost the quality of your training data while keeping your data pipelines clean and manageable.

Installation

pip install openfs

Quick Start

Creating a Store

import openfs as fs
from openfs.stores import FeatureStore
from openfs.boosters import Booster
import os

# <- import files to upload here

# Connect to store bucket
fs.client.connect(
    region_name=os.environ['FSTORE_REGION'],
    endpoint_url=os.environ['FSTORE_ENDPOINT_URL'],
    access_key_id=os.environ['FSTORE_ACCESS_KEY'],
    secret_access_key=os.environ['FSTORE_SECRET_KEY']
)

# Create store
store = FeatureStore("store_name", "description of store", "some_primary_key")

# Upload store
response = store.upload(files, filenames)

Creating a Booster Dataset

# Create booster
booster = Booster(store_id=response['store_id'])

# Add features
booster.add_single("feature_1", alias="alias_for_feature")
booster.add_group(["feature_2", "feature_3"], alias="grouped_feature", how='sum')

# pull features from store (for testing)
df = booster.create_df()

# upload booster
booster.upload(name="booster_name", description="booster description")

Viewing Stores

fb.client.list_stores()



Contributors

We'd love to welcome contributors to openfs to help make training data richer and more open for everyone. We're working on our contributor docs at the moment, but if you're interested in contributing, please send us a message at contact@xplainable.io.





Thanks for trying openfs!

Made with ❤️ in Australia


© copyright xplainable pty ltd

Release files for openfs 1.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for openfs 1.0.2
File Interpreter ABI Platform
openfs-1.0.2-py3-none-any.whl Python 3 none any Details

Release files / openfs-1.0.2-py3-none-any.whl

Download URL openfs-1.0.2-py3-none-any.whl
Size 90.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a3d62cf0d21fe5e20ae470a63deab6a6a1a8998234fc90a5a77b1f88aef0ce4a
BLAKE2b-256 checksum
How to use checksums
cb5808b25ceb9fed35f849b68c8dac36db28ff31a00bd69ff38fbc61d7ff3eca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.11.0

Release history Release notifications | RSS feed

This release

1.0.2 This release

1 release file

1.0.1

1 release file

1.0.0

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page