Feature Store for the Daipe AI Platform
Project description
Feature Store bundle
This package is distributed under the "DataSentics SW packages Terms of Use." See license
Feature store bundle allows you to store features with metadata.
Installation
poetry add feature-store-bundle
Getting started
- Define entity and custom
feature decorator
from featurestorebundle.entity.getter import get_entity
from featurestorebundle.feature.FeaturesStorage import FeaturesStorage
from featurestorebundle.notebook.decorator import feature_decorator_factory
entity = get_entity()
features_storage = FeaturesStorage(entity)
feature_decorator = feature_decorator_factory.create(entity, features_storage)
- Use the
feature decorator
to save features as you create them
import daipe as dp
from pyspark.sql import functions as f
from pyspark.sql import DataFrame
@dp.transformation(dp.read_table("silver.tbl_loans"), display=True)
@feature_decorator(
("Age", "Client's age"),
("Gender", "Client's gender"),
("WorkExperience", "Client's work experience"),
category="personal",
)
def client_personal_features(df: DataFrame):
return (
df.select("UserName", "Age", "Gender", "WorkExperience")
.groupBy("UserName")
.agg(
f.max("Age").alias("Age"),
f.first("Gender").alias("Gender"),
f.first("WorkExperience").alias("WorkExperience"),
)
.withColumn("timestamp", f.lit(today))
)
- Write/Merge all features in one go
import daipe as dp
from featurestorebundle.feature.writer.FeaturesWriter import FeaturesWriter
@dp.notebook_function()
def write_features(writer: FeaturesWriter):
writer.write(features_storage)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for feature-store-bundle-2.3.0.dev1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4c341d6d459270b3907043d1a17b2df979b0ab2b8be697f14ee5fe15572391bd |
|
MD5 | 402ecedc3be3f897ce1a5d75ffeb2fdc |
|
BLAKE2b-256 | df6026b6dbf8b1940c935cfcb16e5f752d63d6434391d7d2fc32aa6b0aa08ce8 |
Close
Hashes for feature_store_bundle-2.3.0.dev1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f10abbd0d63d9579a41805449e7547c55164664c2638ad40eec283e304f27c86 |
|
MD5 | d806d92f418a5776e5367aaa3e93da40 |
|
BLAKE2b-256 | 754df6f85d646254138207e6b890d673a6ec1170818a276edfed6ecba63ad7fb |