Feast Spark

Contains

Spark ingestion jobs for Feast versions 0.9 and below
Feast Job Service
Feast Python SDK Spark extensions

Usage:

import feast_spark
import feast

client = feast.Client()

client.set_project("project1")
entity = feast.Entity(
    name="driver_car_id",
    description="Car driver id",
    value_type=ValueType.STRING,
    labels={"team": "matchmaking"},
)

# Create Feature Tables using Feast SDK
batch_source = feast.FileSource(
    file_format=ParquetFormat(),
    file_url="file://feast/*",
    event_timestamp_column="ts_col",
    created_timestamp_column="timestamp",
    date_partition_column="date_partition_col",
)

stream_source = feast.KafkaSource(
    bootstrap_servers="localhost:9094",
    message_format=ProtoFormat("class.path"),
    topic="test_topic",
    event_timestamp_column="ts_col",
)

ft = feast.FeatureTable(
    name="my-feature-table-1",
    features=[
        Feature(name="fs1-my-feature-1", dtype=ValueType.INT64),
        Feature(name="fs1-my-feature-2", dtype=ValueType.STRING),
        Feature(name="fs1-my-feature-3", dtype=ValueType.STRING_LIST),
        Feature(name="fs1-my-feature-4", dtype=ValueType.BYTES_LIST),
    ],
    entities=["fs1-my-entity-1"],
    labels={"team": "matchmaking"},
    batch_source=batch_source,
    stream_source=stream_source,
)

# Register objects in Feast
client.apply(entity, ft)

# Start spark streaming ingestion job that reads from kafka and writes to the online store
feast_spark.Client(client).start_stream_to_online_ingestion(ft)

Release files for feast-spark 0.2.37

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for feast-spark 0.2.37
File	Size	Uploaded
feast-spark-0.2.37.tar.gz	62.1 kB	Jul 8, 2022	Details

Built distribution (wheel)

Table of built distributions (wheels) for feast-spark 0.2.37
File	Interpreter	ABI	Platform	Reset
feast_spark-0.2.37-py3-none-any.whl 88.6 kB Jul 8, 2022	Python 3	none	any	Details

Total release size: 150.7 kB

Release files / feast-spark-0.2.37.tar.gz

Download URL	feast-spark-0.2.37.tar.gz
Size	62.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`8d1d6690693b5e19140b8b2641a495178e02c6bc184938e46770fc2dafdfccb9`
BLAKE2b-256 checksum How to use checksums	`927379d254cb4a8b57db935ba56fffd7b9edf9086490f1bc0009f2383a3eddbf`
Upload date	Jul 8, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.1 CPython/3.8.13`

Release files / feast_spark-0.2.37-py3-none-any.whl

Download URL	feast_spark-0.2.37-py3-none-any.whl
Size	88.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`84a520aea39d42802ab44ff24aa17d0f62abbb9997e8858895def56a53ff78d8`
BLAKE2b-256 checksum How to use checksums	`591fd5fb360461b472ddaf78ff7ce5ba94ed1d83574080911b84908800621e88`
Upload date	Jul 8, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.1 CPython/3.8.13`