feast-spark-offline-store 0.0.4

Spark support for Feast offline store

Feast Spark Offline Store plugin

This repo contains a plugin for feast to run an offline store on Spark. It can be installed from pip and configured in the feature_store.yaml configuration file to interface with DataSources using Spark.

Contributing

We strongly encourage you to contribute to our repository. Find out more in our contribution guidelines

Requirements

Currently only supports Feast version 0.14.1.

Installation

pip install feast-spark-offline-store

Or to install from source:

git clone [e-mail]:Adyen/feast-spark-offline-store.git
cd feast-spark-offline-store
pip install -e '.[dev]'

Usage

Install feast and feast_spark_offline_store and change the Feast configurations in feature_store.yaml to use feast_spark_offline_store.SparkOfflineStore:

project: example_feature_repo
registry: data/registry.db
provider: local
online_store:
    ...
offline_store:
    type: feast_spark_offline_store.spark.SparkOfflineStore
    spark_conf:
        spark.master: "local[*]"
        spark.ui.enabled: "false"
        spark.eventLog.enabled: "false"
        spark.sql.catalogImplementation: "hive"
        spark.sql.parser.quotedRegexColumnNames: "true"
        spark.sql.session.timeZone: "UTC"

Documentation

See Feast documentation on offline stores and adding custom offline stores.

Support

If you have a feature request, or spotted a bug or a technical problem, create a GitHub issue. For other questions, contact our support team.

License

MIT license. For more information, see the LICENSE file.