Haystack ML Stack
Currently this project contains a FastAPI-based service designed for low-latency scoring of streams data coming from http requests
🚀 Features
- FastAPI Service: Lightweight and fast web service for ML inference.
- Asynchronous I/O: Utilizes
aiobotocorefor non-blocking S3 and DynamoDB operations. - Model Loading: Downloads and loads the ML model (using
cloudpickle) from a configurable S3 path on startup. - Feature Caching: Implements a thread-safe Time-To-Live (TTL) / Least-Recently-Used (LRU) cache (
cachetools.TLRUCache) for DynamoDB features, reducing latency and database load. - DynamoDB Integration: Fetches stream-specific features from DynamoDB to enrich the data before scoring.
- Health Check: Provides a
/healthendpoint to monitor service status and model loading.
- Asynchronous I/O: Utilizes
📦 Installation
This project requires Python 3.11 or later.
-
Install package: The dependencies associated are listed in
pyproject.toml.pip install haystack-ml-stack
⚙️ Configuration
The service is configured using environment variables, managed by pydantic-settings. You can use a .env file for local development.
| Variable Name | Alias | Default | Description |
|---|---|---|---|
S3_MODEL_PATH |
S3_MODEL_PATH |
None |
Required. The s3://bucket/key URL for the cloudpickled ML model file. |
FEATURES_TABLE |
FEATURES_TABLE |
"features" |
Name of the DynamoDB table storing stream features. |
LOGS_FRACTION |
LOGS_FRACTION |
0.01 |
Fraction of requests to log detailed stream data for sampling/debugging (0.0 to 1.0). |
CACHE_MAXSIZE |
(none) | 50000 |
Maximum size of the in-memory feature cache. |
Example env vars
S3_MODEL_PATH="s3://my-ml-models/stream-scorer/latest.pkl"
FEATURES_TABLE="features"
LOGS_FRACTION=0.05
🌐 Endpoints
| Method | Path | Description |
|---|---|---|
| GET | / |
Root endpoint, returns a simple running message. |
| GET | /health |
Checks if the service is running and if the ML model has been loaded. |
| POST | /score |
Main scoring endpoint. Accepts stream data and returns model predictions. |
💻 Technical Details
Model Structure
The ML model file downloaded from S3 is expected to be a cloudpickle-serialized Python dictionary with the following structure:
model = {
"preprocess": <function>, # Function to transform request data into model input.
"predict": <function>, # Function to perform the actual model inference.
"params": <dict/any>, # Optional parameters passed to preprocess/predict.
"stream_features": <list[str]>, # Optional list of feature names to fetch from DynamoDB.
"kafka_features_to_ignore": <list[str]>, # Optional user/stream feature names omitted from stream-scoring Kafka logs.
}
Feature Caching (cache.py)
The ThreadSafeTLRUCache ensures that feature lookups and updates are thread-safe.
The _ttu (time-to-use) policy allows features to specify their own TTL via a cache_ttl_in_seconds key in the stored value.
DynamoDB Feature Fetching (dynamo.py)
The set_stream_features function handles:
-
Checking the in-memory cache for required
stream_features. -
Batch-fetching any missing features from DynamoDB.
-
Parsing the low-level DynamoDB items into Python types.
-
Populating the cache with the fetched data, respecting the feature's TTL.
-
Injecting the fetched feature values back into the streams list in the request payload.
Metadata
Release files for haystack-ml-stack 0.4.15
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| haystack_ml_stack-0.4.15.tar.gz | 38.5 kB | Without provenance | Details for haystack_ml_stack-0.4.15.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| haystack_ml_stack-0.4.15-py3-none-any.whl | Python 3 | none | any | Details for haystack_ml_stack-0.4.15-py3-none-any.whl |
Total release size: 69.8 kB
Release files / haystack_ml_stack-0.4.15.tar.gz
| Download URL | haystack_ml_stack-0.4.15.tar.gz |
|---|---|
| Size | 38.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5deb66e224d338d38edb82fd2beecbe2b49b4604cb6b4bc02485bed9e6c8e036
|
|
BLAKE2b-256 checksum How to use checksums |
a9b3836408856f9a8f0e13327f04ea4c1cefdcb7e26b85e5a2c69833bf13b39e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Release files / haystack_ml_stack-0.4.15-py3-none-any.whl
| Download URL | haystack_ml_stack-0.4.15-py3-none-any.whl |
|---|---|
| Size | 31.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b3bbd642ec806a547bb19732380cee9c16b248a6dd35c34cac9ed37c362e0c51
|
|
BLAKE2b-256 checksum How to use checksums |
53f1e8d2ab16034979a0b3f6dde376108bf75fc924b5cac00f895cf783618e21
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Security for haystack-ml-stack 0.4.15
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.