haive-dataflow
Data processing pipelines and ETL workflows for Haive agents.
A registry, discovery, and serialization system for managing components, persistence, and data flows in the Haive framework. Use it for component management, agent persistence, dataflow orchestration, and FastAPI integration.
Why haive-dataflow?
Production agent systems need more than just agents — they need:
- Component registry — track which agents, tools, and configs are available
- Serialization — save and load complex agent configs across processes
- Persistence — store agent state, conversation history, results
- Streaming — real-time data flows for production pipelines
- API integration — serve agents as HTTP endpoints
haive-dataflow provides all of this. It's the production infrastructure layer.
Features
📦 Component Registry
Register and discover Haive components at runtime:
from haive.dataflow.registry import ComponentRegistry
registry = ComponentRegistry()
# Register agents
registry.register("research_agent", researcher)
registry.register("writer_agent", writer)
# Discover by type
all_agents = registry.list_components(component_type="agent")
# Retrieve
agent = registry.get("research_agent")
🔄 Serialization
Save and restore agent configs:
from haive.dataflow.serialization import serialize_agent, deserialize_agent
# Save to JSON
config_json = serialize_agent(my_agent)
with open("agent.json", "w") as f:
f.write(config_json)
# Restore
with open("agent.json") as f:
restored = deserialize_agent(f.read())
💾 Persistence
Multiple backends with sync and async support:
from haive.dataflow.persistence import PostgresBackend, SupabaseBackend
# PostgreSQL
backend = PostgresBackend(
connection_string="postgresql://haive:haive@localhost/haive",
pool_size=10,
)
# Supabase
backend = SupabaseBackend(
url="https://your-project.supabase.co",
key="your-anon-key",
)
# Save state
await backend.save_state("session_123", agent_state)
# Restore
state = await backend.load_state("session_123")
🌐 FastAPI Integration
Serve agents as HTTP endpoints:
from fastapi import FastAPI
from haive.dataflow.api import create_agent_router
app = FastAPI()
app.include_router(create_agent_router(my_agent), prefix="/agents/researcher")
# Now POST to /agents/researcher/run with JSON body
Installation
pip install haive-dataflow
# With FastAPI integration
pip install haive-dataflow[api]
# With Supabase backend
pip install haive-dataflow[supabase]
Quick Start
from haive.dataflow.registry import ComponentRegistry
from haive.agents.simple.agent import SimpleAgent
from haive.core.engine.aug_llm import AugLLMConfig
# Create and register
registry = ComponentRegistry()
agent = SimpleAgent(name="hello", engine=AugLLMConfig())
registry.register("hello", agent)
# Use
component = registry.get("hello")
result = component.run("Hello world")
Documentation
📖 Full documentation: https://pr1m8.github.io/haive-dataflow/
Related Packages
| Package | Description |
|---|---|
| haive-core | Foundation: engines, graphs, persistence |
| haive-agents | Production agents (registered in dataflow) |
| haive-mcp | MCP integration |
License
MIT © pr1m8
Metadata
Release files for haive-dataflow 1.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| haive_dataflow-1.0.1.tar.gz | 354.5 kB | Without provenance | Details for haive_dataflow-1.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| haive_dataflow-1.0.1-py3-none-any.whl | Python 3 | none | any | Details for haive_dataflow-1.0.1-py3-none-any.whl |
Total release size: 817.1 kB
Release files / haive_dataflow-1.0.1.tar.gz
| Download URL | haive_dataflow-1.0.1.tar.gz |
|---|---|
| Size | 354.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9fe3c3402104d33fcadc813e44332f86980961c12d5069560f088be4aa4643cc
|
|
BLAKE2b-256 checksum How to use checksums |
fd67825a06caae76180ad318a7d57022edf1735ae6d561b6259ca041b2241e07
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Release files / haive_dataflow-1.0.1-py3-none-any.whl
| Download URL | haive_dataflow-1.0.1-py3-none-any.whl |
|---|---|
| Size | 462.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
97ca13c010974b9b7cdd794c7ed757652cceb03a36afdf9e61d79bbbfa669c2e
|
|
BLAKE2b-256 checksum How to use checksums |
98a0ea7a75d4d150b56e37c23caef9c4a4d75b8f589f7d2396d60a89c8f07015
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Security for haive-dataflow 1.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.