GraFlo 
GraFlo is a Python library that turns records from files, SQL databases, RDF, REST APIs or Kafka topics into a labeled property graph. You describe the graph once, in a YAML file called a manifest, and GraFlo creates the schema and writes the vertices and edges into the graph database of your choice, or into a directory on disk.
It is for engineers who build a graph from several sources and want its description in one reviewable file rather than spread across load scripts.
What you can do with it
- Describe a graph once and load data into it. A manifest names the vertex
and edge types, says which properties identify a vertex, and says how each
kind of record becomes vertices and edges. The same manifest loads into
ArangoDB, Neo4j, TigerGraph, FalkorDB, Memgraph, NebulaGraph, PostgreSQL or
the file backend, and records with the same identity become one vertex.
GraFlo also copies an existing graph from Neo4j, ArangoDB or PostgreSQL into
another database (
GraphEngine.migrate_graph). - Change the description over time, with a recorded history. Renaming a
type, combining two types or changing a property type is a typed operation.
Operations are recorded as commits (
graflo commit,log,checkout,verify,revert) that you can replay, check and, for most operations, undo. Two branches of changes to one manifest are reconciled with a three-way merge (graflo merge3), and two manifests written by different teams are combined into one with a union (graflo merge). - Check and infer descriptions. GraFlo infers a manifest from a PostgreSQL
database or an OWL ontology, proposes the properties that identify a record
from sample data, and checks a manifest against a conformance profile
(
graflo check), a set of modeling rules such as "every vertex type declares its identity".
A taste
A manifest has three blocks: schema says what the graph looks like,
ingestion_model says how records map onto it, and bindings says where the
records come from. This one reads CSV files with the columns person_id,
person and department:
schema:
metadata: {name: hr}
graph:
vertex_config:
vertices:
- {name: person, properties: [id, name], identity: [id]}
- {name: department, properties: [name], identity: [name]}
edge_config:
edges: [{source: person, target: department}]
ingestion_model:
resources:
- name: departments
pipeline:
- {vertex: person, from: {id: person_id, name: person}}
- {vertex: department, from: {name: department}}
bindings:
connectors:
- {regex: "^dep.*\\.csv$", sub_path: data, resource_name: departments}
This loads it into ArangoDB:
from graflo import GraphEngine, GraphManifest
from graflo.connections import ArangoConfig
manifest = GraphManifest.from_yaml("manifest.yaml")
manifest.finish_init()
engine = GraphEngine()
engine.define_and_ingest(manifest=manifest, target_db_config=ArangoConfig.from_env())
ArangoConfig.from_env() reads ARANGO_URI, ARANGO_USERNAME,
ARANGO_PASSWORD and ARANGO_DATABASE; every database has such a class. See
Database connections.
Documentation
Full documentation: growgraph.github.io/graflo
- Quick start: two CSV files into a graph, step by step
- Creating a manifest: the three blocks of a manifest
- Examples: runnable examples, one question each, with their data under
examples/ - Concepts: schema, identity, ingestion, connectors, evolution and version control
- Guides: database connections, graph migration, schema inference, API wiring, bulk load
- GraFlo ontology: a manifest as RDF (
graflo manifest-to-rdf,graflo rdf-to-manifest)
Installation
GraFlo needs Python 3.11 or newer. The database clients, RDF and Kafka support are part of the default install.
pip install graflo
Optional extras (see the Installation guide):
dev: pytest and its plugins, hypothesis, ty, pre-commitdocs: ProperDocs and its plugins, for building the documentation siteplot:pygraphvizforgraflo plot-manifestand the--plotfigures ofgraflo mergeandgraflo merge3
pip install "graflo[dev,docs,plot]"
Development
To install from a clone:
git clone git@github.com:growgraph/graflo.git && cd graflo
uv sync --extra dev
See the Contributing Guide for the full workflow.
Tests
The database tests need the database containers. Start them from a clone with the scripts under docker/:
cd docker
./start-all.sh # Start all services
./stop-all.sh # Stop all services
./cleanup-all.sh # Remove containers and volumes
Per-engine compose files and ports are documented in the docker README.
To run the tests:
uv run pytest test
TigerGraph, NebulaGraph and Kafka tests are skipped unless you pass
--run-tigergraph, --run-nebula or --run-kafka.
The suites that need no database run without the containers, and CI runs them on every pull request:
uv run pytest test --ignore=test/db --ignore=test/data_source --ignore=test/object_storage
License
Open source under the Apache License 2.0. Copyright and trademark notices are in NOTICE: the license grants no rights in the GraFlo and GrowGraph marks. Releases before the relicensing shipped under the Business Source License 1.1 and keep those terms; see the changelog.
Contributing
Contributions are welcome. See the Contributing Guide. Contributors accept the Contributor License Agreement once, by commenting on their first pull request.
1 maintainer
Metadata
1 maintainer
Release files for graflo 1.15.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| graflo-1.15.0.tar.gz | 875.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| graflo-1.15.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.9 MB
Release files / graflo-1.15.0.tar.gz
| Download URL | graflo-1.15.0.tar.gz |
|---|---|
| Size | 875.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
091ee96b411d3c2e5e8a2b2477d5bb12ada7a360b104b8f9fbd5dfeee099253d
|
|
BLAKE2b-256 checksum How to use checksums |
337658961a6e4047522267cb07bf4e9f15c42bc266c08207998e59429b83e986
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.21 {"installer":{"name":"uv","version":"0.12.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / graflo-1.15.0-py3-none-any.whl
| Download URL | graflo-1.15.0-py3-none-any.whl |
|---|---|
| Size | 1.0 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bcdc8a29c836c1d92e9a42a09c63cab3c31488f95664542ef08cbd07cf1d11e0
|
|
BLAKE2b-256 checksum How to use checksums |
215ba5fc3137304e8346d31755ff6782537be8ad9793940def39e11566dbef1e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.21 {"installer":{"name":"uv","version":"0.12.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
