nodestream 0.15.0

A Fast, Declarative ETL for Graph Databases.

Nodestream

A Declarative framework for Building, Maintaining, and Analyzing Graph Data 🚀

Nodestream allows you to work with graphs declaratively. With nodestream, you unlock a bounty of features purpose built for working with graphs. Semantically model your graph and map labels and properties directly to your data. Better yet, you are not locked into your choices. Nodestream works with you as you evolve your application by providing migration utilities to change your data schema. Nodestream even decouples you from the underyling database technology so you can even change databases.

Highlights

• Connect to data sources like Kafka, files, apis, and more!
• Evolve your application over time with database migrations (Docs)
• Use your favorite Graph Database to fit any tech stack (Docs)
• Clean up your own data with TTLs (Docs)
• Infinite Customizability Since Nearly Everything is Pluggable!

Website • Blog • Discussions • Contributing • Contributing Developer Guides • Talks from Maintainers

Features

Nodestream has a pleasant CLI interface to get new projects up and running fast.

Not a fan of the defaults? You can change out databases very easily

Then you can start to model your data and nodestream will evolve your database for you. No more messing with constraints or writing database queries.

Getting Started

Conviced? Install nodestream with pip to get started.

  pip install nodestream
  nodestream new --database neo4j my_project && cd my_project
  nodestream run sample -v

We highly recommend following our tutorials here

Packages

Nodestream is built on a Highly Pluggable and Modular Architecture. Thus... we have a lot of packages to keep track of.

Package	Description	Version
nodestream	The core library. Declarative ingestion.
nodestream-plugin-neo4j	Neo4j database connector.
nodestream-plugin-neptune	AWS Neptune database connector.
nodestream-plugin-dotenv	Adds DotEnv integration.
nodestream-plugin-pedantic	A series of lints to enforce reasonable naming standards, etc.
nodestream-plugin-shell	An integration with nodestream to run shell commands.
nodestream-plugin-sbom	Import SBOM files in CycloneDX and SPDX into an opinionated graph data model.
nodestream-plugin-akamai	Parse Akamai properties, redirect configs, and much more and ingests them.
nodestream-plugin-k8s	In incubation. A plugin that orchestrates Nodestream on k8s.

Contributors

Nodestream is a community project. We welcome all contributions. Be sure to checkout or Contributing Docs and our Code of Conduct before contributing.

Zach Probst 💻 👀 🚧	Chad Cloes 💻 👀 🚧	asantos4 💻 👀 🚧	Grant Hoffman 💻 👀	khneal 💻	orozen 💻	Sophia Don Tranho 💻
bechbd 💻	yasonk 💻 👀	Stuart Macleod 💻	Cole Greer 💻	Austin Lucas 💻	rreddy2 💻	Jon Bristow 💻
Add your contributions

Contributing

Need a quick reference guide on how to contribute? Here you go!

Getting Setup

To get started you'll need to install poery.

curl -sSL https://install.python-poetry.org | python3 -

You then can install the project dependencies with the following command:

poetry install

No need to active a virtual environment. Poetry handles that for you with poetry run and poetry shell.

Running Tests

To run tests for the entire project, run the following command:

poetry run pytest