A Python library that contains tools for data discovery, data model generation and ingestion for the Neo4j graph database.
Project description
Neo4j Runway
Neo4j Runway is a Python library that simplifies the process of migrating your relational data into a graph. It provides tools that abstract communication with OpenAI to run discovery on your data and generate a data model, as well as tools to generate ingestion code and load your data into a Neo4j instance.
Key Features
- Data Discovery: Harness OpenAI LLMs to provide valuable insights from your data
- Graph Data Modeling: Utilize OpenAI and the Instructor Python library to create valid graph data models
- Code Generation: Generate ingestion code for your preferred method of loading data
- Data Ingestion: Load your data using Runway's built in implementation of PyIngest - Neo4j's popular ingestion tool
Requirements
Runway uses graphviz to visualize data models. To enjoy this feature please download graphviz.
You'll need a Neo4j instance to fully utilize Runway. Start up a free cloud hosted Aura instance or download the Neo4j Desktop app.
Get Running in Minutes
pip install neo4j-runway
Now let's walk through a basic example.
Here we import the modules we'll be using.
import pandas as pd
from neo4j_runway import Discovery, GraphDataModeler, IngestionGenerator, LLM, PyIngest
Discovery
Now we define a General Description of our data, provide brief descriptions of the columns of interest and load the data with Pandas.
USER_GENERATED_INPUT = {
'general_description': 'This is data on different countries.',
'id': 'unique id for a country.',
'name': 'the country name.',
'phone_code': 'country area code.',
'capital': 'the capital of the country.',
'currency_name': "name of the country's currency.",
'region': 'primary region of the country.',
'subregion': 'subregion location of the country.',
'timezones': 'timezones contained within the country borders.',
'latitude': 'the latitude coordinate of the country center.',
'longitude': 'the longitude coordinate of the country center.'
}
data = pd.read_csv("data/csv/countries.csv")
We then initialize our llm. By default we use GPT-4o and define our OpenAI API key in an environment variable.
llm = LLM()
And we run discovery on our data.
disc = Discovery(llm=llm, user_input=USER_GENERATED_INPUT, data=data)
disc.run()
Data Modeling
We can now pass our Discovery object to a GraphDataModeler to generate our initial data model. A Discovery object isn't required here, but it provides rich context to the LLM to achieve the best results.
gdm = GraphDataModeler(llm=llm, discovery=disc)
gdm.create_initial_model()
If we have graphviz installed, we can take a look at our model.
gdm.current_model.visualize()
Let's make some corrections to our model and view the results.
gdm.iterate_model(user_corrections="""
Make Region node have a HAS_SUBREGION relationship with Subregion node.
Remove The relationship between Country and Region.
""")
gdm.current_model.visualize()
Code Generation
We can now use our data model to generate some ingestion code.
gen = IngestionGenerator(data_model=gdm.current_model,
username="neo4j", password="password",
uri="bolt://localhost:7687", database="neo4j",
csv_dir="data/csv/", csv_name="countries.csv")
pyingest_yaml = gen.generate_pyingest_yaml_string()
Ingestion
We will use the generated PyIngest yaml config to ingest our CSV into our Neo4j instance.
PyIngest(yaml_string=pyingest_yaml, dataframe=data)
We can also save this as a .yaml file and use with the original PyIngest.
gen.generate_pyingest_yaml_file(file_name="countries")
Here's a snapshot of our new graph!
Limitations
The current project is in beta and has the following limitations:
- Single CSV input only for data model generation
- Nodes may only have a single label
- Only uniqueness and node / relationship key constraints are supported
- Relationships may not have uniqueness constraints
- CSV columns that refer to the same node property are not supported in model generation
- Only OpenAI models may be used at this time
- The modified PyIngest function included with Runway only supports loading a local Pandas DataFrame or CSVs
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for neo4j_runway-0.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 85138387d539b8872a2daf20c7e9a58b5ef201b17da521682bccc2e9e3ce36d2 |
|
MD5 | d706883c9a673c026d328d27df1396e3 |
|
BLAKE2b-256 | 9fc4059e19a85c653336b11c124ba14ff07ddf0830ca28fae23e95f122086a25 |