nodestream-plugin-github

Overview

This plugin provides a way to scrape GitHub data from the REST api and ingest them as extractors in nodestream pipelines.

Setup Neo4j

Download and install Neo4j: https://neo4j.com/docs/desktop-manual/current/installation/download-installation/
Create and start database (version 5.7.0: https://neo4j.com/docs/desktop-manual/current/operations/create-dbms/
Install APOC: https://neo4j.com/docs/apoc/5/installation/

Create GitHub credentials

Create and GitHub access codes: https://docs.github.com/en/enterprise-server@3.12/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app NOTE: These values will be used in your .env

Install and run the app

Install python3: https://www.python.org/downloads/
Install poetry: https://python-poetry.org/docs/#installation
Install nodestream: https://nodestream-proj.github.io/nodestream/0.5/docs/tutorial/
Generate a new nodestream project
Add nodestream-github to your project dependencies in your nodestream projects pyproject.toml file.
Install necessary dependencies: poetry install
In nodestream.yaml add the following:

plugins:
  - name: github
    config:
      github_hostname: github.example.com
      auth_token: !env GITHUB_ACCESS_TOKEN
      user_agent: skip-jbristow-test
      per_page: 100
      collecting:
        all_public: True
      rate_limit_per_minute: 225
    targets:
      - my-db:
    pipelines:
      - name: github_repos
      - name: github_teams
targets:
  database: neo4j
  uri: bolt://localhost:7687
  username: neo4j
  password: neo4j123

Set environment variables in your terminal session for: GITHUB_ACCESS_TOKEN.
Verify nodestream has loaded the pipelines: poetry run nodestream show
Use nodestream to run the pipelines: poetry run nodestream run <pipeline-name> --target my-db

Using make

Install make (ie. brew install make)
Run make run

Contributing

When contributing, make sure to sign your commits. To find out more about how to do this, refer to this GitHub documentation.

Authors

Jon Bristow
Zach Probst
Rohith Reddy

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nodestream_plugin_github-0.16.1.tar.gz (16.5 kB view details)

Uploaded Jul 9, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

nodestream_plugin_github-0.16.1-py3-none-any.whl (25.3 kB view details)

Uploaded Jul 9, 2026 Python 3

File details

Details for the file nodestream_plugin_github-0.16.1.tar.gz.

File metadata

Download URL: nodestream_plugin_github-0.16.1.tar.gz
Upload date: Jul 9, 2026
Size: 16.5 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for nodestream_plugin_github-0.16.1.tar.gz
Algorithm	Hash digest
SHA256	`2422372093d09a204095fbf1d89a69bcb0bd9e152e7e203de6be53daf37c59f8`
MD5	`8249c9c31fe073cc1c0492cd51bc15bf`
BLAKE2b-256	`2dc7a3b2203f0ac721db3eee11e96f31a93422d7f7d9eb9c6524f0bd147d85d2`

See more details on using hashes here.

Provenance

The following attestation bundles were made for nodestream_plugin_github-0.16.1.tar.gz:

Publisher: release.yaml on nodestream-proj/nodestream-plugin-github

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: nodestream_plugin_github-0.16.1.tar.gz
- Subject digest: 2422372093d09a204095fbf1d89a69bcb0bd9e152e7e203de6be53daf37c59f8
- Sigstore transparency entry: 2132688088
- Sigstore integration time: Jul 9, 2026
Source repository:
- Permalink: nodestream-proj/nodestream-plugin-github@3c23eb637aef0c5e67a743eeb189aa9c4e6f1879
- Branch / Tag: refs/tags/0.16.1
- Owner: https://github.com/nodestream-proj
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yaml@3c23eb637aef0c5e67a743eeb189aa9c4e6f1879
- Trigger Event: release