nodestream-plugin-github
Overview
This plugin provides a way to scrape GitHub data from the REST api and ingest them as extractors in nodestream pipelines.
Setup Neo4j
- Download and install Neo4j: https://neo4j.com/docs/desktop-manual/current/installation/download-installation/
- Create and start database (version 5.7.0: https://neo4j.com/docs/desktop-manual/current/operations/create-dbms/
- Install APOC: https://neo4j.com/docs/apoc/5/installation/
Create GitHub credentials
- Create and GitHub access
codes: https://docs.github.com/en/enterprise-server@3.12/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app
NOTE: These values will be used in your
.env
Install and run the app
- Install python3: https://www.python.org/downloads/
- Install poetry: https://python-poetry.org/docs/#installation
- Install nodestream: https://nodestream-proj.github.io/nodestream/0.5/docs/tutorial/
- Generate a new nodestream project
- Add
nodestream-githubto your project dependencies in your nodestream projects pyproject.toml file. - Install necessary dependencies:
poetry install - In
nodestream.yamladd the following:
plugins:
- name: github
config:
github_hostname: github.example.com
auth_token: !env GITHUB_ACCESS_TOKEN
user_agent: skip-jbristow-test
per_page: 100
collecting:
all_public: True
rate_limit_per_minute: 225
targets:
- my-db:
pipelines:
- name: github_repos
- name: github_teams
targets:
database: neo4j
uri: bolt://localhost:7687
username: neo4j
password: neo4j123
- Set environment variables in your terminal session for:
GITHUB_ACCESS_TOKEN. - Verify nodestream has loaded the pipelines:
poetry run nodestream show - Use nodestream to run the pipelines:
poetry run nodestream run <pipeline-name> --target my-db
Using make
- Install make (ie.
brew install make) - Run
make run
Contributing
When contributing, make sure to sign your commits. To find out more about how to do this, refer to this GitHub documentation.
Authors
- Jon Bristow
- Zach Probst
- Rohith Reddy
Metadata
Release files for nodestream-plugin-github 0.16.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| nodestream_plugin_github-0.16.2.tar.gz | 85.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| nodestream_plugin_github-0.16.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 184.1 kB
Release files / nodestream_plugin_github-0.16.2.tar.gz
| Download URL | nodestream_plugin_github-0.16.2.tar.gz |
|---|---|
| Size | 85.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ae28e1d1fd4e1d401dc9596cb2a27921d89c32d10d6ceac2696c7aa05834d6eb
|
|
BLAKE2b-256 checksum How to use checksums |
a9c38fbbe32c21c83c826d244715185b0280c3fa390cfce4789ebc85360426ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 14, 2026.
Transparency logRelease files / nodestream_plugin_github-0.16.2-py3-none-any.whl
| Download URL | nodestream_plugin_github-0.16.2-py3-none-any.whl |
|---|---|
| Size | 98.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
51ff98d72e35bec68238be27f5f20d51b6ada2c2af167864fe3f7550f6a0ce22
|
|
BLAKE2b-256 checksum How to use checksums |
da6e53375a9fd28e9cea1e9df2ed000c6ac4d9d36b2b3cb34034a57a9e707fc0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 14, 2026.
Transparency log
