github-to-sqlite 0.5

Save data from GitHub to a SQLite database

github-to-sqlite

Save data from GitHub to a SQLite database.

How to install

$ pip install github-to-sqlite

Authentication

Create a GitHub personal access token: https://github.com/settings/tokens

Run this command and paste in your new token:

$ github-to-sqlite auth

This will create a file called auth.json in your current directory containing the required value. To save the file at a different path or filename, use the --auth=myauth.json option.

Retrieving issues for a repository

The issues command retrieves all of the issues belonging to a specified repository.

$ github-to-sqlite issues github.db simonw/datasette

If an auth.json file is present it will use the token from that file. It works without authentication for public repositories but you should be aware that GitHub have strict IP-based rate limits for unauthenticated requests.

You can point to a different location of auth.json using -a:

$ github-to-sqlite issues github.db simonw/datasette -a /path/to/auth.json

You can use the --issue option to only load just one specific issue:

$ github-to-sqlite issues github.db simonw/datasette --issue=1

Retrieving issue comments for a repository

The issue-comments command retrieves all of the comments on all of the issues in a repository.

It is recommended you run issues first, so that each imported comment can have a foreign key poining to its issue.

$ github-to-sqlite issues github.db simonw/datasette
$ github-to-sqlite issue-comments github.db simonw/datasette

You can use the --issue option to only load comments for a specific issue within that repository, for example:

$ github-to-sqlite issue-comments github.db simonw/datasette --issue=1

Fetching repos belonging to a user or organization

The repos command fetches repos belonging to a user or organization.

Without any other arguments, this command will fetch all repos that the currently authenticated user owns, collaborates on or can access via one of their organizations:

$ github-to-sqlite repos github.db

To fetch repos belonging to a specific user or organization, provide their username as an argument:

$ github-to-sqlite repos github.db dogsheep # organization
$ github-to-sqlite repos github.db simonw # user

Fetching repos that have been starred by a user

The starred command fetches the repos that have been starred by a user.

$ github-to-sqlite starred github.db simonw

If you are using an auth.json file you can omit the username to retrieve the starred repos for the authenticated user.