org21-git-scan 1.0.1

Scan your GitHub or GitLab org — generates metadata JSON for service, feature, and ownership discovery

Org21 Git Scanner

Scan your GitHub or GitLab organization to discover services, features, and code ownership — then upload the results to the Org21 platform for automatic knowledge graph enrichment.

Supports GitHub, GitLab.com, and self-hosted GitLab instances.

What It Does

The scanner reads your Git organization's metadata only — it never accesses source code content. It collects:

Data	How It's Used
Repository names & descriptions	Identifies your services
README first paragraph	Helps name services accurately
Sub-module structure (build files)	Identifies features within services
Domain directory structure	Identifies capability areas
Contributor commit counts	Determines code ownership
Contributor emails	Bridges identity across tools (Jira, HiBob, Git)
CODEOWNERS file	Maps explicit ownership declarations
Collaborator permissions	Identifies repo admins (strongest ownership signal)
First commit author	Identifies who created each repo
File extension counts	Identifies IaC, deployment, and language patterns
AI SDK usage	Detects repos using OpenAI, Anthropic, Gemini, etc.
Visibility (private/public/internal)	Privacy-aware processing
Archived status	Tracks inactive services

No source code is read, stored, or transmitted. Only repository metadata.

Quick Start

1. Install

pip install org21-git-scan

Requires Python 3.9+.

2. Set Up Authentication

The scanner supports two authentication methods: token (recommended) or username/password (for Enterprise and self-hosted setups).

Option A — Access Token (recommended)

For GitHub:

1. Go to github.com/settings/tokens
2. Click Generate new token (fine-grained)
3. Select your organization under Resource owner
4. Set Repository access → All repositories (or select specific ones)
5. Under Permissions → Repository permissions, grant:
   • Contents → Read-only
   • Metadata → Read-only
6. Click Generate token and copy it

For GitLab (Personal Access Token):

1. Go to gitlab.com/-/user_settings/personal_access_tokens
2. Create a token with scopes: read_api, read_repository
3. Copy the token (glpat-xxxx)

For GitLab (Group Access Token — fine-grained, recommended):

1. Go to your group → Settings → Access Tokens
2. Create a token with only these permissions (all read-only):

Category	Permission	Why
Repository	Read	CODEOWNERS, README, directory tree, file extensions
Group	Read	List projects in group (including subgroups)
Member	Read	Collaborator permission levels (admin/write/read)
Global Search	Read	AI usage detection (searches for openai/anthropic/etc.)

3. Set role to Reporter (minimum needed)
4. Copy the token (glpat-xxxx)

Important: The scanner only reads metadata — it never creates, modifies, or deletes anything. No write permissions are needed. Do NOT grant Branch, Merge Request, Pipeline, CI/CD, Work Item, or Enterprise User permissions.

Option B — Username & Password

For GitHub Enterprise or self-hosted GitLab with local accounts, you can use username and password instead of a token.

Note: GitHub.com requires a token (password auth is deprecated for github.com). Username/password works for GitHub Enterprise Server and self-hosted GitLab.

3. Run the Scanner

GitHub with token:

org21-git-scan --provider github --org YourOrgName --token ghp_your_token

GitHub Enterprise with username/password:

org21-git-scan --provider github --org YourOrgName --username your_user --password your_pass

GitLab.com with token:

org21-git-scan --provider gitlab --org YourGroupName --token glpat_your_token

Self-hosted GitLab with token:

org21-git-scan --provider gitlab --org YourGroupName --token glpat_your_token --url https://gitlab.internal.company.com

Self-hosted GitLab with username/password:

org21-git-scan --provider gitlab --org YourGroupName --username your_user --password your_pass --url https://gitlab.internal.company.com

This produces a git-metadata.json file in your current directory.

4. Upload to Org21

Upload the git-metadata.json file to your Org21 dashboard:

1. Log into your Org21 dashboard
2. Go to Settings → Integrations → Git
3. Click Upload Scan Results
4. Select your git-metadata.json file

Your services, features, and ownership data will appear in the knowledge graph within minutes.

Options

org21-git-scan --help

usage: org21-git-scan [-h] [--provider {github,gitlab}] --org ORG
                      [--token TOKEN] [--username USERNAME]
                      [--password PASSWORD] [--url URL] [--output OUTPUT]
                      [--max-repos MAX_REPOS] [--verbose]

options:
  --provider {github,gitlab}  Git provider (default: github)
  --org ORG                   GitHub organization or GitLab group name (required)
  --token TOKEN               Access token (or set GITHUB_TOKEN / GITLAB_TOKEN env var)
  --username USERNAME         Username for basic auth (or set GIT_USERNAME env var)
  --password PASSWORD         Password for basic auth (or set GIT_PASSWORD env var)
  --url URL                   GitLab instance URL for self-hosted (default: https://gitlab.com)
  --output, -o OUTPUT         Output file path (default: git-metadata.json)
  --max-repos MAX_REPOS       Maximum repos to scan (default: 100)
  --verbose, -v               Show detailed scanning progress

Authentication: Use either --token OR --username + --password. Not both.

Using Environment Variables

# GitHub with token
export GITHUB_TOKEN=ghp_your_token
org21-git-scan --provider github --org YourOrgName

# GitLab with token
export GITLAB_TOKEN=glpat_your_token
org21-git-scan --provider gitlab --org YourGroupName

# Any provider with username/password
export GIT_USERNAME=your_user
export GIT_PASSWORD=your_pass
org21-git-scan --provider gitlab --org YourGroupName --url https://gitlab.internal.com

Provider Comparison

Feature	GitHub	GitLab
List repos	✓ (org repos)	✓ (group projects, including subgroups)
Contributors + commits	✓	✓ (includes email directly)
CODEOWNERS	✓ (3 standard locations)	✓ (3 standard locations)
Collaborator permissions	✓ (admin/maintain/write/read)	✓ (Owner/Maintainer/Developer/Reporter/Guest)
README parsing	✓	✓
Sub-module detection	✓	✓
File extensions	✓	✓
AI usage search	✓ (Code Search API)	✓ (Group Search API)
First commit author	✓	✓
Self-hosted	✗ (GitHub Enterprise only)	✓ (any GitLab instance via --url)
Visibility levels	private / public	private / internal / public

What Gets Scanned

For each non-forked repository in your organization:

Item	API Calls	Notes
Repository info	1 per page	Name, description, language, last push, visibility, archived status
Contributors	1 per repo	Login, display name, commit count, email
Languages	1 per repo	Language breakdown
CODEOWNERS	Up to 3 per repo	Ownership patterns and assigned owners
README	1 per repo	Heading + first paragraph (for service naming)
Collaborators	1 per repo	Permission levels (admin/write/read)
First commit	1 per repo	Proxy for repo creator
Directory tree	1 per repo	Build manifests, sub-modules, domain directories, file extensions
Sub-READMEs	1 per documented dir	Subdirectory documentation
AI usage	~10 per org	Code search for AI SDK keywords

Typical scan time: 2-3 minutes for a 50-repo organization.

Output Format

The scanner produces a single JSON file with identical format regardless of provider:

{
  "org": "YourOrgName",
  "scanned_at": "2026-04-09T10:30:00Z",
  "provider": "github",
  "repos": [
    {
      "name": "auth-service",
      "description": "Authentication and authorization service",
      "readme_heading": "Auth Service",
      "readme_summary": "Multi-tenant authentication with OAuth2, PKCE, and session management.",
      "language": "Python",
      "languages": {"Python": 45000, "Shell": 2000},
      "file_extensions": {".py": 120, ".yaml": 15, ".md": 8},
      "last_push": "2026-04-07T15:30:00Z",
      "visibility": "private",
      "archived": false,
      "contributors": [
        {"login": "alice", "name": "Alice Smith", "commits": 142, "email": "[e-mail]"}
      ],
      "collaborators": [
        {"login": "alice", "name": "Alice Smith", "permission": "admin"}
      ],
      "codeowners": [
        {"pattern": "/src/oauth/", "owners": ["@alice", "@bob"]}
      ],
      "sub_modules": [
        {"path": "api-gateway", "name": "api-gateway", "build_file": "package.json"}
      ],
      "created_by": "alice",
      "admins": ["alice"],
      "uses_ai": true,
      "ai_references": ["openai", "langchain"]
    }
  ]
}

Privacy & Security

• No source code is accessed — only repository metadata (names, structure, contributors)
• No data leaves your machine — the JSON file is generated locally
• You control what's shared — review the JSON before uploading
• Fine-grained token — only read permissions needed, no write access
• Credentials stay local — tokens and passwords are never sent to Org21, only used for Git API calls on your machine
• Revoke anytime — delete the token or change the password to revoke access immediately
• Basic auth supported — for environments where token generation is restricted (e.g., self-hosted GitLab with LDAP)
• Self-hosted GitLab — for organizations that can't use cloud Git, the scanner runs entirely within your network

How Org21 Uses This Data

Once uploaded, Org21 analyzes the metadata to enrich your knowledge graph:

Input	Output
Repositories	Services with human-readable names
Sub-modules & domain directories	Features within services
Contributors, CODEOWNERS, admins	Ownership — who owns which service
Contributor emails	Identity bridging across Jira, Git, and HR systems
File extensions	Service classification (IaC, deployment, application)
AI references	AI stack discovery (which services use AI)

This data cross-references with other connectors (Jira, HiBob, NetSuite) to build a complete picture of your organization's technology landscape.

Scheduling Regular Scans

# Crontab (daily at 2am)
0 2 * * * GITHUB_TOKEN=ghp_xxx org21-git-scan --provider github --org YourOrg -o /path/to/git-metadata.json

# Or run manually after significant changes
org21-git-scan --provider github --org YourOrg

Then upload the updated file to your Org21 dashboard.

Troubleshooting

Error	Solution
Error: Access token required	Set --token or the appropriate env var
403 Forbidden	Token doesn't have required permissions — check scopes
404 Not Found	Organization/group name is wrong or token doesn't have access
Rate limit exceeded	Wait and retry, or use a token with higher limits
Scan takes too long	Use --max-repos 50 to limit scope
Self-hosted GitLab SSL error	Ensure your GitLab URL uses a valid certificate

Support

For questions or issues: Contact your Org21 account team