📊 Linux Foundation Project Reporting System

Comprehensive multi-repository analysis tool for Linux Foundation projects

Generate detailed reports on Gerrit projects, contributor activity, Jenkins jobs, GitHub CI/CD workflows, and development practices across repositories.

🗒️ Published Reports

https://lfreleng-actions.github.io/project-reporting-tool/

🗄️ Report Data and Retention

The daily 📊 Production Reports workflow (.github/workflows/reporting-production.yaml) keeps its outputs in two places:

• GitHub Pages (deployed with actions/deploy-pages): the latest report.html, report.md, report_raw.json and metadata.json per project. Each run rebuilds the whole site from report artifacts and replaces the previous deployment. An active project without a report from that run keeps the newest unexpired reports-<slug> artifact holding a report.html from an earlier scheduled or dispatched run of this workflow on the default branch; once none remains, it drops off the site. A failed API lookup for such an artifact fails the run instead and leaves the previous deployment live.
• Workflow run artifacts: raw-data-<slug>, reports-<slug>, clone-manifest-<slug> and clone-log-<slug>, kept for 90 days, after which GitHub deletes them.

To fetch data from an earlier run within those 90 days, download the artifacts from that workflow run in the Actions tab, or use .github/scripts/download-artifacts.sh. The workflow no longer copies outputs into the lfreleng-actions/project-reporting-artifacts repository, so that repository receives no new daily folders.

⚡ Quick Start

# Run without installing
uvx lf-releng-project-reporting generate \
  --project my-project \
  --repos-path ./repos

# Or install as a tool, then run lf-releng-project-reporting directly
uv tool install lf-releng-project-reporting

🚀 Key Features

• 📈 Git Analytics - Commit activity, lines of code, contributor metrics across configurable time windows
• 📋 INFO.yaml Reporting - Project metadata, committer activity, and lifecycle state tracking from info-master
• 🔍 Feature Detection - Automatic detection of CI/CD, documentation, dependency management, security tools
• 👥 Contributor Intelligence - Author and organization analysis with domain mapping
• 🌐 API Integration - GitHub, Gerrit, and Jenkins API support
• 🎯 CI-Management Integration - Authoritative Jenkins job allocation using JJB definitions (99%+ accuracy)
• 📊 Interactive Reports - JSON (data), Markdown (readable), HTML (interactive), ZIP (bundled)
• ⚡ High Performance - Parallel processing with caching support

📚 Documentation

🎯 Getting Started

• Getting Started Guide - Complete installation and setup walkthrough
• Commands Reference - Full command-line reference with quick reference
• FAQ - Frequently asked questions
• Usage Examples - Real-world scenarios and patterns

⚙️ Setup & Configuration

• Configuration Guide - All configuration options (GitHub API, INFO.yaml, performance)
• Configuration Merging - How project configs inherit and override defaults
• Deployment Guide - Production deployment and operations
• CI/CD Integration - GitHub Actions, GitLab CI, and automation

🔧 Advanced Usage

• Performance Guide - Optimization, caching, and scaling
• Feature Discovery - Understanding automatic feature detection
• INFO.yaml Reporting - Project metadata and committer activity tracking
• Troubleshooting - Problem solving and debugging

👨‍💻 Development

• Developer Guide - Architecture, API reference, and contributing
• Template Development - Customizing Jinja2 templates and creating new output formats
• Testing Guide - Test suite documentation
• Migration Guide - Production migration from legacy system

🔍 Development Tools

• Template Audit Script - python scripts/audit_templates.py - Comprehensive audit of all Jinja2 templates to verify field accesses match context builders, preventing runtime errors

💻 Installation

Using UV (Recommended)

# Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh

# Run the published package without installing
uvx lf-releng-project-reporting --help

# Or install it as a tool
uv tool install lf-releng-project-reporting

Developing from a checkout

# Install dependencies
uv sync

# Run the tool
# Note: repos-path should match the directory created by gerrit-clone-action
# which defaults to the Gerrit server hostname (e.g., ./gerrit.o-ran-sc.org)
uv run lf-releng-project-reporting generate --project O-RAN-SC --repos-path ./gerrit.o-ran-sc.org

→ Detailed Setup Instructions

🎯 Common Use Cases

Note: The --repos-path should point to the directory created by gerrit-clone-action, which uses the Gerrit server hostname as the directory name (e.g., ./gerrit.o-ran-sc.org for O-RAN-SC, ./gerrit.onap.org for ONAP).

📊 Output Formats

reports/
  <PROJECT>/
    ├── report_raw.json              # Complete dataset (canonical)
    ├── report.md                    # Markdown report (readable)
    ├── report.html                  # Interactive HTML (sortable tables)
    ├── config_resolved.json         # Applied configuration
    └── <PROJECT>_report_bundle.zip  # Complete bundle

🔌 CI/CD Integration

GitHub Actions

- name: Generate Report
  run: |
    uv run lf-releng-project-reporting generate \
      --project "${{ matrix.project }}" \
      --repos-path "./${{ matrix.server }}" \
      --cache \
      --quiet
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

🔧 Requirements

• Python: 3.11+ (supports 3.11, 3.12, 3.13, 3.14)
• Dependencies: PyYAML, httpx, Jinja2, typer, rich
• Optional: GitHub token for API features (required for workflow status colors)

GitHub Token Requirements

For full workflow status reporting (colored status indicators), you need a GitHub Personal Access Token (Classic) with these permissions:

Required Scopes:

• ☑ repo - Full repository access (or public_repo for public repositories)
• ☑ actions:read - Read GitHub Actions workflow runs and status

Note: Fine-grained tokens are not supported as they cannot span organizations.

Setup:

# Set environment variable
export GITHUB_TOKEN=ghp_your_token_here
# OR for CI/production:
export CLASSIC_READ_ONLY_PAT_TOKEN=ghp_your_token_here

# Then run the tool
lf-releng-project-reporting generate --project my-project --repos-path ./repos

Create token: https://github.com/settings/tokens

Without a token: The tool detects workflows but shows them as grey (unknown status) instead of colored status indicators.

See also: Configuration Guide for detailed token setup

Jenkins Authentication (Optional)

Some Jenkins servers require authentication to view job information. If you encounter a "returned 0 jobs" error, you need to provide Jenkins credentials.

Setup:

# Generate API token in Jenkins: User → Configure → API Token → Add new Token
export JENKINS_USER="your-username"
export JENKINS_API_TOKEN="your-api-token"

# Then run the tool
lf-releng-project-reporting generate --project my-project --repos-path ./repos

Create token: Log into your Jenkins instance → Your Username → Configure → API Token

Without credentials: The tool will fail with an error if Jenkins requires authentication.

See also: Troubleshooting Guide for detailed setup

📖 Key Documentation Files

💡 Quick Tips

• 🎯 First time? Start with Getting Started Guide
• ⚡ Slow? Add --cache --workers 8 for parallel processing
• 🐛 Issues? Check Troubleshooting Guide
• ❓ Questions? See FAQ
• 📖 Need help? Run lf-releng-project-reporting --help
• 🔍 Developing templates? Run python scripts/audit_templates.py to verify all field accesses

🛠️ Development Scripts

Template Audit Script

python scripts/audit_templates.py

This script performs a comprehensive audit of all Jinja2 templates to:

• Extract all field accesses from templates (e.g., repo.field, org.field)
• Analyze context builders to see what fields they provide
• Identify mismatches that could cause "Undefined variable" errors at runtime

When to use:

• Before committing template changes
• After modifying context builders
• When debugging template rendering errors
• During code review to verify template correctness

Output:

• ✅ Green checkmarks - All required fields present
• ⚠️ Warnings - Extra fields exist (safe, unused)
• ❌ Red errors - Missing fields that will cause runtime failures

Example output:

================================================================================
TEMPLATE FIELD ACCESS AUDIT
================================================================================

📄 html/sections/summary.html.j2
  summary:
    - active_count
    - current_count
    - repositories_analyzed

✅ No critical issues found!
   Extra fields are safe - they're unused.

🤝 Support

• Documentation: Complete Index
• Issues: GitHub Issues

📜 License

Apache-2.0 License - Copyright 2025 The Linux Foundation