lf-releng-project-reporting 0.2.0

Comprehensive multi-repository analysis tool for Linux Foundation projects

📊 Linux Foundation Project Reporting System

Comprehensive multi-repository analysis tool for Linux Foundation projects

Generate detailed reports on Gerrit projects, contributor activity, Jenkins jobs, GitHub CI/CD workflows, and development practices across repositories.

---

🗒️ Published Reports

https://lfreleng-actions.github.io/project-reporting-tool/

⚡ Quick Start

# Install
pip install .

# Generate your first report
lf-releng-project-reporting generate \
  --project my-project \
  --repos-path ./repos

---

🚀 Key Features

• 📈 Git Analytics - Commit activity, lines of code, contributor metrics across configurable time windows
• 📋 INFO.yaml Reporting - Project metadata, committer activity, and lifecycle state tracking from info-master
• 🔍 Feature Detection - Automatic detection of CI/CD, documentation, dependency management, security tools
• 👥 Contributor Intelligence - Author and organization analysis with domain mapping
• 🌐 API Integration - GitHub, Gerrit, and Jenkins API support
• 🎯 CI-Management Integration - Authoritative Jenkins job allocation using JJB definitions (99%+ accuracy)
• 📊 Interactive Reports - JSON (data), Markdown (readable), HTML (interactive), ZIP (bundled)
• ⚡ High Performance - Parallel processing with caching support

---

📚 Documentation

🎯 Getting Started

• Getting Started Guide - Complete installation and setup walkthrough
• Commands Reference - Full command-line reference with quick reference
• FAQ - Frequently asked questions
• Usage Examples - Real-world scenarios and patterns

⚙️ Setup & Configuration

• Configuration Guide - All configuration options (GitHub API, INFO.yaml, performance)
• Configuration Merging - How project configs inherit and override defaults
• Deployment Guide - Production deployment and operations
• CI/CD Integration - GitHub Actions, GitLab CI, and automation

🔧 Advanced Usage

• Performance Guide - Optimization, caching, and scaling
• Feature Discovery - Understanding automatic feature detection
• INFO.yaml Reporting - Project metadata and committer activity tracking
• Troubleshooting - Problem solving and debugging

👨‍💻 Development

• Developer Guide - Architecture, API reference, and contributing
• Template Development - Customizing Jinja2 templates and creating new output formats
• Testing Guide - Test suite documentation
• Migration Guide - Production migration from legacy system

🔍 Development Tools

• Template Audit Script - python scripts/audit_templates.py - Comprehensive audit of all Jinja2 templates to verify field accesses match context builders, preventing runtime errors

---

💻 Installation

Using UV (Recommended)

# Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install dependencies
uv sync

# Run the tool
uv run lf-releng-project-reporting generate --project my-project --repos-path ./repos

Using pip

# Install from source
pip install .

# Run the tool
# Note: repos-path should match the directory created by gerrit-clone-action
# which defaults to the Gerrit server hostname (e.g., ./gerrit.o-ran-sc.org)
lf-releng-project-reporting generate --project O-RAN-SC --repos-path ./gerrit.o-ran-sc.org

→ Detailed Setup Instructions

---

🎯 Common Use Cases

Use Case	Command
Basic report (O-RAN-SC)	lf-releng-project-reporting generate --project O-RAN-SC --repos-path ./gerrit.o-ran-sc.org
Basic report (ONAP)	lf-releng-project-reporting generate --project ONAP --repos-path ./gerrit.onap.org
With caching	lf-releng-project-reporting generate --project O-RAN-SC --repos-path ./gerrit.o-ran-sc.org --cache --workers 8
Check config	lf-releng-project-reporting generate --project O-RAN-SC --repos-path ./gerrit.o-ran-sc.org --dry-run
Get help	lf-releng-project-reporting --help

Note: The --repos-path should point to the directory created by gerrit-clone-action, which uses the Gerrit server hostname as the directory name (e.g., ./gerrit.o-ran-sc.org for O-RAN-SC, ./gerrit.onap.org for ONAP).

---

📊 Output Formats

reports/
  <PROJECT>/
    ├── report_raw.json              # Complete dataset (canonical)
    ├── report.md                    # Markdown report (readable)
    ├── report.html                  # Interactive HTML (sortable tables)
    ├── config_resolved.json         # Applied configuration
    └── <PROJECT>_report_bundle.zip  # Complete bundle

---

🔌 CI/CD Integration

GitHub Actions

- name: Generate Report
  run: |
    uv run lf-releng-project-reporting generate \
      --project "${{ matrix.project }}" \
      --repos-path "./${{ matrix.server }}" \
      --cache \
      --quiet
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

---

🔧 Requirements

• Python: 3.10+ (supports 3.10, 3.11, 3.12, 3.13)
• Dependencies: PyYAML, httpx, Jinja2, typer, rich
• Optional: GitHub token for API features (required for workflow status colors)

GitHub Token Requirements

For full workflow status reporting (colored status indicators), you need a GitHub Personal Access Token (Classic) with these permissions:

Required Scopes:

• ☑ repo - Full repository access (or public_repo for public repositories)
• ☑ actions:read - Read GitHub Actions workflow runs and status

Note: Fine-grained tokens are not supported as they cannot span organizations.

Setup:

# Set environment variable
export GITHUB_TOKEN=ghp_your_token_here
# OR for CI/production:
export CLASSIC_READ_ONLY_PAT_TOKEN=ghp_your_token_here

# Then run the tool
lf-releng-project-reporting generate --project my-project --repos-path ./repos

Create token: https://github.com/settings/tokens

Without a token: The tool detects workflows but shows them as grey (unknown status) instead of colored status indicators.

See also: Configuration Guide for detailed token setup

Jenkins Authentication (Optional)

Some Jenkins servers require authentication to view job information. If you encounter a "returned 0 jobs" error, you need to provide Jenkins credentials.

Setup:

# Generate API token in Jenkins: User → Configure → API Token → Add new Token
export JENKINS_USER="your-username"
export JENKINS_API_TOKEN="your-api-token"

# Then run the tool
lf-releng-project-reporting generate --project my-project --repos-path ./repos

Create token: Log into your Jenkins instance → Your Username → Configure → API Token

Without credentials: The tool will fail with an error if Jenkins requires authentication.

See also: Troubleshooting Guide for detailed setup

---

📖 Key Documentation Files

Topic	Document
Getting Started	docs/GETTING_STARTED.md
Commands	docs/COMMANDS.md
FAQ	docs/FAQ.md
Configuration	docs/CONFIGURATION.md
Usage Examples	docs/USAGE_EXAMPLES.md
Performance	docs/PERFORMANCE.md
Troubleshooting	docs/TROUBLESHOOTING.md
CI/CD Setup	docs/CI_CD_INTEGRATION.md
Developer Guide	docs/DEVELOPER_GUIDE.md

---

💡 Quick Tips

• 🎯 First time? Start with Getting Started Guide
• ⚡ Slow? Add --cache --workers 8 for parallel processing
• 🐛 Issues? Check Troubleshooting Guide
• ❓ Questions? See FAQ
• 📖 Need help? Run lf-releng-project-reporting --help
• 🔍 Developing templates? Run python scripts/audit_templates.py to verify all field accesses

---

🛠️ Development Scripts

Template Audit Script

python scripts/audit_templates.py

This script performs a comprehensive audit of all Jinja2 templates to:

• Extract all field accesses from templates (e.g., repo.field, org.field)
• Analyze context builders to see what fields they provide
• Identify mismatches that could cause "Undefined variable" errors at runtime

When to use:

• Before committing template changes
• After modifying context builders
• When debugging template rendering errors
• During code review to verify template correctness

Output:

• ✅ Green checkmarks - All required fields present
• ⚠️ Warnings - Extra fields exist (safe, unused)
• ❌ Red errors - Missing fields that will cause runtime failures

Example output:

================================================================================
TEMPLATE FIELD ACCESS AUDIT
================================================================================

📄 html/sections/summary.html.j2
  summary:
    - active_count
    - current_count
    - repositories_analyzed

✅ No critical issues found!
   Extra fields are safe - they're unused.

---

🤝 Support

• Documentation: Complete Index
• Issues: GitHub Issues

---

📜 License

Apache-2.0 License - Copyright 2025 The Linux Foundation

---