FQG - Function Quest Game

Reinforcement Learning Training Environment for Symphony Conductor

FQG is a TorchRL-based training environment that simulates extension orchestration challenges with known scoring rubrics. It trains AI models (like Symphony Conductor) to make intelligent orchestration decisions through narrative-based challenges.

✨ Features

• 🎯 Narrative-Based Challenges - Story-driven orchestration problems with clear goals
• 📊 Ground Truth Validation - Known optimal paths for training validation
• 📈 Comprehensive Metrics - Detailed tracking of training progress
• 🔧 Hierarchical Configuration - Flexible YAML-based configuration system
• 🤖 AI-Powered Generation - Create challenges from natural language descriptions
• 🚀 TorchRL Integration - Standard RL environment interface
• 💾 Lazy Loading - Memory-efficient generator-based collections
• 🎓 Progressive Difficulty - Curriculum learning with level progression

📦 Installation

Quick Start (uv - recommended)

# Install uv if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install FQG
cd fqg
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -e ".[dev]"

Using pip

cd fqg
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -e ".[dev]"

Verify Installation

python -c "from fqg import FQG; print('✓ FQG installed successfully!')"

🚀 Quick Start

Basic Usage

from fqg import FQG

# Initialize FQG
fqg = FQG(tier_name="vivace", metrics_enabled=True)

# Get a level
level = fqg.level(1)

# Get a challenge
challenge = level.challenges().get(1)

# Run an episode
tensordict = challenge.reset()
done = False

while not done:
    # Your policy selects action
    action = select_action(tensordict)
    tensordict['action'] = action
    
    # Step environment
    tensordict = challenge.step(tensordict)
    done = tensordict['done'].item()

# View metrics
metrics = fqg.metrics()
summary = metrics.summary(tier_name="vivace")
print(f"Success rate: {summary['success_rate']:.2%}")

Configuration Overrides

# Override YAML configuration with constructor parameters
fqg = FQG(
    tier_name="vivace",
    max_steps=30,           # Override max steps per episode
    step_penalty=-0.02,     # Override step penalty
    eval_fraction=0.20      # Override evaluation set fraction
)

Filtering and Sampling

# Filter challenges
easy_challenges = level.challenges().filter(complexity="easy")
household = level.challenges().filter(tags=["household"])

# Sample challenges
sampled = level.challenges().sample(n=5, strategy="uniform", seed=42)

# Chain operations
easy_household = level.challenges().filter(complexity="easy").filter(tags=["household"])

📚 Documentation

Getting Started

• Examples - 7 runnable Python scripts covering common use cases
• Tutorial Notebooks - Interactive Jupyter notebooks for learning
• Troubleshooting Guide - Solutions to common issues

Reference Documentation

• API Documentation - Complete API reference with examples
• Configuration Schema - YAML configuration reference
• Architecture Guide - System design and patterns
• Metrics System - Metrics collection and analysis
• Challenge Generation - AI-powered generation

🏗️ Project Structure

fqg/
├── __init__.py              # Public API exports
├── core/                    # Core infrastructure
│   ├── base.py              # FQGConfigBase, FQGError
│   ├── fqg.py               # FQG main entry point
│   ├── loader.py            # ConfigLoader
│   └── yaml_utils.py        # YAML parsing utilities
├── global_config/           # Global configuration module
├── tier/                    # Tier configuration module
├── level_collection/        # Level collection module
├── level/                   # Level module
├── challenge/               # Challenge module (TorchRL environment)
├── extension/               # Extension manifest module
├── path/                    # Path configuration module
├── quality/                 # Quality/scoring module
├── execution/               # Extension execution module
├── scoring/                 # Reward computation module
├── state/                   # Episode state management
├── metrics/                 # Metrics collection and analysis
└── generation/              # AI-powered challenge generation

tests/
├── unit/                    # Unit tests (80% of tests)
├── integration/             # Integration tests (15%)
└── acceptance/              # End-to-end tests (5%)

examples/                    # Runnable usage examples
notebooks/                   # Interactive tutorials
docs/                        # Complete documentation

🔧 Configuration

FQG uses a five-level hierarchical configuration system:

Global Config (.fqg/config.yaml)
  ↓
Tier Config (.fqg/game/<tier_name>/config.yaml)
  ↓
Levels Collection Config (.fqg/game/<tier_name>/levels/config.yaml)
  ↓
Level Config (.fqg/game/<tier_name>/levels/level1/config.yaml)
  ↓
Challenge Config (.fqg/game/<tier_name>/levels/level1/challenge1.yaml)
  ↓
Constructor Parameters (highest priority)

Priority Order: Constructor > Challenge > Level > Collection > Tier > Global

See Configuration Schema for complete reference.

🧪 Development

Run Tests

# All tests
pytest

# With coverage
pytest --cov=fqg --cov-report=html

# Specific test file
pytest tests/unit/core/test_fqg.py

# Specific test
pytest tests/unit/core/test_fqg.py::test_fqg_initialization

Type Checking

mypy fqg

Linting and Formatting

# Check linting
ruff check .

# Fix linting issues
ruff check --fix .

# Format code
ruff format .

# Check formatting
ruff format --check .

All Quality Checks

# Run all checks at once
ruff check . && ruff format --check . && mypy fqg && pytest

Code Complexity

# Check complexity with radon
radon cc fqg -a -nb

# Check complexity with lizard
lizard fqg

📊 Metrics

FQG automatically collects comprehensive metrics:

# Enable metrics
fqg = FQG(tier_name="vivace", metrics_enabled=True)

# Run training...

# Query metrics
metrics = fqg.metrics()

# Get recent episodes
episodes = metrics.episodes(tier_name="vivace", level_index=1, limit=100)

# Get summary statistics
summary = metrics.summary(tier_name="vivace")
print(f"Total episodes: {summary['total_episodes']}")
print(f"Success rate: {summary['success_rate']:.2%}")
print(f"Average reward: {summary['avg_reward']:.2f}")

# Get path distribution
path_dist = metrics.path_distribution(tier_name="vivace", level_index=1)

Metrics are stored in SQLite (.fqg/metrics.db) for persistence and analysis.

🤖 Challenge Generation

Generate challenges from natural language descriptions:

from fqg.generation import FQGGenerator

# Initialize generator
generator = FQGGenerator(model="deepseek-r1")

# Generate challenge
challenge_config = generator.generate(
    description="User wants to send an email with attachment",
    domain="enterprise",
    complexity="medium",
    noise_ratio=0.2
)

# Save to YAML
challenge_config.save(".fqg/game/vivace/levels/level1/email_challenge.yaml")

🎯 Use Cases

Training RL Agents

for episode in range(10000):
    challenge = sample_challenge()
    state = challenge.reset()
    
    while not done:
        action = agent.select_action(state)
        state, reward, done, info = challenge.step(action)
    
    agent.update(episode_data)

Curriculum Learning

current_level = 1
while current_level <= max_levels:
    success_rate = train_on_level(current_level)
    
    if success_rate > 0.70:  # 70% success threshold
        current_level += 1

Challenge Design

# Create custom challenges
generator = FQGGenerator()
challenge = generator.generate(
    description="Your custom scenario",
    domain="your_domain",
    complexity="medium"
)

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run quality checks (ruff check . && mypy fqg && pytest)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Guidelines

• Follow existing code style (enforced by ruff)
• Add type hints to all functions
• Write tests for new features
• Update documentation
• Keep commits focused and atomic

📈 Project Status

• ✅ Phase 0-11: Complete (589 tests passing)
• 🚧 Phase 12: Documentation & Polish (in progress)
• 📊 Test Coverage: 80%+
• 🎯 Type Coverage: 100% (mypy strict mode)
• 📝 Documentation: Comprehensive

🐛 Troubleshooting

Common issues and solutions:

Import Error

# Reinstall in development mode
pip install -e .

Configuration Error

# Verify .fqg directory structure
ls -la .fqg/game/vivace/levels/level1/

Metrics Database Locked

# Close other FQG instances or delete database
rm .fqg/metrics.db

See Troubleshooting Guide for more solutions.

📄 License

MIT License - see LICENSE file for details.

🙏 Acknowledgments

• Built with TorchRL for RL environment interface
• Uses Pydantic for configuration validation
• Inspired by OpenAI Gym and Gymnasium

📞 Support

• Documentation: docs/
• Examples: examples/
• Tutorials: notebooks/
• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

Made with ❤️ for the Symphony project