devtrack-sdk 0.4.2

Middleware-based API analytics and observability tool for FastAPI and Django

Comprehensive request tracking and analytics toolkit for FastAPI and Django applications

Built for developers who care about API usage, performance, and observability

📖 View Documentation | 🚀 Quick Start | 🛠️ CLI Toolkit

---

📋 Table of Contents

• 🌟 Features
• 🚀 Quick Start
• 📦 Installation
• 🔧 Framework Integration
• 🛠️ CLI Toolkit
• 🗄️ Database Integration
• 📊 API Endpoints
• ⚙️ Configuration
• 🔍 Advanced Usage
• 🔐 Security
• 📈 Performance
• 📚 Documentation
• 🤝 Contributing
• 📄 License

---

🌟 Features

✨ Core Features

• Zero Configuration: Works out of the box with sensible defaults
• Dual Framework Support: FastAPI and Django middleware
• Real-Time Dashboard: Interactive dashboard at /__devtrack__/dashboard with live metrics
• Advanced Querying: Filter and search logs with multiple criteria
• Export Capabilities: Export logs to JSON or CSV formats
• Health Monitoring: System health checks and component status
• CLI Toolkit: 8 powerful commands for managing your DevTrack instance

🗄️ Database Features

• DuckDB Integration: High-performance embedded database
• Persistent Storage: Data survives application restarts
• Advanced Analytics: Built-in statistical analysis

🎯 Tracking Capabilities

• Comprehensive Logging: 15+ fields per request
• Performance Metrics: Duration, response size, latency tracking
• User Context: User ID, role, and authentication data
• Request Details: Path parameters, query params, request body
• Client Information: IP address, user agent, referer

---

🚀 Quick Start

1. Install DevTrack SDK

pip install devtrack-sdk

2. Choose Your Framework

FastAPI

from fastapi import FastAPI
from devtrack_sdk.middleware import DevTrackMiddleware
from devtrack_sdk.controller import router as devtrack_router

app = FastAPI()
app.include_router(devtrack_router)
app.add_middleware(DevTrackMiddleware)

Django

# settings.py
MIDDLEWARE = [
    # ... other middleware
    'devtrack_sdk.django_middleware.DevTrackDjangoMiddleware',
]

# urls.py
from devtrack_sdk.django_urls import devtrack_urlpatterns

urlpatterns = [
    # ... your other URL patterns
    *devtrack_urlpatterns,
]

3. Initialize Database

devtrack init --force

4. Access Dashboard

Once your app is running, visit:

http://localhost:8000/__devtrack__/dashboard

The dashboard provides real-time insights into your API performance with interactive charts and metrics.

5. Start Monitoring (Optional)

devtrack monitor --interval 3

---

📦 Installation

Prerequisites

• Python 3.10 or higher
• FastAPI or Django application

Install from PyPI

pip install devtrack-sdk

Install from Source

git clone https://github.com/mahesh-solanke/devtrack-sdk.git
cd devtrack-sdk
pip install -e .

Dependencies

• fastapi>=0.90 - FastAPI framework support
• django>=4.0.0 - Django framework support
• httpx>=0.24 - HTTP client for CLI
• starlette>=0.22 - ASGI framework
• rich>=13.3 - Rich CLI interface
• typer>=0.9 - CLI framework
• duckdb>=1.1.0 - Embedded database

---

🔧 Framework Integration

FastAPI Integration

Basic Setup

from fastapi import FastAPI
from devtrack_sdk.middleware import DevTrackMiddleware
from devtrack_sdk.controller import router as devtrack_router

app = FastAPI(title="My API")
app.include_router(devtrack_router)
app.add_middleware(DevTrackMiddleware)

Advanced Configuration

app.add_middleware(
    DevTrackMiddleware,
    exclude_path=['/docs', '/redoc', '/health']
)

Django Integration

Basic Setup

# settings.py
MIDDLEWARE = [
    # ... other middleware
    'devtrack_sdk.django_middleware.DevTrackDjangoMiddleware',
]

# urls.py
from devtrack_sdk.django_urls import devtrack_urlpatterns

urlpatterns = [
    # ... your other URL patterns
    *devtrack_urlpatterns,
]

For custom middleware and advanced configurations, see the documentation.

---

🛠️ CLI Toolkit

DevTrack SDK includes a comprehensive CLI toolkit with 8 powerful commands:

📦 Version Information

devtrack version

Shows SDK version, framework support, database type, and CLI features count.

🗄️ Database Management

Initialize Database

devtrack init --force

Creates a new DuckDB database with progress indicators and shows database information.

Reset Database

devtrack reset --yes

Deletes all log entries with confirmation prompt (skip with --yes flag).

📤 Export Capabilities

# Export to JSON
devtrack export --format json --limit 1000 --output-file logs.json

# Export to CSV
devtrack export --format csv --limit 500 --output-file logs.csv

# Export with filters
devtrack export --status-code 404 --days 7 --format json

🔍 Advanced Querying

# Basic query
devtrack query --limit 50

# Filter by status code
devtrack query --status-code 404 --days 7

# Filter by HTTP method
devtrack query --method POST --verbose

# Filter by path pattern
devtrack query --path-pattern "/api/users" --limit 20

📊 Real-time Monitoring

# Start monitoring with 3-second intervals
devtrack monitor --interval 3 --top 15

# Monitor with custom database path
devtrack monitor --db-path /custom/path/db.db --interval 5

📈 Statistics

# Show stats from database
devtrack stat

# Show stats from HTTP endpoint
devtrack stat --endpoint

# Show top 10 endpoints sorted by hits
devtrack stat --top 10 --sort-by hits

# Show top 5 endpoints sorted by latency
devtrack stat --top 5 --sort-by latency

🏥 Health Checks

# Check database health
devtrack health

# Check database and HTTP endpoint health
devtrack health --endpoint

📚 Help

# Show comprehensive help
devtrack help

# Show help for specific command
devtrack init --help
devtrack query --help

---

🗄️ Database Integration

DuckDB Features

• High Performance: Embedded database with excellent query performance
• Zero Configuration: No external database server required
• ACID Compliance: Reliable data storage
• SQL Support: Full SQL query capabilities
• Cross-Platform: Works on Windows, macOS, and Linux

Automatic Database Setup

DevTrack SDK automatically creates and manages the DuckDB database. No manual setup required - just install and configure the middleware.

Stored Data Includes:

• Request Details: Path, method, status code, timestamp
• Performance Metrics: Duration, response size, latency
• Client Information: IP address, user agent, referer
• User Context: User ID, role, authentication data
• Request Data: Query parameters, path parameters, request body
• Trace Information: Unique request identification

Database Management

DevTrack SDK provides comprehensive database management through:

• CLI Commands: devtrack init, devtrack reset, devtrack export
• API Endpoints: /__devtrack__/stats, /__devtrack__/logs
• Django Management: python manage.py devtrack_init, devtrack_stats
• Python API: Direct database operations for advanced use cases

---

📊 API Endpoints

GET /devtrack/dashboard

Serves the built-in real-time dashboard with interactive charts and metrics.

Features:

• Traffic overview with time-series charts
• Error trends and top failing routes
• Performance metrics (p50/p95/p99 latency)
• Consumer segmentation analysis
• Searchable request logs table
• Auto-refresh functionality

Access: Visit http://localhost:8000/__devtrack__/dashboard after starting your application.

GET /devtrack/stats

Returns comprehensive statistics and logs from the database.

Query Parameters:

• limit (int, optional): Limit number of entries returned
• offset (int, default: 0): Offset for pagination
• path_pattern (str, optional): Filter by path pattern
• status_code (int, optional): Filter by status code

Response: Returns summary statistics and log entries array.

DELETE /devtrack/logs

Delete logs from the database with various filtering options.

Query Parameters:

• all_logs (bool, default: false): Delete all logs
• path_pattern (str, optional): Delete logs by path pattern
• status_code (int, optional): Delete logs by status code
• older_than_days (int, optional): Delete logs older than N days
• log_ids (str, optional): Comma-separated list of log IDs to delete

DELETE /devtrack/logs/{log_id}

Delete a specific log by its ID.

Response Format

{
    "message": "Successfully deleted log with ID 123",
    "deleted_count": 1,
    "log_id": 123
}

GET /devtrack/metrics/traffic

Get traffic metrics over time.

Query Parameters:

• hours (int, default: 24): Number of hours to look back

Response: Returns traffic counts grouped by time intervals.

GET /devtrack/metrics/errors

Get error trends and top failing routes.

Query Parameters:

• hours (int, default: 24): Number of hours to look back

Response: Returns error trends over time and top failing routes.

GET /devtrack/metrics/perf

Get performance metrics (p50/p95/p99 latency).

Query Parameters:

• hours (int, default: 24): Number of hours to look back

Response: Returns latency percentiles over time and overall statistics.

GET /devtrack/consumers

Get consumer segmentation data.

Query Parameters:

• hours (int, default: 24): Number of hours to look back

Response: Returns consumer segments with request counts, error rates, and latency metrics.

For detailed API documentation, see the documentation.

---

⚙️ Configuration

Environment Variables

# Database configuration
DEVTRACK_DB_PATH=/custom/path/devtrack_logs.db

# Middleware configuration
DEVTRACK_EXCLUDE_PATHS=/health,/metrics,/admin
DEVTRACK_MAX_ENTRIES=10000

# Environment
ENVIRONMENT=production

Exclude Paths

You can exclude specific paths from tracking:

# FastAPI
app.add_middleware(
    DevTrackMiddleware,
    exclude_path=['/docs', '/redoc', '/health']
)

# Django
class CustomDevTrackMiddleware(DevTrackDjangoMiddleware):
    def __init__(self, get_response=None):
        exclude_paths = ['/health', '/metrics', '/admin']
        super().__init__(get_response, exclude_path=exclude_paths)

Custom Configuration

# Custom middleware configuration
class ConfigurableDevTrackMiddleware(DevTrackMiddleware):
    def __init__(self, app, config=None):
        self.config = config or {}
        exclude_paths = self.config.get('exclude_paths', [])
        super().__init__(app, exclude_path=exclude_paths)

---

🔍 Advanced Usage

Custom Data Extraction

DevTrack SDK allows custom data extraction by extending the base extractor. You can add custom fields like request IDs, app versions, and environment information to your logs.

Custom Database Operations

DevTrack SDK provides a flexible database interface that can be extended for custom operations like date range queries, performance metrics, and advanced analytics.

Integration with Monitoring Tools

DevTrack SDK integrates seamlessly with popular monitoring tools like Prometheus, Grafana, and Datadog. You can extend the middleware to export metrics and integrate with your existing monitoring infrastructure.

---

🔐 Security

Security Features

• No API Keys Required: Basic usage doesn't require authentication
• Automatic Data Filtering: Sensitive data is automatically filtered
• Configurable Exclusions: Exclude sensitive paths from tracking
• Environment Awareness: Different configurations for different environments

Sensitive Data Filtering

DevTrack SDK automatically filters sensitive fields like passwords, tokens, and API keys. You can extend the filtering to include additional sensitive fields specific to your application.

Access Control

DevTrack SDK endpoints can be protected with authentication and authorization. You can require login, admin access, or custom permissions for accessing statistics and log data.

Production Security Recommendations

1. Environment Variables: Use environment variables for sensitive configuration
2. Access Control: Implement proper authentication for stats endpoints
3. Path Exclusions: Exclude sensitive paths from tracking
4. Monitoring: Monitor the stats endpoint for unusual activity
5. Data Retention: Implement data retention policies
6. Encryption: Consider encrypting sensitive log data

---

📈 Performance

Performance Characteristics

• Low Overhead: Minimal impact on request processing time
• Non-blocking: Asynchronous operations don't block request handling
• Efficient Storage: DuckDB provides excellent query performance
• Memory Efficient: Configurable limits prevent memory issues

Performance Monitoring

DevTrack SDK includes built-in performance monitoring capabilities. You can track request duration, identify slow endpoints, and monitor application performance in real-time.

Optimization Tips

1. Exclude High-Traffic Paths: Exclude health checks and metrics endpoints
2. Limit Stored Entries: Set reasonable limits for in-memory storage
3. Use Database: Use DuckDB for persistent storage instead of in-memory
4. Batch Operations: Batch database operations when possible
5. Monitor Performance: Use the built-in performance monitoring

Memory Management

DevTrack SDK provides configurable memory management options. You can set limits on stored entries, implement custom cleanup strategies, and optimize memory usage for your specific requirements.

Production Environment Variables

ENVIRONMENT=production
DEVTRACK_DB_PATH=/var/lib/devtrack/logs.db
DEVTRACK_EXCLUDE_PATHS=/health,/metrics,/admin
DEVTRACK_MAX_ENTRIES=10000
LOG_LEVEL=INFO

---

📚 Documentation

• GitHub Repository: https://github.com/mahesh-solanke/devtrack-sdk
• Documentation Files: https://github.com/mahesh-solanke/devtrack-sdk/tree/main/docs
• FastAPI Integration: docs/fastapi_integration.md
• Django Integration: docs/django_integration.md
• Examples: examples/

---

🤝 Contributing

We welcome contributions! Here's how you can help:

Development Setup

git clone https://github.com/mahesh-solanke/devtrack-sdk.git
cd devtrack-sdk
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
pip install -e .
pip install pytest flake8 black isort pre-commit
pre-commit install

Running Tests

pytest
pytest --cov=devtrack_sdk

Submitting Changes

1. Fork the repository
2. Create a feature branch (git checkout -b feat/awesome-feature)
3. Make your changes
4. Run tests (pytest)
5. Commit your changes (git commit -m '✨ Add awesome feature')
6. Push to the branch (git push origin feat/awesome-feature)
7. Open a Pull Request

For detailed contributing guidelines, see CONTRIBUTING.md.

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

📅 Release Timeline

• See the full roadmap and release plan in RoadMap

---

🎉 Acknowledgements

• FastAPI: Inspired by FastAPI's middleware design
• Django: Django's middleware system
• DuckDB: High-performance embedded database
• Rich: Beautiful CLI interface
• Typer: Modern CLI framework
• Open Source Community: For tooling and inspiration

---

📞 Support

• GitHub Issues: https://github.com/mahesh-solanke/devtrack-sdk/issues
• GitHub Discussions: https://github.com/mahesh-solanke/devtrack-sdk/discussions
• LinkedIn Company: DevTrackHQ
• Email: Contact me

---

Made with ❤️ by Mahesh Solanke