speedhive-tools 0.9.2

Utilities for MyLaps Event Results API.

Speedhive Tools Core Library

A programmatic Python client, mathematical data processing engine, and command-line (CLI) toolkit for scraping and analyzing MyLaps Speedhive race results.

This package forms the core engine driving the speedhive-tools-ui dashboard service. It supports full offline database caching, outlier detection using Interquartile Range (IQR) analysis, driver consistency scoring, and automatic track record curation.

---

⚙️ Installation

To install the package in development mode along with its dependencies:

git clone https://github.com/ncrosty58/speedhive-tools.git
cd speedhive-tools
python -m venv .venv
source .venv/bin/activate
pip install -e .

---

🏗️ Library Architecture

The library is designed with a clear separation of scraping adapters, database persistence, mathematical model analysis, and workflow orchestration.

src/speedhive/
├── generated/      # Auto-generated OpenAPI HTTP models and client endpoints
├── client.py       # Base HTTP client with endpoint authentication mapping
├── wrapper.py      # SpeedhiveClient high-level client wrapper exposing direct endpoints
├── storage.py      # SpeedhiveStorage SQLite relational persistence cache layer
├── ndjson.py       # Streaming NDJSON line serialization utilities
├── utils/          # Math/text analyzers (outliers, IQR, lap times, record text parsing)
├── analyzers/      # CLI analysis scripts (consistency reports, drivers report)
├── exporters/      # Data extractors (laps exporter, track records, SQLite db dumpers)
├── workflows/      # Multi-step workflows (incremental cache sync, dump loaders)
└── cli/            # Central command-line routing wrapper

---

🐍 Programmatic Python API

1. The Direct API Wrapper (SpeedhiveClient)

SpeedhiveClient wraps raw, autogenerated OpenAPI requests to expose high-level Python endpoints:

from speedhive.wrapper import SpeedhiveClient

client = SpeedhiveClient.create()

# Live scraping of Speedhive properties
org = client.get_organization(30476)
events = client.get_events(30476, limit=10)
sessions = client.get_sessions(event_id=12345)
laps = client.get_laps(session_id=67890)

2. High-Level Scraping & Workflows

Complex recursive scraping loops (such as scanning all track records or finding class-specific records) are decoupled into pure workflows:

from speedhive.workflows.track_records.extract import (
    extract_records_from_api,
    extract_fastest_record_from_api,
)

# Traverse all events in an organization to scrape records
records = extract_records_from_api(client, org_id=30476, limit_events=20)

3. Database Persistence (SpeedhiveStorage)

All results can be stored in a relational SQLite schema for offline data analysis and quick reloading:

from speedhive.storage import SpeedhiveStorage

storage = SpeedhiveStorage("speedhive.db")

# Read cached organization structure
cached_org = storage.get_organization(30476)
cached_laps = storage.get_laps(session_id=67890)

---

💻 Command-Line Interface (CLI)

The CLI acts as a wrapper around the workflow orchestration files. After installing the package, the executable command speedhive is registered.

Core Commands

1. Sync Cache Database

Scrapes and stores organization results to a local database cache:

speedhive sync-org --org 30476 --mode incremental --recent-backfill-events 5

Options:

• --org: Organization ID.
• --mode: incremental (only scrapes new events) or full (re-scrapes all history).
• --db-path: Custom path to SQLite file (defaults to web_data/speedhive.db).

2. Report Driver Consistency

Analyzes driver consistency ranks across races using standard deviations and coefficient of variation (CV):

speedhive report-consistency --org 30476 --min-laps 15 --top 20

Options:

• --min-laps: Minimum laps required to calculate consistency.
• --threshold: Speed outlier rejection threshold (default 0.85).
• --ignore-outliers: Filter mathematical outliers using IQR calculations before ranking.

3. Extract Driver Lap Times

Outputs raw lap times and statistics for a specific competitor:

speedhive extract-driver-laps --org 30476 --driver "John Doe" --ignore-outliers

4. Import / Export Database Snapshots

Exports or imports raw databases into unified offline folder hierarchies containing manifests and tables:

speedhive export-db-dump --org 30476 --output-dir ./snapshots
speedhive import-dump --org 30476 --dump-dir ./snapshots

5. Curate Track Records

Syncs local track records and outputs candidate review lists:

speedhive scan-track-records --org 30476
speedhive refresh-track-records --org 30476

---

🧪 Running Tests

Ensure all core library components (HTTP wrapper, persistence adapters, and outlier processors) pass local validations:

pytest tests/