🧪 Latest features land on the dev branch first. main is the stable, versioned release line.

---

🧭 Why TSFLab

Forecasting papers multiply every year, yet each one can compare against only a few baselines, re-run under its own code and data conventions. Keeping hundreds of methods in one benchmark by hand, verifying their code, and re-evaluating them on new data no longer scales, and fixed benchmark snapshots cannot show how a model holds up as the world moves on.

TSFLab automates that loop. Coding agents read new papers, implement them behind one verified interface, and evaluate them under one protocol on static datasets and on rolling real-time tracks that refresh every week. TSFLab ships no agent of its own: it is the infrastructure (catalog, contracts, data, protocols, evidence) that any coding agent operates through declarative skills.

---

✨ What is inside

Module	What it does
📚 Paper reading	Scans arXiv and Hugging Face Papers, deduplicates against the catalog, and records each paper's structure, equations, and pinned official code
🧩 Code & interface	217 methods as peers in one flat catalog, composed from 53 shared components, one forecasting signature, a 13-check verification battery with pinned-reference comparison; every card opens with a tagline, tags, and a six-slot composition
🗃️ Data	101 dataset presets (77 conventional, including the GIFT-Eval family, and 24 spatiotemporal or covariate; 16 of them are frozen releases of the real-time tracks), each with a card and one TSFLab protocol; plus 16 rolling real-time tracks (stocks, traffic, air quality, weather, grid, solar)
⚙️ Experiments	Declarative TOML sweeps, pre-run validation, seeds, budgets, GPU leases, queues, and recovery; tsf data analyze profiles a dataset and tsf model compose dry-runs a recombination for AutoResearch
🏆 Release & compare	Run records → submissions → a leaderboard recomputed from evidence; weights as pinned hf:// bundles

---

🏁 Quick start

Work in the repository with an agent:

git clone https://github.com/Diaugeia/TSFLab.git
cd TSFLab
codex          # or any other coding agent

> Set up the environment for my GPU.
> Benchmark DLinear, PatchTST and iTransformer on ETTh1 and give me a leaderboard.
> Implement the paper at <arXiv URL> as a catalog model and verify it.
> Forecast this week's traffic round with PatchTST and submit it.

Or install the framework and scaffold your own project:

uv tool install "git+https://github.com/Diaugeia/TSFLab"   # provides `tsf`
tsf init my-forecasting-project --modules data,models,experiments,release,autoresearch
cd my-forecasting-project        # default: all modules; also writes the agent guide and skills
tsf data download etth1          # pinned, checksum-verified from the Hub
tsf run configs/runs/example.toml --dry-run
tsf run configs/runs/example.toml

Scaffolded run configs inherit the installed catalog through tsflab:// paths, so upgrading TSFLab upgrades their defaults. Install by module with extras: tsflab[data], [models], [experiments], [hub], [realtime], [autoresearch], or [all].

Read the catalog progressively:

uv run tsf catalog                                      # counts, then the next commands
uv run tsf catalog search "reversible normalization"    # L0: one line per match
uv run tsf catalog show PatchTST                          # L1: facts, composition, key ideas
uv run tsf catalog show revin --depth 2               # L2: full card (--depth 3: paths)
uv run tsf data analyze etth1                        # profile a dataset, map it to components
uv run tsf realtime list

Search accepts --kind model|component|dataset and, for models, --capability. See docs/en/workflows.md.

---

📈 Rolling real-time evaluation

Each week the weekly workflow releases new observations (versioned on the Hugging Face Hub), scores the rounds whose target window is now observed, and opens a new round. Forecasts must be submitted before their targets exist, so no model, including ours, can have seen its evaluation data.

Track	Data	Setting	Horizon
stock_hs300	CSI-300 constituents, daily log returns (AKShare)	time series	5 trading days
stock_nasdaq100	NASDAQ-100 constituents, daily log returns (Nasdaq API, Yahoo fallback)	time series	5 trading days
stock_sp500	S&P 500 constituents, daily log returns (Nasdaq API, Yahoo fallback)	time series	5 trading days
traffic_pems_{ba,la,sac,sb}	Caltrans PeMS Districts 4, 7, 3, 8: hourly flow at 2,472 / 1,926 / 801 / 1,105 stations	spatiotemporal	24 h
air_openaq_cn	OpenAQ hourly PM2.5, government monitors in China	spatiotemporal	24 h
air_openaq_{us,eu}	OpenAQ hourly PM2.5, US / European reference monitors	spatiotemporal	24 h
air_airnow_us	EPA AirNow hourly PM2.5, US monitors (no key)	spatiotemporal	24 h
weather_openmeteo_temp, solar_openmeteo_ghi	Open-Meteo hourly temperature at 82 US/EU cities, irradiance at 55 PV sites	spatiotemporal	24 h
grid_ercot	ERCOT hourly load in 8 weather zones	spatiotemporal	24 h
grid_eia_us, solar_eia_us	EIA-930 hourly demand / solar generation per US balancing authority	spatiotemporal	24 h

uv run tsf realtime forecast --track traffic_pems_sb --model DLinear   # produce a forecast
uv run tsf realtime replay --track traffic_pems_sb --end 2023-12-25 --weeks 12   # backtest the protocol

Each track is also a frozen static dataset (rt_<track>, tsf catalog list --kind dataset). See docs/en/realtime.md.

---

🤝 Contributing

There are three ways to take part:

1. Propose a method or report a problem — open an Add a paper or Report a problem or ask a question issue. An agent triages it; accepted papers and reproducible fixes are checked with tsf repo check, reviewed by a second agent pass, and merged by the agent workflow.
2. Submit results — add a submission.json under apps/web/submissions/ (see SUBMITTING.md); CI validates it against the contract before it can reach the leaderboard.
3. Forecast a real-time round — add forecasts/<YourModel>.json to an open round before its deadline.

The literature is also scanned weekly by the agent workflow. See CONTRIBUTING.md for code contributions.

---

📖 Documentation

• Workflow documentation: catalog, models, data, AutoResearch, verification, experiments
• Projects and the Hub: tsf init, hf:// assets, weights bundles
• Real-time tracks: rounds, forecasts, scoring, weekly automation

Exact command options stay in tsf <command> --help.

---

🗂️ Repository layout

Path	Contents
src/tsflab/	One package per module, mirrored by the CLI: catalog (cards, registries, verification), data, models (flat catalog, _components, _slots), experiments (config, runner, evaluation, execution), release (Hub, submissions), realtime, research (rounds, recombination), agent (assets, tasks, tsf init), core (contracts), cli
configs/, catalog/, verification/	Run, model, and dataset presets, real-time track configs, configs/fixtures/ (smoke and synthetic test inputs, not datasets), dataset cards, verification evidence
dataset/	Local dataset bytes fetched with tsf data download (not packaged)
apps/web/	TSFLab Leaderboard: static site, submission pipeline, submissions/, real-time rounds
experiments/	Local research workspace; only */scripts/ is tracked

---

📜 License

TSFLab is released under the MIT License. Copyright © 2026 Diaugeia.AI.

Ordinary paper architectures are maintained locally under the project license. Released pretrained foundation models use optional official packages and unchanged checkpoints through the offline runtime boundary; see THIRD_PARTY_NOTICES.md. Real-time data remain subject to their providers' terms (Caltrans PeMS, OpenAQ, exchange data via AKShare).

---

⭐ Star History