Fracast-0

Fracast-0 is a compact time-series foundation model for zero-shot forecasting. The released model has 85,001 parameters and keeps a single full-resolution context stream. Its core idea is simple: a causal dilated-convolution block is reused across a geometric ladder of time scales, with a small scale-conditioning signal at each level. The model is attention-free, fully convolutional, and designed to make the relationship between parameter count and temporal context explicit.

This repository is the self-contained pretraining release. It contains:

• the Fracast-0 model and its reusable modules;
• the public-data download and fast-corpus preparation flow;
• the TinyCast synthetic-corpus build/conversion wrapper;
• a CPU/MPS smoke recipe and the full CUDA pretraining recipe;
• a single configuration-driven entry point, main.py.

The repository includes the small release checkpoints under weights/. Downloaded datasets, training runs, evaluation products, and runtime logs are created under data/, output/, and tmp/, all of which are gitignored.

Installation

Install the Python package and its bundled Fracast-0 checkpoints with one command:

python -m pip install fracast

The package is self-contained: FracastModel.from_pretrained() reads the FP32 or W8 weights from the installed wheel without downloading a model repository. The GitHub workflow builds every push to main and publishes a new pyproject.toml version to PyPI when that version is not already online.

Model

The default model is configured in config/base.yaml.

The implementation is split by responsibility:

• model/fracast/model.py: block ordering and model assembly;
• module/fracast/: reusable Fracast-0 blocks;
• module/periodic/: period detection, phase encoding, seasonal fill;
• dataport/: corpus readers and the training data port;
• pipeline/: the two supported flows, build_corpus and train;
• script/data/: public-data download and TinyCast synthetic-shard tooling;
• script/corpus/: corpus conversion, merge, and audit tools.

Requirements

Python 3.10 or newer is recommended. TinyCast's upstream package requires Python 3.10+, so the TinyCast synthetic-corpus path needs it.

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

The core requirements are installable on Python 3.9. Rebuilding the TinyCast synthetic shards additionally requires Python 3.10+ and the pinned upstream package:

python -m pip install -r requirements-tinycast.txt

The training code chooses CUDA > MPS > CPU automatically. CUDA is the intended pretraining device. MPS and CPU are supported for the smoke recipe and for local development. Mixed-precision autocast is enabled only on CUDA.

One-command reproduction

After creating and activating the virtual environment, the complete public data path and the full recipe are:

./script/reproduce.sh prepare-data --workers 8
./script/reproduce.sh train

The first command downloads the five pinned source datasets, converts and merges all five fast corpora, builds the four TinyCast synthetic shards, and runs the corpus audits. It is resumable: completed corpus roots are skipped, while an incomplete generated root is rebuilt from its raw source.

For a quick local check that downloads nothing:

./script/reproduce.sh run --profile smoke

Both commands use PYTHON=python by default. Set PYTHON=.venv/bin/python if the interpreter is not already active.

Quick smoke test

The underlying smoke commands are also directly runnable:

python script/data/make_demo_corpus.py
python main.py config/corpus/demo.yaml
python main.py config/fracast/pretrain_smoke.yaml

The smoke recipe writes to tmp/fracast-smoke/. It exercises the same Arrow-to-fast-corpus path, model builder, data loader, rollout loss, checkpoint writer, and resume logic used by the full recipe.

To test resume without changing the recipe:

python main.py config/fracast/pretrain_smoke.yaml \
  -o train.total_steps=2 -o train.resume=false
python main.py config/fracast/pretrain_smoke.yaml \
  -o train.total_steps=4 -o train.resume=auto

The second command must report a restored optimizer state and random-number state, then continue from step 2.

Public pretraining data

The five source datasets are pinned to immutable Hugging Face revisions in script/data/download_public_data.py. Their approximate download sizes are:

The complete download is about 2.6 TiB before conversion. Check available storage and network capacity before starting:

python script/data/download_public_data.py --list
python script/data/download_public_data.py pretrain lotsa chronos boom fev
# or: python script/data/download_public_data.py --all

Downloads go to:

data/pretrain_full/
data/lotsa_full/
data/chronos_full/
data/boom/
data/fev/

Convert raw data to fast corpus

Each source is converted independently. The conversion produces contiguous float32 arrays, int64 offsets, compact indices, and a manifest. The conversion is CPU/IO work and does not need a GPU:

python main.py config/corpus/pretrain_full.yaml --workers 32
python main.py config/corpus/lotsa_full.yaml --workers 32
python main.py config/corpus/chronos_full.yaml --workers 32
python main.py config/corpus/boom_full.yaml --workers 8
python main.py config/corpus/fev_full.yaml --workers 4

After all parts finish, merge each source into a global index:

python script/corpus/merge_corpus_parts.py --corpus data/corpus_fast/pret
python script/corpus/merge_corpus_parts.py --corpus data/corpus_fast/lotsa
python script/corpus/merge_corpus_parts.py --corpus data/corpus_fast/chronos
python script/corpus/merge_corpus_parts.py --corpus data/corpus_fast/boom
python script/corpus/merge_corpus_parts.py --corpus data/corpus_fast/fev

The converter only accepts complete p*/index.npz parts. A partially written part is skipped by the training reader and can be rebuilt without touching the other parts.

TinyCast synthetic shards

TinyCast publishes four synthetic shards. The official builder is CUDA-only and requires the optional TinyCast package; the wrapper below keeps that requirement explicit and then converts the shards to the Fracast-0 fast-corpus layout:

python script/data/prepare_tinycast_synth.py

If the shards were already built, convert them without rebuilding:

python script/data/prepare_tinycast_synth.py --skip-build

The output is data/corpus_fast/tinycast_synth/. The wrapper preserves the official per-series scale sidecar used by the committing objective.

Full pretraining

After all six corpus roots exist, start the full recipe through the same reproduction entry point:

./script/reproduce.sh train

The full recipe uses:

• six corpus roots: pret, lotsa, chronos, boom, fev, tinycast_synth;
• micro-batch 512 with 8 gradient-accumulation steps;
• effective batch 4,096;
• 36,621 optimizer steps;
• CUDA bf16 autocast, torch.compile, fused AdamW, TF32, and cuDNN benchmark;
• checkpoints every 1,145 steps plus best.pt, last.pt, and the final averaged checkpoint.

On a Mac, use the local recipe instead of the full CUDA recipe:

./script/reproduce.sh train --profile local

config/fracast/pretrain_local.yaml is the full-data CPU/MPS recipe for a machine that has already prepared all six corpus roots; it uses a smaller micro-batch and disables CUDA-only optimizations. The smoke profile remains the fastest way to verify installation without downloading the corpora.

Resume

Training checkpoints contain the model, head, optimizer state, all random streams, global step, and best validation value. The default is:

train:
  resume: auto

Resume from the current output directory:

python main.py config/fracast/pretrain_full.yaml

Start a fresh run explicitly:

python main.py config/fracast/pretrain_full.yaml -o train.resume=false

Resume from a named checkpoint:

python main.py config/fracast/pretrain_full.yaml \
  -o train.resume=output/fracast-0-full/last.pt

Configuration and overrides

Every flow is selected by _run.kind in YAML. main.py only handles discovery, dry runs, overrides, and optional process-level parallelism:

python main.py config/corpus --list
python main.py config/corpus/demo.yaml --dry-run
python main.py config/fracast/pretrain_smoke.yaml \
  -o train.total_steps=8 -o train.log_every=1

The supported kinds are exactly:

No pipeline module defines its own CLI or main(). Temporary configurations should be written under tmp/ and passed to main.py.

Python inference and benchmark

The installed package loads its bundled W8 checkpoint by default:

import numpy as np
from fracast import FracastModel

model = FracastModel.from_pretrained(device="cpu")
context = np.sin(np.arange(240, dtype=np.float32) / 7.0)
forecast = model.forecast(context)
assert forecast.shape == (48, 9)

# The native head predicts 48 steps. Longer horizons append median blocks and
# re-normalize each 2,048-point context before the next forward pass.
long_forecast = model.forecast(context, horizon=96)
assert long_forecast.shape == (96, 9)

# Independent channels can share one batch invocation.
batch = model.forecast_batch(
    np.stack([context, context * 0.5 + 2.0]), horizon=96
)
assert batch.shape == (2, 96, 9)

Pass weights="fp32" to use the bundled full-precision checkpoint. The first argument may also be a local checkpoint directory or a Hugging Face repository id such as ztxtech/fracast-0. Inputs are [T], [1,T], or [V,T]; outputs are [H,Q], [1,H,Q], or [V,H,Q], where H defaults to the native 48 steps and Q=9. The model uses the latest 2,048 observations, masks missing values, and forecasts channels independently.

Run the release benchmark with:

./script/reproduce.sh benchmark
# or, for a custom device and batch:
python script/benchmark_inference.py --weights w8 --device cuda --batch 32

The JSON report is written under output/benchmark/ and includes load time, parameter count, resident RSS, p50/p95 latency, and series throughput.

Outputs

All runtime directories are ignored by Git. Only the small weights/ safetensors release artifacts are committed; PyTorch training checkpoints and benchmark outputs are not.

Reproducibility

• Dataset revisions are pinned in the download script.
• train.seed controls Python, NumPy, PyTorch, data order, augmentation, and rollout sampling.
• config_used.yaml is written into every output run.
• The fast-corpus merge step checks manifest counts against actual offsets and lengths.
• script/tests/ contains CPU-only structural, causal, parameter-count, and resume-alignment checks, plus public weight-loader coverage.

Run the release checks with:

python script/tests/test_freq.py
python script/tests/test_fracast_unit.py
python script/tests/test_head_future_conv.py
python script/tests/test_resume_stream.py
python script/tests/test_fracast_release.py

Third-party code

Fracast-0 includes small, explicitly documented portions adapted from TinyCast under the Apache-2.0 license. The upstream commit, source paths, and local locations are recorded in THIRD_PARTY_NOTICES.md; the license text is in LICENSES/TinyCast-Apache-2.0.txt.