saas-metrics: Point-in-Time FP&A Backtesting & Automated Excel Models

For Strategic Finance, PE Operating Partners, and Quant Analysts. saas-metrics turns raw billing exports and SEC filings into written, tested definitions, a point-in-time forecast backtest, and an Excel model whose Checks tab proves the workbook matches the Python engine.

• SEC XBRL extraction with ASC 606 splicing. Revenue, deferred revenue, implied billings, RPO, stock-based compensation and free cash flow are taken from companyfacts, resolved across tag changes, and reconciled to each filer's own 10-K. No LLM, no estimate, no invented number.
• Zero look-ahead bias. The walking-forward harness masks every filing after the origin date, runs an independent guard over the history slice, and scores the forecast against the quarters that followed. The frozen MongoDB model is scored next to it.
• Python to audited Excel. The same engine writes a workbook in which every metric is a live formula over blue input cells; the Checks tab recomputes each formula and reads PASS only when it agrees with Python.

Quickstart

pip install "saas-metrics[web]"            # the CLI, the MCP server and the Streamlit workbench
saas-metrics sample billing.csv            # a synthetic billing export to try
saas-metrics app                           # opens the workbench in your browser

pip install saas-metrics alone gives the CLI and the Python library (Python 3.12+); the mcp extra adds only the MCP server. From a clone, uv sync --extra web installs everything and streamlit run app.py works too.

Where files go. Company configs ship inside the package. Downloads, the DuckDB store and generated workbooks go to your user data folder (~/Library/Application Support/saas-metrics on macOS, ~/.local/share/saas-metrics on Linux, %LOCALAPPDATA%\saas-metrics on Windows), or to $SAAS_METRICS_HOME if you set it. A source checkout keeps them in the repository's data/ folder.

SEC access. Anything that downloads from SEC EDGAR (the SEC Comps tab, benchmark, quality, fetch) needs a contact, as the SEC asks: set SEC_USER_AGENT="Your Name you@example.com" in the environment or a .env file. The point-in-time backtest also needs a built store: saas-metrics fetch then saas-metrics build.

The workbench

The Streamlit app is a thin UI over the library, not a second implementation:

Every upload passes the adapter's pre-flight checks before a number appears. A failed row is shown, not silently dropped; a warning is surfaced beside the summary.

Architecture

flowchart LR
    A["Billing CSVs<br/>Stripe · Chargebee · Salesforce"] --> C["DuckDB Engine<br/>canonical ledger · waterfall · cohorts"]
    B["SEC EDGAR<br/>XBRL companyfacts"] --> C
    C --> D["FastMCP Server<br/>deterministic tools"]
    C --> E["Streamlit Workbench<br/>comps · ARR · backtest"]
    D --> F["Audited Excel<br/>live formulas · Checks tab"]
    E --> F

The pipeline is deliberately one-way: raw data -> validated canonical tables -> deterministic Python -> an auditable artefact. Source values keep their filing or invoice provenance; a missing tag or an ambiguous column remains blank and is flagged rather than guessed.

Why spreading matters

An annual plan invoiced in January looks like a customer who arrives in January and churns in February. On the bundled sample export, counting invoices in the month billed produces 214 churn events. Spreading each invoice by day over its service period gives 106.

Pass --service-start and --service-end, and each line is recognised evenly by day, in whole cents, so it still sums to what was billed. Service after the export's last month is deferred and reported, not counted.

What it checks before computing anything

The billing engine has 24 pre-flight checks in the full MongoDB build, and the standalone billing path checks the same things before any table is built:

• Errors (always block): missing required columns, empty values, unparseable dates or amounts.
• Warnings (block unless --allow-warnings): a period with no billing at all, a customer whose net revenue for a period is negative, and dates in the future.
• Info: split invoices and credits that were netted, how many lines were spread, and coverage.

The definitions are written out in docs/definitions.md and enforced by tests. In short: New is first-ever revenue; Reactivation is revenue after a gap; Expansion, Contraction and Churn are measured against the previous period; NRR and GRR compare a period with the same period a year earlier; ARR is revenue times 12 at monthly grain and times 4 at quarterly grain.

The frozen backtest

The MongoDB model was frozen at Q2 FY26 (quarter ended July 31, 2025) using only information filed by 2025-08-27. It then forecast the next four quarters. Revenue over those four quarters was $46.8M (+1.7%) above the frozen plan, but the quarterly path was less comfortable:

• The driver model's Base case had a 3.4% mean absolute error; a simple trend extrapolation did better at 3.0%. Management's guidance was wider still.
• Actual revenue fell inside the Downside-to-Upside range in only one quarter, so the scenario band was too narrow.
• A third baseline, a damped trend fitted only to history before the cutoff, did worse than both naive methods (8.3% mean absolute error): it carried the FY25 slowdown forward.
• The largest single quarterly revenue error came from the timing of Enterprise Advanced term licenses, which are recognised up front. Free cash flow beat the plan mainly because last year's same-quarter cash-conversion ratio was a poor guide.

The walking-forward harness repeats that discipline at several origins. Each origin sees only facts filed on or before the origin date; guard_no_future raises LookAheadBiasError if a loader smuggles in a later period. The default four-origin run scores revenue at a 1.99% MAPE with a -1.14% bias, and FCF at a 96.45% MAPE with a +1.01% bias. Those are results from a small sample, not a claim that the method beats a naive forecast.

The GTM case for hiring 30 Enterprise AEs pays back on its own assumptions, but those assume roughly 2.5 times MongoDB's current sales efficiency. At the company's actual efficiency it does not pay back within 36 months. The memo recommends a smaller pilot gated on quota attainment.

SEC XBRL peer comps

Trailing four fiscal quarters, each company on its own fiscal calendar, from cached companyfacts:

How to read it:

• Non-GAAP margin is approximate: GAAP operating income plus stock-based compensation and amortization. The companies' own definitions exclude more.
• TTD's gross margin is not comparable: The Trade Desk reports no gross profit line, so its gross margin is computed from Platform operations.
• Checks on every row: each company's quarters must sum to its 10-K year, revenue minus cost must equal its gross profit tag where one exists, and the row must be the latest quarter filed.
• Excel output: ratios stay as live formulas over the TTM dollar columns.

DDOG, SNOW and TTD are benchmark-only configs: company, benchmark and nothing else. MDB and the fictional SYNTHCO are full operating-model configs with SEC windows, drivers, cohort assumptions and checks. The two kinds are intentionally not interchangeable.

CLI

# Billing: canonical ledger, waterfall, retention and an audited workbook
saas-metrics sample billing.csv
saas-metrics ledger billing.csv --adapter stripe --duckdb billing.duckdb --analyze
saas-metrics report billing.csv --customer "Customer" --date "Invoice Date" --amount "Amount" \
    --service-start "Service Start" --service-end "Service End" --segment "Plan Tier" --channel "Channel" \
    --excel metrics.xlsx --out tables/

# SEC XBRL and revenue quality
saas-metrics benchmark --tickers MDB DDOG SNOW TTD
saas-metrics quality --tickers MDB DDOG SNOW TTD --out quality.xlsx

# Cohorts and the point-in-time harness
saas-metrics cohorts --input billing.csv --out out/
saas-metrics cohorts --ledger MDB
saas-metrics backtest
saas-metrics backtest --pit --horizon 4 --evaluations 4

# Full operating model (MDB is the default config)
saas-metrics --config configs/MDB.yaml fetch
saas-metrics --config configs/MDB.yaml build
saas-metrics --config configs/MDB.yaml qc
saas-metrics --config configs/MDB.yaml workbook
saas-metrics --config configs/MDB.yaml backtest
saas-metrics --config configs/MDB.yaml memo

# MCP server
saas-metrics mcp

The legacy saasfpa entry point is still installed:

saasfpa benchmark --tickers MDB DDOG SNOW TTD
saasfpa cohorts --input billing.csv --out out/
saasfpa unit-econ --n-ae 20 --rep-attrition 0.25 --target-payback 24

MCP configuration

The FastMCP server exposes four deterministic tools: ingest_billing_ledger, generate_cohort_report, run_sec_peer_benchmark and analyze_revenue_quality. Add this to the Claude Desktop configuration file (on macOS, ~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "saas-metrics": {
      "command": "uvx",
      "args": ["--from", "saas-metrics[mcp]", "saas-metrics", "mcp"],
      "env": {
        "SEC_USER_AGENT": "Your Name you@example.com"
      }
    }
  }
}

Ingested ledgers go to data/billing.duckdb under the work root unless SAAS_METRICS_LEDGER_DB names another file. If Claude Desktop does not inherit your shell PATH, use the absolute path to uvx (run which uvx). From a source checkout, use command: "uv" and args: ["run", "--directory", "/absolute/path/to/saas-metrics", "saas-metrics", "mcp"].

Reproducing the tests

Python 3.12 and uv.

make install
make test
ruff check src tests scripts

The test suite is offline: the DuckDB store is rebuilt from parsed SEC tables in tests/fixtures, companyfacts come from frozen fixtures, and billing examples are synthetic. The only network marker is network, which is excluded from the default run.

Layout

app.py                 launches the workbench (src/saas_metrics/app.py) for `streamlit run` / Streamlit Cloud
configs/               one YAML per company (MDB full model; DDOG, SNOW, TTD benchmark; examples/SYNTHCO);
                       shipped inside the wheel as saas_metrics/configs
src/saas_metrics/      engine, adapters, SEC/XBRL, quality, backtest, FastMCP server, the Streamlit workbench
src/saasfpa/           the legacy compatibility package
src/saas_metrics/sql/  analytical views, the billing engine's and the cohort engine's SQL
tests/                 offline tests, fixtures and the golden snapshot
models/                the audited Excel operating model
memos/                 memo template and rendered memo
docs/                  definitions, sources and the toolkit plan

Limitations

• One real company and four test quarters are enough to diagnose this forecast, not to claim the method is generally better than a naive one. On this sample, quarter by quarter, it was not.
• The ARR proxy is not MongoDB's disclosed ARR, and the cohort panel is synthetic; splits that depend on the assumed churn rate are directional.
• Headcount is disclosed once a year, so the split between hiring pace and cost per head is uncertain for the latest quarters, though their sum is exact.
• The bridge attribution depends on the substitution order.
• Peer comps use each company's XBRL tags as filed. A changed or missing tag is caught by the checks or the notes, but definitions still differ between companies.

The sample data is synthetic. The licence is MIT.