crypto-composite-market-data 0.9.0

Public multi-exchange crypto market-data composite toolkit.

crypto-composite-market-data

Public multi-exchange crypto market-data composite toolkit.

crypto-composite-market-data builds reproducible JSON artifacts from public Binance, OKX and Bybit endpoints. It focuses on market-data normalization, timestamp-aligned OHLCV composition, composite orderbook ladder buckets, multi-symbol universe runs, local artifact inspection, and data-quality reporting.

The dashboard screenshot uses the checked-in synthetic sample artifacts; it does not show live market data or trading signals.

Note: PyPI/Shields badges can take a few minutes to refresh after a new release.

Why this exists

Public crypto market data is fragmented. A single exchange can be stale, unavailable, noisy, rate-limited, or structurally different from the broader venue set.

This project helps developers and researchers answer practical data-quality questions before using market data downstream:

• Which venues returned usable data?
• Is composite OHLCV coverage complete or partial?
• Are prices dispersed across venues?
• Is orderbook depth concentrated in one venue?
• Which assets in a universe have enough public data to inspect?
• Which JSON artifacts were generated and where are they?

See Why composite public market data for the exact coverage, dispersion, and public orderbook concentration semantics.

Who it is for

• Data engineers building repeatable public crypto data pipelines.
• Quant researchers who need auditable input artifacts before analysis.
• Dashboard builders who want stable JSON files instead of exchange-specific payloads.
• Open-source contributors who want a small, testable market-data infrastructure project.

What it does

• Fetches public OHLCV, recent trades, orderbook snapshots, funding and open-interest data.
• Normalizes records into stable Python dataclasses.
• Builds timestamp-aligned composite OHLCV by market type.
• Builds public composite orderbook ladder buckets near a reference price.
• Reports venue coverage, price dispersion and composite status.
• Runs explicit multi-symbol universes and writes per-asset artifact folders.
• Serves a read-only local artifact dashboard API.
• Writes reproducible JSON artifacts for research, dashboards and downstream analytics.

What it does not do

• No trading signals.
• No order execution.
• No private exchange account APIs.
• No position sizing.
• No financial advice.
• No private orderflow or market-maker intent claim.
• No profitability or statistical-edge claim.

Install

From PyPI:

pip install crypto-composite-market-data

From TestPyPI for release candidates:

python -m pip install \
  --index-url https://test.pypi.org/simple/ \
  --extra-index-url https://pypi.org/simple/ \
  crypto-composite-market-data

For local development:

git clone https://github.com/thanhlq8-max/crypto-composite-market-data.git
cd crypto-composite-market-data
python -m pip install -r requirements-dev.txt
python -m pip install -e .
python -m pytest -q

5-minute quickstart

Create a small multi-asset universe:

crypto-composite universe \
  --assets BTC-USDT,ETH-USDT,SOL-USDT \
  --venues binance,okx,bybit \
  --market-types spot_usdt,perp_usdt \
  --timeframes 15m \
  --limit 100 \
  --out-dir artifacts-universe

Inspect the generated files:

artifacts-universe/
|-- BTC-USDT/
|   |-- composite_ohlcv_15m.json
|   |-- composite_orderbook_ladder_15m.json
|   |-- data_quality.json
|   `-- run_summary.json
|-- ETH-USDT/
|-- SOL-USDT/
`-- universe_summary.json

Serve the read-only local artifact API:

crypto-composite dashboard \
  --artifact-root artifacts-universe \
  --host 127.0.0.1

Open:

http://127.0.0.1:18080/
http://127.0.0.1:18080/api/health
http://127.0.0.1:18080/api/artifacts

See docs/QUICKSTART_5_MIN.md for Windows and venv-safe commands.

Single-asset run

crypto-composite run \
  --asset BTC-USDT \
  --venues binance,okx,bybit \
  --market-types spot_usdt,perp_usdt \
  --timeframes 15m \
  --limit 300 \
  --depth 100 \
  --out-dir artifacts

Equivalent module form:

python -m crypto_composite.cli run \
  --asset BTC-USDT \
  --venues binance,okx,bybit \
  --market-types spot_usdt,perp_usdt \
  --timeframes 15m \
  --out-dir artifacts

Useful output preview

A data_quality.json artifact summarizes whether downstream consumers should trust the generated composite data:

{
  "timeframes": {
    "15m": {
      "ohlcv_status": "COMPOSITE_DATA_OK",
      "book_status": "COMPOSITE_BOOK_OK",
      "ohlcv_coverage": 1.0,
      "book_coverage": 1.0
    }
  }
}

A universe_summary.json artifact summarizes multi-symbol runs without producing rankings or trading signals:

{
  "asset_count": 3,
  "assets": ["BTC-USDT", "ETH-USDT", "SOL-USDT"],
  "boundaries": [
    "No trading signals, order execution, position sizing, or financial advice."
  ]
}

Sample illustrative artifacts are available under examples/sample_artifacts.

Consume the checked-in fixture through the public validation and quality APIs:

python examples/inspect_quality.py --artifact-root examples/sample_artifacts

See docs/TUTORIAL_CONSUME_ARTIFACTS.md for the offline downstream example.

Output artifacts

For each timeframe:

artifacts/raw_scan_15m.json
artifacts/composite_ohlcv_15m.json
artifacts/composite_orderbook_ladder_15m.json

Combined outputs:

artifacts/composite_ohlcv.json
artifacts/composite_orderbook_ladder.json
artifacts/data_quality.json
artifacts/run_summary.json

Multi-symbol output:

artifacts-universe/<ASSET>/...
artifacts-universe/universe_summary.json

See docs/ARTIFACT_SCHEMA.md.

Supported scope

Area	Supported
Venues	Binance, OKX, Bybit
Asset format	BASE-USDT, for example BTC-USDT
Market types	spot_usdt, perp_usdt
Timeframes	1m, 5m, 15m, 1h
Data access	Public REST endpoints only
Python	3.11, 3.12, 3.13

Composite OHLCV model

For each timestamp and market type, the engine computes:

• weighted open;
• high / low envelope;
• median close;
• quote-volume-weighted close;
• total base and quote volume;
• venue weights;
• venue coverage;
• price dispersion percentage;
• data-quality score.

Status labels:

COMPOSITE_DATA_OK
COMPOSITE_DATA_PARTIAL
COMPOSITE_DATA_WEAK

Composite orderbook ladder model

The ladder engine merges public orderbook snapshots into near-book price buckets. For each bucket, it computes:

• quote depth;
• contributing venue count;
• venue depth map;
• HHI concentration;
• persistence proxy;
• spoof-risk proxy;
• vacuum score.

Status labels:

COMPOSITE_BOOK_OK
COMPOSITE_BOOK_PARTIAL
COMPOSITE_BOOK_WEAK

Static HTML report

Generate a shareable artifact-quality report:

crypto-composite report --artifact-root artifacts-universe --out-file report.html

The report summarizes quality score, venue coverage, composite OHLCV status, orderbook status, price dispersion, validator warnings/errors, and JSON artifact links. It is an inspection page only; it is not a trading signal, prediction, execution instruction, or financial-advice document.

See docs/STATIC_REPORT.md.

Local dashboard API

Serve a read-only local artifact API:

crypto-composite dashboard --artifact-root artifacts-universe --host 127.0.0.1 --port 18080

Endpoints:

/
/api/health
/api/artifacts
/api/dashboard-snapshot
/api/artifact?path=<relative-json-path>

Dashboard V3 adds a practical monitoring brief organized as DID / DOING / NEXT / Confidence-Risk, exact zone distance and reference location, composite price and public-depth charts, evidence-grade zones, and spot/perpetual dislocation context. All observations are derived from generated public-data artifacts and remain non-predictive.

Export the same dashboard as static HTML for GitHub Pages or offline sharing:

crypto-composite dashboard-export \
  --artifact-root artifacts-universe \
  --out-file site/index.html \
  --artifact-base-url artifacts

See docs/DASHBOARD_API.md.

Community and contribution

Useful first contributions:

• Add a new public exchange connector.
• Improve mocked connector parser coverage.
• Add CSV or DuckDB artifact export.

See CONTRIBUTING.md, CODE_OF_CONDUCT.md, the GitHub issue templates, and docs/GOOD_FIRST_ISSUES.md.

External users can select Downstream use case in the GitHub issue chooser to report a verifiable integration without making trading-performance claims.

Limitations

Public exchange APIs can rate-limit, return incomplete data, or temporarily fail.

The orderbook ladder is a public snapshot bucket proxy; it is not a private, full-depth, consolidated matching-engine book.

This project is data infrastructure. Any downstream research or visualization must preserve the no-signal, no-execution, no-financial-advice boundary unless explicitly implemented in a separate project with its own validation and legal review.

Packaging

See docs/PACKAGING.md for TestPyPI/PyPI release preparation.

Development roadmap

See:

• docs/ROADMAP.md
• docs/USEFUL_OUTPUTS.md
• docs/DASHBOARD_PLAN.md
• docs/DASHBOARD_API.md
• docs/STATIC_REPORT.md
• docs/TUTORIAL_CONSUME_ARTIFACTS.md
• docs/WHY_COMPOSITE_MARKET_DATA.md
• docs/GITHUB_PAGES_DEMO.md
• docs/PACKAGING.md
• docs/COMMUNITY_GROWTH.md
• docs/ADOPTION_PLAYBOOK.md
• docs/CLAUDE_FOR_OSS_READINESS.md
• docs/RELEASE_CHECKLIST.md

License

Apache License 2.0. See LICENSE.

Artifact quality scoring

After generating artifacts, compute a compact quality score:

crypto-composite score-artifacts --artifact-root artifacts-universe --write

This writes quality_score.json with a reproducible A-F grade based on venue coverage, composite OHLCV coverage, price dispersion, orderbook coverage and existing data-quality status. The score is for artifact inspection only; it is not a trading signal or prediction score.

See docs/ARTIFACT_QUALITY_SCORE.md.