MDRAP Logo

MDRAP

Market Data Reliability & Acceleration Platform
A deterministic, low-latency market data validation, normalization, and cryptographic audit sidecar.

Demo • Docs • Install • Contributing

💡 Why This Project?

Highlight	Architecture & Strategy Advantage
🛡️ Crossed Quote & Anomaly Defense	Eliminates phantom arbitrage losses by intercepting crossed books (`bid > ask`), sequence jumps, and rolling Welford price corridor anomalies before strategy execution.
⚡ Microsecond-to-Nanosecond Tiers	Standalone native C kernel evaluates ticks at 45 ns/tick wire-to-SHM; end-to-end Python pipeline processes at 8.3 µs p50 with zero mandatory dependencies.
📦 Zero Silent Drops	Corrupted, malformed, or delayed frames are safely quarantined in a queryable SQLite ledger with explicit bitmask reason codes and complete upstream provenance.
🔒 Tamper-Evident Merkle Proofs	SHA-256 Merkle chain guarantees mathematical auditability: modifying any historical database record immediately invalidates the cryptographic integrity proof.

🏛️ How It Works

flowchart LR
    subgraph Ingress ["1. Ingress & Normalize"]
        A["Raw Feeds\n(Binance, ITCH, Polygon)"] --> B["Gateway Normalization\n(RawEvent)"]
    end
    subgraph Core ["2. Reliability & Fastpath"]
        B --> C{"Quality Engine\n(15 Rules)"}
        C -->|Valid / Suspicious| D["Multi-Feed Reconciler\n(CanonicalEvent)"]
        C -->|Invalid| Q["Quarantine Ledger\n(Reason Bitmask)"]
    end
    subgraph Handoff ["3. Handoff & Durability"]
        D --> E["IngestLog WAL\n(CRC-32 Crash Durability)"]
        D --> F["SPSC Ring Buffer\n(Zero-Copy SHM <1µs)"]
        E --> G["SQLite / DuckDB\n(Projections & Audit)"]
    end

🎬 Quickstart & Demo

MDRAP Real-Time Market Data Anomaly Detection & Audit Demo

5-Minute Golden Path

# doc-test
import tempfile, time
from mdrap import Engine, EngineConfig, RawEvent

wal_dir = tempfile.mkdtemp()
engine = Engine.open(wal_dir, config=EngineConfig(db_path=":memory:"))

now = time.time()
raw = RawEvent(
    source="BATS",
    payload={
        "instrument": "AAPL",
        "event_type": "QUOTE",
        "bid": 150.25,
        "ask": 150.30,
        "exchange_ts": now,
        "sequence": 1,
    },
    receive_timestamp=now,
)

decision = engine.submit(raw)
event = decision.canonical_event

print(f"Status: {event.quality_status.name}")
print(f"BBO: {event.bid_price} / {event.ask_price}")
ticks = engine.query("AAPL", limit=5)
print(f"Queried ticks: {len(ticks)}")
engine.close()

Expected output:

Status: VALID
BBO: 150.25 / 150.3
Queried ticks: 1

🚀 Installation

Linux / macOS (pip)

git clone https://github.com/Aryan-20-04/mdrap.git
cd mdrap
pip install -e .

# Verify installation & active acceleration tier
python cli.py doctor

Windows (PowerShell)

git clone https://github.com/Aryan-20-04/mdrap.git
cd mdrap
pip install -e .

# Run diagnostics
python cli.py doctor

Docker

docker build -t mdrap:latest .
docker run --rm -it -p 8000:8000 mdrap:latest mdrap doctor

⚡ Performance Benchmarks

All figures are measured on committed benchmark hardware (Intel Core i7-12700H, 12 cores, Windows 11 / Linux parity) and trace directly to committed JSON outputs in benchmarks/.

Stage	Scope / Conditions	p50 Latency	p99 Latency	Throughput (EPS)	Reproduce Command
Native C Kernel	In-memory tick evaluation (`fastpath.c`)	45.0 ns	45.0 ns	22,228,643	`python benchmarks/bench_scale.py`
In-Memory IPC Fanout	10 subscribers, zero-copy handoff	0.4 µs	0.6 µs	1,529,870	`python benchmarks/run_honest_benchmarks.py`
Python Pipeline Step	In-memory normalization, 15 rules, dedup	8.3 µs	14.3 µs	120,280	`python benchmarks/run_honest_benchmarks.py`
Disk Durability (Grouped)	SQLite WAL, `fsync="grouped_by_time"` (batch 200)	6.82 ms	6.91 ms	29,654	`python benchmarks/run_honest_benchmarks.py`
Disk Durability (Always)	SQLite WAL, `fsync="always"` (batch 100)	4.60 ms	4.84 ms	21,615	`python benchmarks/run_honest_benchmarks.py`

🔍 Deep Dive & Documentation

Feature Status Matrix

Feature	Status	Docs	Test File
Deterministic Quality Engine (15 Rules)	Stable	Quality Rules	`tests/test_adapters_and_rules.py` (1,500+ core test suite passed cleanly)
Append-Only IngestLog WAL (CRC-32)	Stable	Event Schema	`tests/test_wal_format_v2.py`
Single-Node Canonical Engine API	Stable	SDK Integration	`tests/test_engine.py`
Cryptographic Merkle Audit Verification	Stable	Audit Verification	`tests/test_audit_remediation.py`
FastAPI REST & WebSocket Server	Stable	API Reference	`tests/test_api_endpoints.py`
Shared Memory SPSC Ring Buffer	Beta	Architecture	`tests/test_shm_decoupled.py`
Native C Fastpath Kernel	Beta	Benchmarks	`tests/test_differential_fuzz.py`
ITCH & MoldUDP64 Feed Dissector	Beta	Feed Adapters	`tests/test_adversarial_fuzzing.py`
Streaming WAL Journal Replication	Beta	Operations	`tests/test_journal_replication_daemon.py`
AF_XDP Kernel Bypass Receiver	Experimental	Roadmap	`tests/test_af_xdp.py`
Distributed Multi-Host Raft Consensus	Roadmap	Roadmap	Planned (Mode B)

Pick Your Path (By Persona)

Persona	3-Line Example	Guide
Quant Researcher	`from mdrap.chd import CHDClient, HistoricalRequest` `client = CHDClient(cache_dir="./data/cache")` `ticks = client.download(HistoricalRequest("binance", "spot", "BTCUSDT", "trades", "2026-01-01"))`	Research & Backtesting Guide
Trader / PM	`from mdrap.service import StreamClient` `client = StreamClient(host="127.0.0.1", port=9988)` `for tick in client.subscribe(["NVDA"]): print(tick)`	Live Monitoring Guide
Developer	`engine = Engine.open("./wal", config=EngineConfig(db_path=":memory:"))` `decision = engine.submit(raw_event)` `ticks = engine.query("AAPL", limit=10)`	SDK Integration Guide
SRE / DevOps	`curl -f http://127.0.0.1:8000/v1/health` `curl -f http://127.0.0.1:8000/metrics` `mdrap doctor --json`	Operations Guide
Compliance / Risk	`from mdrap.storage import Store` `store = Store("data/mdrap.db")` `report = store.verify_audit_integrity()`	Audit Verification Guide

Scope & Known Limitations

Scope Boundary: MDRAP is a single-node middleware sidecar. It does not provide proprietary feed licenses (bring your own API keys) and does not execute broker orders.
Feed Runtime: Registration exposes metadata only. There is no supervisor that owns adapter lifecycle or background process spawning.
Failover: Local active-passive replication via TCP journal streaming (journal_replicator.py); multi-datacenter distributed Raft consensus is on the roadmap.
Backpressure: When downstream consumers cannot keep up with high-frequency streams, oldest frames are evicted to prioritize real-time low latency while updating drop counters.
Compliance Tooling: Emits audit trails that can support SEC Rule 613 (CAT) or MiFID II RTS 25 compliance workflows, but software execution does not by itself confer legal certification.

Project Structure

mdrap/
├── src/mdrap/              # Core market data engine package
│   ├── engine.py           # Canonical engine API & step orchestration
│   ├── gateway.py          # Multi-venue normalization (RawEvent -> CanonicalEvent)
│   ├── quality.py          # 15 deterministic microstructure quality checks
│   ├── reconciliation.py   # Multi-feed arbitration and provenance tracking
│   ├── ingestlog.py        # CRC-32 write-ahead log (WAL) for crash durability
│   ├── shm.py              # Zero-copy SPSC shared-memory ring buffer (<1µs)
│   ├── fastpath.c          # Native C hot-path kernel (AVX2/SIMD + LTO)
│   ├── bbo.py              # Consolidated NBBO and micro-price tracking
│   ├── depth.py            # Level-2 aggregated order book depth ladder
│   └── security.py         # SHA-256 Merkle audit trail and API key hashing
├── tests/                  # 1,540+ automated unit & integration test suites
├── benchmarks/             # Reproducible latency and throughput harnesses
├── docs/                   # Architectural guides, API specs, and runbooks
├── tools/                  # Verification guardrails (check_docs.py, VHS demo)
└── assets/                 # Vector SVG logos and brand assets

📚 Documentation Index

Getting Started: Quickstart Guide · Concepts & Data Model
User Guides: Research & Backtesting · Live Monitoring · SDK Integration · Operations Guide · Audit Verification
Technical Reference: CLI Reference · REST & WebSocket API · Configuration Options · Quality Rules Catalog · Event Schema
Platform Specs: Security Architecture · Benchmark Methodology · Operational Runbooks · Architecture Roadmap

🤝 Contributing, Security & License

Contributing: Issues and pull requests are welcomed! See DEVELOPMENT.md for code styling and test verification gates.
Security: Report vulnerabilities according to our Security Policy.
License: Released under the MIT License. Copyright (c) 2026 MDRAP Authors.

Made with ❤️ by the MDRAP Authors

Metadata

Release files for mdrap-core 4.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mdrap-core 4.0.0
File	Size	Uploaded	Provenance
mdrap_core-4.0.0.tar.gz	709.7 kB	Oct 23, 2026	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for mdrap-core 4.0.0
File	Interpreter	ABI	Platform	Reset
mdrap_core-4.0.0-py3-none-any.whl 760.2 kB Oct 23, 2026 Without provenance	Python 3	none	any	Details

Total release size: 1.5 MB

Release files / mdrap_core-4.0.0.tar.gz

Download URL	mdrap_core-4.0.0.tar.gz
Size	709.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`27c4cd2a8f8236392cda5b4484a8a493f6e42e252ce47cc763d2954fe522fec4`
BLAKE2b-256 checksum How to use checksums	`c2a37940921432d4f63bb6d2fc567d5011f23b2998d0b1153340618abaeacff5`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.1`

Release files / mdrap_core-4.0.0-py3-none-any.whl

Download URL	mdrap_core-4.0.0-py3-none-any.whl
Size	760.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`01c8ef642ccbd6e0f9e85ce94cd956546205e76fef1c74ebb0874e62044afffb`
BLAKE2b-256 checksum How to use checksums	`d75ae7d4fe175b237124726a68a8096237ad4693ce5651c0d17bd30999e2e9fa`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.1`

Security for mdrap-core 4.0.0

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.0.0 This release

Oct 23, 2026 2 release files No provenance

3.1.0

Oct 9, 2026 1 release file No provenance

mdrap-core 4.0.0