orderpulse 0.2.19

High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings

OrderPulse

Ultra-fast exchange feed parser and in-memory order book engine written in Rust with Python bindings using PyO3.

OrderPulse is designed for quantitative trading infrastructure where binary market feeds must be decoded with minimal latency and reused efficiently across analytics, simulation, and execution pipelines.

---

Why This Library Exists

In real trading systems, the biggest bottleneck is usually not strategy logic. It is:

• binary feed parsing
• repeated disk I/O
• inefficient Python loops
• rebuilding order books repeatedly
• memory duplication

This library solves that.

The architecture is intentionally designed like a production-grade market data engine:

• Rust handles parsing and memory management
• Python becomes orchestration layer only
• Binary file is read once
• Messages are shared using Arc<Vec<Message>>
• Multiple consumers can process same data without re-reading disk
• Orderbook reconstruction is incremental and event-driven

This is exactly how low-latency market data infrastructure is designed in professional trading firms.

---

Architecture

                    Binary Exchange Feed
                              |
                              v
                 +-----------------------+
                 | ReadMsgFromBinary     |
                 |-----------------------|
                 | Parse binary packets  |
                 | Filter token streams  |
                 | Store in shared RAM   |
                 +-----------------------+
                              |
                    Shared Arc<Vec<Message>>
                              |
         ------------------------------------------------
         |                                              |
         v                                              v
+----------------------+                +---------------------------+
| BinaryLoader         |                | OrderBookGenerator        |
|----------------------|                |---------------------------|
| Sequential streaming |                | Incremental book builder  |
| Cursor-based access  |                | Top-of-book snapshots     |
| Zero reread          |                | Bid/ask reconstruction    |
+----------------------+                +---------------------------+

---

Installation

Build Rust Extension

maturin develop --release

or

pip install maturin
maturin build --release

---

Python API

The library exposes three primary classes:

Class	Purpose
ReadMsgFromBinary	Parse and hold binary feed in memory
BinaryLoader	Sequential streaming over parsed messages
OrderBookGenerator	Build live order book snapshots

---

1. ReadMsgFromBinary

This is the entry point of the entire system.

It performs:

• binary file parsing
• decoding order packets
• decoding trade packets
• optional token filtering
• storing all parsed messages into shared memory

The important architectural decision:

File is read ONLY ONCE.

After parsing, every component works from RAM.

---

Constructor

ReadMsgFromBinary(path, token=None)

Parameters

Parameter	Type	Description
path	str	Binary market feed file
token	`int	None`	Optional instrument filter

---

Example — Full Feed Load

from fastreader import ReadMsgFromBinary

reader = ReadMsgFromBinary(
    path="nse_market_feed.bin"
)

What Happens Internally

Disk Binary File
       ↓
Rust Decoder
       ↓
Order + Trade Messages
       ↓
Stored Into Shared Arc<Vec<Message>>

No additional disk access occurs afterward.

---

Example — Single Instrument Feed

In production trading systems, strategies usually subscribe to a small subset of instruments.

This constructor-level filtering avoids unnecessary memory usage.

reader = ReadMsgFromBinary(
    path="market_feed.bin",
    token=26000
)

This means:

• only token 26000 messages are retained
• all unrelated packets are discarded during parsing
• memory footprint becomes significantly smaller
• downstream analytics become faster

---

Functions in ReadMsgFromBinary

---

total_messages()

Returns total parsed packets.

count = reader.total_messages()
print(count)

Production Use Case

Useful for:

• feed completeness checks
• session diagnostics
• replay validation
• packet gap analysis

---

total_orders()

Returns total order messages.

orders = reader.total_orders()
print(orders)

Typical Order Messages

These may include:

• add order
• modify order
• cancel order
• delete order

---

total_trades()

Returns total trade messages.

trades = reader.total_trades()
print(trades)

Production Usage

Useful for:

• trade intensity analysis
• execution profiling
• aggressor flow estimation
• VWAP reconstruction

---

summary()

Quick feed overview.

reader.summary()

Example Output

Total Messages: 12093811
Total Orders: 11800210
Total Trades: 293601

Why Traders Use This

Usually the first command after loading any exchange capture.

Acts as:

• sanity check
• parser verification
• session health indicator

---

reset_cursor()

Resets internal iterator.

reader.reset_cursor()

Why This Matters

In replay systems:

• one strategy may consume messages once
• another backtest may need replay again

Cursor reset avoids reloading the file.

---

get_all_messages(limit=None)

Returns all parsed packets.

messages = reader.get_all_messages(limit=5)

for msg in messages:
    print(msg)

Example Output

Order Message: SeqNo1001, msg_len64, Msg_Type'N', ...
Trade Message: SeqNo1002, msg_len56, Msg_Type'T', ...

Engineering Insight

This function is useful for:

• debugging parsers
• validating exchange packet structure
• building analytics prototypes
• replay verification

---

get_order_messages(limit=None)

Returns only order packets.

orders = reader.get_order_messages(limit=10)

Typical Usage

Used when reconstructing:

• queue position
• order flow imbalance
• liquidity dynamics
• passive/active pressure

---

get_trade_messages(limit=None)

Returns only trade packets.

trades = reader.get_trade_messages(limit=10)

Production Use Cases

Used for:

• tape reading
• execution analytics
• trade clustering
• market impact modeling

---

2. BinaryLoader

BinaryLoader is a lightweight sequential reader.

It does NOT:

• reopen files
• allocate memory again
• duplicate messages

It simply streams over already parsed RAM data.

This design is critical in high-frequency simulation systems.

---

Constructor

BinaryLoader(reader)

Example

from fastreader import BinaryLoader

loader = BinaryLoader(reader)

Internal Design

BinaryLoader
      ↓
Shared Arc<Vec<Message>>
      ↓
Cursor-based sequential fetch

This is essentially a replay engine.

---

get_next_message()

Returns next sequential packet.

while True:

    msg = loader.get_next_message()

    if msg == "END":
        break

    print(msg)

Why This Is Important

Professional trading systems are event-driven.

Strategies consume:

• one packet at a time
• in strict sequence order
• without loading entire dataset repeatedly

This method replicates live-feed style consumption.

---

reset_cursor()

Restart sequential replay.

loader.reset_cursor()

Use Case

Useful for:

• strategy reruns
• simulation resets
• deterministic testing
• model validation

---

3. OrderBookGenerator

This is the order book reconstruction engine.

It processes:

• order add
• modify
• cancel
• trade execution

and incrementally maintains:

• bid levels
• ask levels
• mid price
• depth snapshots

This is the core of market microstructure infrastructure.

---

Constructor

from fastreader import OrderBookGenerator

ob = OrderBookGenerator()

---

create_orderbook(reader)

Builds top-of-book snapshots from parsed messages.

rows = ob.create_orderbook(reader)

print(rows)

---

Internal Processing Pipeline

Market Messages
        ↓
OrderBookManager
        ↓
Incremental Bid/Ask Update
        ↓
Top 5 Levels Extraction
        ↓
Mid Price Computation
        ↓
Snapshot Output

---

Example Snapshot

local_ts1711000100,
exch_ts1711000001,
mid_price22451.25,
bid_price_022450.95,
bid_qty_0150,
ask_price_022451.55,
ask_qty_0100

---

What Makes This Fast

1. Rust Memory Safety

No Python object overhead during parsing.

---

2. Shared Memory Architecture

Arc<Vec<Message>>

Messages are:

• stored once
• shared safely
• reused everywhere
• zero-copy shared

---

3. Sequential Replay Design

Cursor-based replay avoids:

• reallocation
• reindexing
• repeated parsing

---

4. Event-Driven Order Book

Order book updates are incremental.

The engine does NOT rebuild entire depth every tick.

This is how institutional trading engines work.

---

Complete Production Example

from fastreader import (
    ReadMsgFromBinary,
    BinaryLoader,
    OrderBookGenerator
)

# =====================================================
# STEP 1 — LOAD BINARY FEED
# =====================================================

reader = ReadMsgFromBinary(
    path="nse_capture.bin",
    token=26000
)

reader.summary()

# =====================================================
# STEP 2 — EVENT REPLAY
# =====================================================

loader = BinaryLoader(reader)

while True:

    msg = loader.get_next_message()

    if msg == "END":
        break

    # strategy logic here
    print(msg)

# =====================================================
# STEP 3 — ORDER BOOK RECONSTRUCTION
# =====================================================

ob = OrderBookGenerator()

rows = ob.create_orderbook(reader)

print(f"Generated snapshots: {rows}")