orderpulse 0.2.18

High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings

fastreader

🚀 High-performance binary market data parser and order book engine for Python, powered by Rust + PyO3.

---

Overview

fastreader is a high-performance Python library built using:

• Rust
• PyO3
• Python bindings

The library is designed for ultra-fast exchange binary feed parsing and top-of-book orderbook reconstruction.

Unlike traditional Python-only parsers, heavy computation is executed inside Rust, giving:

• Much faster parsing speed
• Lower memory overhead
• Efficient sequential processing
• High-throughput message handling

The library is ideal for:

• Quantitative trading systems
• HFT research
• Tick-level backtesting
• Exchange feed analytics
• Order-flow analysis
• Orderbook reconstruction
• Market microstructure research

---

Architecture

The library is divided into clear responsibilities.

Binary Feed File
        ↓
ReadMsgFromBinary
        ↓
BinaryLoader / MessageBatch
        ↓
OrderBookBuilder
        ↓
Analytics / Research / Backtesting

---

Core Design Philosophy

The library separates responsibilities carefully.

ReadMsgFromBinary

Responsible only for:

• Reading binary files
• Parsing exchange messages
• Filtering messages
• Counting messages
• Selecting messages

It does NOT perform sequential streaming access anymore.

---

BinaryLoader

Responsible only for:

• Storing already parsed messages
• Sequential one-by-one fetching
• Cursor-based iteration

This class now contains:

get_next_message()

ONLY.

---

MessageBatch

Responsible only for:

• Storing selected groups of parsed messages

Useful for:

• Batch processing
• Orderbook reconstruction
• Chunk-wise workflows

---

OrderBookBuilder

Responsible only for:

• Reconstructing orderbook snapshots
• Processing parsed messages

It does NOT read binary files directly.

---

Features

✅ Ultra-fast binary parsing

✅ Rust-powered backend

✅ Python-friendly APIs

✅ Token filtering

✅ Sequential message fetching

✅ Batch processing

✅ Orderbook reconstruction

✅ Memory-efficient architecture

✅ Large-file support

---

Installation

Install

pip install orderpulse

---

Quick Start

Step 1 — Import Classes

from fastreader import (
    ReadMsgFromBinary,
    BinaryLoader,
    OrderBookBuilder
)

---

Step 2 — Read Binary File

reader = ReadMsgFromBinary(
    "market_data.bin",
    token=26000
)

Explanation:

• Binary file is read once
• Messages are parsed inside Rust
• Only token 26000 messages are stored

---

Step 3 — Print Summary

reader.summary()

Output:

Total Messages: 1520000
Total Orders: 1300000
Total Trades: 220000

---

Step 4 — Sequential Streaming Using BinaryLoader

loader = BinaryLoader(reader)

Important:

BinaryLoader does NOT read binary files.

It only stores already parsed messages from:

ReadMsgFromBinary

---

Step 5 — Fetch Messages One-by-One

while True:

    msg = loader.get_next_message()

    if msg == "END":
        break

    print(msg)

This is useful for:

• Streaming-style workflows
• Real-time simulations
• Sequential processing
• Tick replay systems

---

Core Classes

1. ReadMsgFromBinary

Main binary parsing engine.

---

Constructor

ReadMsgFromBinary(path, token=None)

Parameters

Parameter	Type	Description
path	str	Binary feed file path
token	int	Optional instrument token filter

---

Example

reader = ReadMsgFromBinary(
    "market_data.bin",
    token=26000
)

---

Available Methods

---

total_messages()

Returns total parsed messages.

count = reader.total_messages()

---

total_orders()

Returns total order messages.

orders = reader.total_orders()

---

total_trades()

Returns total trade messages.

trades = reader.total_trades()

---

summary()

Prints message statistics.

reader.summary()

---

reset_cursor()

Resets internal selection cursor used by:

select_next_messages()

Example:

reader.reset_cursor()

---

Message Extraction APIs

---

get_all_messages()

Returns formatted strings for all messages.

messages = reader.get_all_messages(limit=10)

---

get_order_messages()

Returns only order messages.

orders = reader.get_order_messages(limit=10)

---

get_trade_messages()

Returns only trade messages.

trades = reader.get_trade_messages(limit=10)

---

Selection APIs

These methods return MessageBatch.

---

select_all_messages()

batch = reader.select_all_messages()

---

select_order_messages()

batch = reader.select_order_messages()

---

select_trade_messages()

batch = reader.select_trade_messages()

---

select_next_messages(limit)

Chunk-wise message selection.

batch = reader.select_next_messages(1000)

Useful for:

• Large files
• Memory control
• Chunk processing pipelines

---

2. BinaryLoader

Sequential message loader.

Important:

This class does NOT parse binary files.

It only stores already parsed messages from:

ReadMsgFromBinary

---

Constructor

loader = BinaryLoader(reader)

---

total_messages()

Returns total stored messages.

count = loader.total_messages()

---

reset_cursor()

Resets sequential cursor.

loader.reset_cursor()

---

get_next_message()

Returns next formatted message.

msg = loader.get_next_message()

Returns:

"END"

when all messages are consumed.

---

Example

while True:

    msg = loader.get_next_message()

    if msg == "END":
        break

    print(msg)

---

3. MessageBatch

Lightweight container for selected parsed messages.

Used for:

• Batch workflows
• Orderbook reconstruction
• Chunk processing

---

len()

print(batch.len())

---

is_empty()

print(batch.is_empty())

---

to_list()

messages = batch.to_list(limit=10)

---

4. OrderBookBuilder

Top-of-book reconstruction engine.

Processes parsed order/trade messages sequentially.

---

Create Builder

ob = OrderBookBuilder()

---

create_orderbook_all_messages()

Build orderbook from all reader messages.

rows = ob.create_orderbook_all_messages(reader)

---

create_orderbook()

Build orderbook using MessageBatch.

rows = ob.create_orderbook(batch)

---

Complete Workflow Example

from fastreader import (
    ReadMsgFromBinary,
    BinaryLoader,
    OrderBookBuilder
)

# Read binary file
reader = ReadMsgFromBinary(
    "market_data.bin",
    token=26000
)

# Print summary
reader.summary()

# Sequential loader
loader = BinaryLoader(reader)

# Stream messages
while True:

    msg = loader.get_next_message()

    if msg == "END":
        break

    print(msg)

# Create batch
batch = reader.select_order_messages()

# Build orderbook
ob = OrderBookBuilder()

rows = ob.create_orderbook(batch)

print("Generated Rows:", rows)

---

Message Format

Messages are returned as formatted strings.

Example:

Order Message:
SeqNo12345,
msg_len64,
Msg_Type'O',
Exch_ts123456789,
local_ts123456790,
order_id11111,
Token26000,
order_Type'B',
Price25000,
Quantity100,
missed0

---

Message Fields

Field	Description
SeqNo	Stream sequence number
msg_len	Packet byte length
Msg_Type	Exchange message type
Exch_ts	Exchange timestamp
local_ts	Local parser timestamp
order_id	Order identifier
order_id_buy	Buy order identifier
order_id_sell	Sell order identifier
Token	Instrument token
order_Type	Order side/type
Price	Price
Quantity	Quantity
missed	Gap indicator

---

Performance

fastreader is optimized for:

• Large exchange files
• Low-latency parsing
• High-throughput processing
• Efficient memory handling
• Sequential replay systems

Because heavy logic is written in Rust:

• Parsing is significantly faster
• Memory usage is lower
• Python overhead is minimized

---

Recommended Workflow

Binary File
    ↓
ReadMsgFromBinary
    ↓
BinaryLoader / MessageBatch
    ↓
OrderBookBuilder
    ↓
Analytics / Alpha / Backtesting

---