orderpulse 0.2.31

High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings

fastreader / OrderPulse

A high-performance Rust + Python library for reading NSE binary market feed files, extracting order/trade messages, and building an in-memory order book snapshot from those messages.

This library is designed for quant developers, market-data engineers, and backtesting systems that need fast parsing of large binary feed files without loading unnecessary data into Python.

---

What this library does

fastreader gives Python users three main tools:

Class	Purpose	Best use case
MessageCacheReader	Loads all order/trade messages into memory first	Backtesting, repeated analysis, debugging, smaller/medium files
StreamingBinaryLoader	Reads one binary message at a time directly from disk	Very large files, low-memory processing, live-style replay
OrderbookBuilder	Applies order/trade messages and builds market depth	Creating snapshots, full depth, top-N levels, payoff/orderbook analytics

Internally, the heavy work is done in Rust for speed, while Python receives simple classes, lists, dictionaries, and strings.

---

Architecture explanation

Binary Feed File
      |
      |  contains raw NSE order/trade packets
      v
+----------------------------+
| Binary Parser Layer        |
| - validates first header   |
| - skips padding spaces     |
| - reads message type       |
| - parses OrderPacket       |
| - parses TradePacket       |
+----------------------------+
      |
      v
+----------------------------+
| Reader Layer               |
|                            |
| MessageCacheReader         |
|   loads all messages       |
|   stores in RAM            |
|                            |
| StreamingBinaryLoader      |
|   keeps file handle open   |
|   reads next message only  |
+----------------------------+
      |
      v
+----------------------------+
| OrderbookBuilder           |
| - applies filters          |
| - processes N/M/X/T msgs   |
| - updates OrderBookManager |
+----------------------------+
      |
      v
+----------------------------+
| Python Output Layer        |
| - snapshot dict            |
| - full depth dict          |
| - CSV row string           |
| - formatted message string |
+----------------------------+

Message flow

1. A binary file is opened.
2. The first valid message header is checked.
3. Each message is identified by msg_type.
4. Supported message types are parsed:
   • N = new order
   • M = modified order
   • X = cancelled/deleted order
   • T = trade
5. OrderbookBuilder sends the parsed message into OrderBookManager.
6. The user asks for snapshot, full depth, or CSV row output.

---

Installation / build

This project is a Rust extension module exposed to Python using pyo3.

Typical local build:

maturin develop --release

Or build wheel:

maturin build --release

Then use in Python:

from fastreader import MessageCacheReader, StreamingBinaryLoader, OrderbookBuilder

---

Quick start: stream file and build order book

from fastreader import StreamingBinaryLoader, OrderbookBuilder

file_path = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

reader = StreamingBinaryLoader()
reader.open_stream(file_path, count_messages=False)

builder = OrderbookBuilder()

while True:
    processed = builder.orderbook_add_msg(reader)
    if not processed:
        break

snapshot = builder.get_snapshot(token=1001, levels=5)
print(snapshot)

Example expected output:

{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}

If no order book exists for that token:

{
    'token': 1001,
    'found': False,
    'mid_price': 0,
    'best_bid': None,
    'best_ask': None,
    'spread': None,
    'bids': [],
    'asks': []
}

---

Public Python API

1. MessageCacheReader

MessageCacheReader loads all order and trade messages into RAM. Use this when you want to read the file once and then analyze the messages many times.

When to use

Use MessageCacheReader when:

• the file is small or medium size,
• you want repeated access to all messages,
• you want message summaries,
• you want to build order book multiple times with different filters.

Avoid it for extremely large binary files because it stores all parsed messages in memory.

---

MessageCacheReader()

Creates an empty cache reader.

from fastreader import MessageCacheReader

reader = MessageCacheReader()

Expected output:

# No output. Empty reader object is created.

---

load_to_cache(file_path)

Loads all supported order/trade messages from a binary file into memory.

Signature

reader.load_to_cache(file_path: str) -> int

Parameters

Parameter	Type	Meaning
file_path	str	Full path of the binary feed file

Returns

Number of messages loaded into cache.

Example

from fastreader import MessageCacheReader

reader = MessageCacheReader()
count = reader.load_to_cache("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin")
print(count)

Expected output:

1254300

Meaning: 1,254,300 order/trade messages were loaded into RAM.

---

get_all_messages()

Returns all cached messages as readable strings.

Signature

reader.get_all_messages() -> list[str]

Example

messages = reader.get_all_messages()
print(messages[:2])

Expected output:

[
    "Order Message: SeqNo 1, MsgLen 38, MsgType 'N', ExchTs 1766998800000000000, LocalTs 1766998800000100000, OrderId 123456789, Token 1001, Side 'B', Price 245700, Quantity 150, Missed 0",
    "Trade Message: SeqNo 2, MsgLen 45, MsgType 'T', ExchTs 1766998801000000000, LocalTs 1766998801000100000, BuyOrderId 123456789, SellOrderId 987654321, Token 1001, Price 245750, Quantity 50, Missed 0"
]

---

get_order_message()

Returns only order messages from the cache.

Signature

reader.get_order_message() -> list[str]

Order messages include:

• N = new order
• M = modify order
• X = cancel/delete order

Example

orders = reader.get_order_message()
print(len(orders))
print(orders[0])

Expected output:

984500
Order Message: SeqNo 1, MsgLen 38, MsgType 'N', ExchTs 1766998800000000000, LocalTs 1766998800000100000, OrderId 123456789, Token 1001, Side 'B', Price 245700, Quantity 150, Missed 0

Note: In the current lib.rs, the function name is get_order_message(), not get_all_order_message().

---

get_trade_message()

Returns only trade messages from the cache.

Signature

reader.get_trade_message() -> list[str]

Example

trades = reader.get_trade_message()
print(len(trades))
print(trades[0])

Expected output:

269800
Trade Message: SeqNo 2, MsgLen 45, MsgType 'T', ExchTs 1766998801000000000, LocalTs 1766998801000100000, BuyOrderId 123456789, SellOrderId 987654321, Token 1001, Price 245750, Quantity 50, Missed 0

---

get_all_trade_message()

Alias for get_trade_message().

Signature

reader.get_all_trade_message() -> list[str]

Example

trades = reader.get_all_trade_message()
print(trades[:1])

Expected output:

[
    "Trade Message: SeqNo 2, MsgLen 45, MsgType 'T', ExchTs 1766998801000000000, LocalTs 1766998801000100000, BuyOrderId 123456789, SellOrderId 987654321, Token 1001, Price 245750, Quantity 50, Missed 0"
]

---

get_cache_summary()

Returns metadata about the cached file and message counts.

Signature

reader.get_cache_summary() -> dict

Example

summary = reader.get_cache_summary()
print(summary)

Expected output:

{
    'file_source': '/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin',
    'total_messages': 1254300,
    'total_orders': 984500,
    'total_trades': 269800,
    'memory_usage_bytes': 80275200
}

---

2. StreamingBinaryLoader

StreamingBinaryLoader keeps a binary file open and reads one message at a time. This is the best choice for large files because it does not load all messages into RAM.

When to use

Use StreamingBinaryLoader when:

• the binary file is very large,
• you want low-memory processing,
• you want to replay feed messages one by one,
• you want to build order book directly from disk.

---

StreamingBinaryLoader()

Creates a new streaming reader.

from fastreader import StreamingBinaryLoader

reader = StreamingBinaryLoader()

Expected output:

# No output. Empty stream reader object is created.

---

open_stream(file_path, count_messages=True)

Opens the binary feed file for streaming.

Signature

reader.open_stream(file_path: str, count_messages: bool = True) -> int

Parameters

Parameter	Type	Meaning
file_path	str	Full path of binary feed file
count_messages	bool	If True, scans the file once to count messages. If False, opens quickly and returns 0.

Important meaning of open_stream

open_stream() does not load the full file into memory. It only:

1. opens the file handle,
2. validates that the file starts with a valid feed message,
3. resets the cursor to the start,
4. optionally counts total messages,
5. keeps the file ready for get_next_message() or orderbook_add_msg().

Example: fast open without counting

reader = StreamingBinaryLoader()
count = reader.open_stream(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin",
    count_messages=False,
)
print(count)

Expected output:

0

Meaning: file is opened, but total message count was skipped for speed.

Example: open and count messages

reader = StreamingBinaryLoader()
count = reader.open_stream(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin",
    count_messages=True,
)
print(count)

Expected output:

1254300

Meaning: file was scanned once and 1,254,300 messages were found.

---

reset_cursor()

Moves the file cursor back to the start.

Signature

reader.reset_cursor() -> None

Example

reader.get_next_message()
reader.get_next_message()

reader.reset_cursor()

first_message_again = reader.get_next_message()
print(first_message_again)

Expected output:

Order Message: SeqNo 1, MsgLen 38, MsgType 'N', ExchTs 1766998800000000000, LocalTs 1766998800000100000, OrderId 123456789, Token 1001, Side 'B', Price 245700, Quantity 150, Missed 0

---

get_next_message()

Reads the next message from the stream and returns it as a formatted string.

Signature

reader.get_next_message() -> str

Example

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

print(reader.get_next_message())
print(reader.get_next_message())

Expected output:

Order Message: SeqNo 1, MsgLen 38, MsgType 'N', ExchTs 1766998800000000000, LocalTs 1766998800000100000, OrderId 123456789, Token 1001, Side 'B', Price 245700, Quantity 150, Missed 0
Trade Message: SeqNo 2, MsgLen 45, MsgType 'T', ExchTs 1766998801000000000, LocalTs 1766998801000100000, BuyOrderId 123456789, SellOrderId 987654321, Token 1001, Price 245750, Quantity 50, Missed 0

When the file ends:

END

---

3. OrderbookBuilder

OrderbookBuilder is the engine that builds and queries the order book.

It accepts messages from:

1. MessageCacheReader,
2. StreamingBinaryLoader,
3. Python list[dict] decoded messages.

---

OrderbookBuilder()

Creates a new empty order book builder.

from fastreader import OrderbookBuilder

builder = OrderbookBuilder()

Expected output:

# No output. Empty order book is created.

---

apply_filter(logic_criteria=None)

Filters which message types should be processed by the order book builder.

Signature

builder.apply_filter(logic_criteria: list[str] | None = None) -> None

Supported message filters

Filter	Meaning
"N"	Process new orders
"M"	Process modified orders
"X"	Process cancelled/deleted orders
"T"	Process trades
None	Process all supported messages

Example: process only order add/modify/delete

builder.apply_filter(["N", "M", "X"])

Expected output:

# No output. Future build/process calls will skip trades.

Example: process only trades

builder.apply_filter(["T"])

Example: clear filter

builder.apply_filter(None)

---

orderbook_add_msg(source)

Reads exactly one message from a StreamingBinaryLoader and applies it to the order book.

Signature

builder.orderbook_add_msg(source: StreamingBinaryLoader) -> bool

Returns

Return	Meaning
True	One message was read and applied to the order book
False	Stream ended, or the message was skipped by filter

Example

from fastreader import StreamingBinaryLoader, OrderbookBuilder

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

builder = OrderbookBuilder()

processed = builder.orderbook_add_msg(reader)
print("Processed:", processed)

Expected output:

Processed: True

Practical loop

while True:
    processed = builder.orderbook_add_msg(reader)
    if not processed:
        break

Important: If you use a filter and the next message is skipped, this function returns False. In that case, a simple break loop may stop early. For full-file filtered processing, prefer build_from_source(reader) because it continues until the stream ends or the limit is reached.

---

build_from_list(source)

Builds order book from either:

1. a MessageCacheReader, or
2. a Python list[dict] containing decoded messages.

Signature

builder.build_from_list(source: MessageCacheReader | list[dict]) -> int

Returns

Number of messages actually processed.

---

Example: build from MessageCacheReader

from fastreader import MessageCacheReader, OrderbookBuilder

reader = MessageCacheReader()
reader.load_to_cache("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin")

builder = OrderbookBuilder()
count = builder.build_from_list(reader)

print("Processed:", count)

Expected output:

Processed: 1254300

---

Example: build from Python list of dictionaries

from fastreader import OrderbookBuilder

messages = [
    {
        "msg_type": "N",
        "exch_ts": 1766998800000000000,
        "order_id": 101,
        "token": 1001,
        "order_type": "B",
        "price": 245700,
        "quantity": 150,
        "local_ts": 1766998800000100000,
        "flags": False,
    },
    {
        "msg_type": "N",
        "exch_ts": 1766998800001000000,
        "order_id": 102,
        "token": 1001,
        "order_type": "S",
        "price": 245800,
        "quantity": 200,
        "local_ts": 1766998800001100000,
        "flags": False,
    },
]

builder = OrderbookBuilder()
count = builder.build_from_list(messages)
print("Processed:", count)
print(builder.get_snapshot(1001, levels=5))

Expected output:

Processed: 2
{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150)],
    'asks': [(245800, 200)]
}

Required dictionary keys for order messages

For msg_type = N, M, or X:

Key	Required	Type	Meaning
msg_type	yes	str or int	N, M, or X
order_id	yes	int	Exchange order id
token	yes	int	Instrument token
order_type	yes	str or int	B for buy, S for sell
price	yes	int	Price in feed tick format
quantity	yes	int	Quantity
exch_ts	no	int	Exchange timestamp; default 0
local_ts	no	int	Local timestamp; default 0
flags	no	bool	Missed flag; default False

Required dictionary keys for trade messages

For msg_type = T:

Key	Required	Type	Meaning
msg_type	yes	str or int	T
buy_order_id	yes	int	Buy order id
sell_order_id	yes	int	Sell order id
token	yes	int	Instrument token
trade_quantity	yes	int	Traded quantity
trade_price	no	int	Traded price; default 0
exch_ts	no	int	Exchange timestamp; default 0
local_ts	no	int	Local timestamp; default 0
flags	no	bool	Missed flag; default False

---

build_from_source(source, limit=None)

Builds the order book from either a cache reader or a streaming reader.

Signature

builder.build_from_source(source: MessageCacheReader | StreamingBinaryLoader, limit: int | None = None) -> int

Parameters

Parameter	Type	Meaning
source	MessageCacheReader or StreamingBinaryLoader	Source of messages
limit	int or None	Maximum number of processed messages for streaming mode

Example: build from stream with no limit

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

builder = OrderbookBuilder()
count = builder.build_from_source(reader)

print("Processed:", count)

Expected output:

Processed: 1254300

Example: process only first 10,000 messages

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

builder = OrderbookBuilder()
count = builder.build_from_source(reader, limit=10_000)

print("Processed:", count)

Expected output:

Processed: 10000

---

get_snapshot(token, levels=None)

Returns top-N order book levels for one instrument token.

Signature

builder.get_snapshot(token: int, levels: int | None = None) -> dict

Default levels is 5.

Example

snapshot = builder.get_snapshot(token=1001, levels=5)
print(snapshot)

Expected output:

{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}

---

get_orderbook_snapshot(token, levels=None)

Alias for get_snapshot().

Signature

builder.get_orderbook_snapshot(token: int, levels: int | None = None) -> dict

Example

snapshot = builder.get_orderbook_snapshot(1001, levels=5)
print(snapshot)

Expected output:

{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}

---

get_full_depth(token)

Returns full available order book depth for one token.

Signature

builder.get_full_depth(token: int) -> dict

Example

full_depth = builder.get_full_depth(1001)
print(full_depth)

Expected output:

{
    'token': 1001,
    'found': True,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [
        (245700, 150),
        (245650, 300),
        (245600, 75),
        (245550, 50)
    ],
    'asks': [
        (245800, 200),
        (245850, 100),
        (245900, 250),
        (245950, 125)
    ]
}

If token is not found:

{
    'token': 999999,
    'found': False,
    'best_bid': None,
    'best_ask': None,
    'spread': None,
    'bids': [],
    'asks': []
}

---

snapshot_header()

Returns the CSV header for get_snapshot_row().

Signature

builder.snapshot_header() -> str

Example

print(builder.snapshot_header())

Expected output:

local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4

---

get_snapshot_row(token, levels=None)

Returns one CSV row string for a token snapshot.

Signature

builder.get_snapshot_row(token: int, levels: int | None = None) -> str

Default levels is 5.

Example

print(builder.snapshot_header())
print(builder.get_snapshot_row(1001, levels=5))

Expected output:

local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
0,0,245750,245700,150,245800,200,245650,300,245850,100,245600,75,245900,250,245550,50,245950,125,0,0,0,0

Current implementation sets local_ts and exch_ts to 0 in the CSV row. The order book levels and mid price come from the order book state.

---

Complete examples

Example 1: Read messages one by one

from fastreader import StreamingBinaryLoader

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

for _ in range(5):
    msg = reader.get_next_message()
    print(msg)

Expected output:

Order Message: SeqNo 1, MsgLen 38, MsgType 'N', ExchTs 1766998800000000000, LocalTs 1766998800000100000, OrderId 123456789, Token 1001, Side 'B', Price 245700, Quantity 150, Missed 0
Trade Message: SeqNo 2, MsgLen 45, MsgType 'T', ExchTs 1766998801000000000, LocalTs 1766998801000100000, BuyOrderId 123456789, SellOrderId 987654321, Token 1001, Price 245750, Quantity 50, Missed 0
...

---

Example 2: Build order book from cache

from fastreader import MessageCacheReader, OrderbookBuilder

file_path = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

data = MessageCacheReader()
loaded = data.load_to_cache(file_path)
print("Loaded:", loaded)

builder = OrderbookBuilder()
processed = builder.build_from_list(data)
print("Processed:", processed)

print(builder.get_snapshot(1001, 5))

Expected output:

Loaded: 1254300
Processed: 1254300
{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}

---

Example 3: Build order book from stream

from fastreader import StreamingBinaryLoader, OrderbookBuilder

file_path = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

reader = StreamingBinaryLoader()
reader.open_stream(file_path, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(reader)

print("Processed:", processed)
print(builder.get_snapshot(1001, 5))

Expected output:

Processed: 1254300
{
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}

---

Example 4: Save snapshots to CSV

from fastreader import StreamingBinaryLoader, OrderbookBuilder

file_path = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
token = 1001

reader = StreamingBinaryLoader()
reader.open_stream(file_path, count_messages=False)

builder = OrderbookBuilder()
builder.build_from_source(reader)

with open("snapshot.csv", "w") as f:
    f.write(builder.snapshot_header() + "\n")
    f.write(builder.get_snapshot_row(token, levels=5) + "\n")

Expected snapshot.csv:

local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
0,0,245750,245700,150,245800,200,245650,300,245850,100,245600,75,245900,250,245550,50,245950,125,0,0,0,0

---

Example 5: Process only order messages, skip trades

from fastreader import StreamingBinaryLoader, OrderbookBuilder

reader = StreamingBinaryLoader()
reader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

builder = OrderbookBuilder()
builder.apply_filter(["N", "M", "X"])

processed = builder.build_from_source(reader)
print("Processed only orders:", processed)

Expected output:

Processed only orders: 984500

---

Error handling

The Rust layer converts common failures into Python exceptions.

File not found

reader = StreamingBinaryLoader()
reader.open_stream("missing_file.bin")

Expected error:

RuntimeError: No such file or directory

Invalid binary file

If the first valid message type is not one of T, N, M, or X, opening the file fails.

Expected error:

RuntimeError: invalid first message type: <value>

Wrong source type

builder = OrderbookBuilder()
builder.build_from_source("not_a_reader")

Expected error:

TypeError: build_from_source expects MessageCacheReader or StreamingBinaryLoader

Missing dictionary key

builder.build_from_list([
    {"msg_type": "N", "token": 1001}
])

Expected error:

TypeError: missing key: order_id

---

Performance guide

Use StreamingBinaryLoader for large files

reader.open_stream(file_path, count_messages=False)
builder.build_from_source(reader)

This avoids loading the full file into RAM.

Use MessageCacheReader for repeated research

reader.load_to_cache(file_path)
builder_1.build_from_list(reader)
builder_2.build_from_list(reader)

This is faster for repeated analysis but uses more memory.

Avoid count_messages=True for production replay

count_messages=True scans the file once before real processing. For very large files, prefer:

reader.open_stream(file_path, count_messages=False)

---

Recommended user workflows

For quick debugging

reader = StreamingBinaryLoader()
reader.open_stream(file_path, count_messages=False)
print(reader.get_next_message())

For backtesting

cache = MessageCacheReader()
cache.load_to_cache(file_path)

builder = OrderbookBuilder()
builder.build_from_list(cache)

For production-style low-memory processing

stream = StreamingBinaryLoader()
stream.open_stream(file_path, count_messages=False)

builder = OrderbookBuilder()
builder.build_from_source(stream)

For one-token snapshot

snapshot = builder.get_snapshot(token=1001, levels=5)

For full book depth

depth = builder.get_full_depth(token=1001)

For CSV export

header = builder.snapshot_header()
row = builder.get_snapshot_row(token=1001, levels=5)

---

Developer notes from current lib.rs

The current module exposes only these Python classes:

MessageCacheReader
StreamingBinaryLoader
OrderbookBuilder

The current module exports these Python-visible methods:

MessageCacheReader.new
MessageCacheReader.load_to_cache
MessageCacheReader.get_all_messages
MessageCacheReader.get_order_message
MessageCacheReader.get_trade_message
MessageCacheReader.get_all_trade_message
MessageCacheReader.get_cache_summary

StreamingBinaryLoader.new
StreamingBinaryLoader.open_stream
StreamingBinaryLoader.reset_cursor
StreamingBinaryLoader.get_next_message

OrderbookBuilder.new
OrderbookBuilder.apply_filter
OrderbookBuilder.orderbook_add_msg
OrderbookBuilder.build_from_list
OrderbookBuilder.build_from_source
OrderbookBuilder.get_full_depth
OrderbookBuilder.get_snapshot
OrderbookBuilder.get_orderbook_snapshot
OrderbookBuilder.snapshot_header
OrderbookBuilder.get_snapshot_row

These helper functions exist internally in Rust but are not directly exposed to Python:

format_message
format_snapshot_row
parse_order_packet
parse_trade_packet
validate_binary_header
read_next_message_from_file
count_messages_in_file

---

Small naming suggestion

Your current function name is:

get_order_message()

You already added an alias for trades:

get_all_trade_message()

For symmetry and easier user experience, you may also add this alias in lib.rs:

pub fn get_all_order_message(&self) -> Vec<String> {
    self.get_order_message()
}

Then users can write:

orders = reader.get_all_order_message()
trades = reader.get_all_trade_message()

This is optional but cleaner for the public API.

---

Best practice summary

Need	Use
Read entire file once and inspect many times	MessageCacheReader
Process huge file with low memory	StreamingBinaryLoader
Build top-5 market depth	OrderbookBuilder.get_snapshot()
Build full order book depth	OrderbookBuilder.get_full_depth()
Export snapshot as CSV	snapshot_header() + get_snapshot_row()
Skip trades	apply_filter(["N", "M", "X"])
Process only first N streaming messages	build_from_source(reader, limit=N)

---

Minimal working script

from fastreader import StreamingBinaryLoader, OrderbookBuilder

FILE_PATH = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
TOKEN = 1001

reader = StreamingBinaryLoader()
reader.open_stream(FILE_PATH, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(reader)

print("Processed messages:", processed)
print("Snapshot:", builder.get_snapshot(TOKEN, levels=5))
print("CSV header:", builder.snapshot_header())
print("CSV row:", builder.get_snapshot_row(TOKEN, levels=5))

Expected output:

Processed messages: 1254300
Snapshot: {
    'token': 1001,
    'found': True,
    'mid_price': 245750,
    'best_bid': (245700, 150),
    'best_ask': (245800, 200),
    'spread': 100,
    'bids': [(245700, 150), (245650, 300), (245600, 75)],
    'asks': [(245800, 200), (245850, 100), (245900, 250)]
}
CSV header: local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,...
CSV row: 0,0,245750,245700,150,245800,200,245650,300,245850,100,...