orderpulse 0.2.49

High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings

fastreader / OrderPulse

High-performance Python bindings for reading NSE order and trade binary feed files, enriching messages with contract metadata, and building order book snapshots.

fastreader is designed for Python users who need Rust-speed parsing with a simple Jupyter-friendly API. The Rust backend reads NSE binary feed files, decodes order/trade packets, exposes each message as a Python dictionary through PyO3, optionally enriches tokens using a symbol master CSV, and builds market-depth/order-book outputs.

Note: All example outputs in this README use realistic placeholder values. Actual values depend on your binary feed file and contract master CSV.

---

1. What this library does

Use fastreader / OrderPulse when you want to:

• Parse NSE binary order and trade feed files from Python.
• Load all messages into memory for fast repeated analysis.
• Stream large feed files one message at a time without loading everything into memory.
• Convert Rust-decoded feed packets into Python dictionaries.
• Filter order and trade messages.
• Enrich messages with token metadata such as symbol, strike, option type, expiry, lot size, and contract name.
• Build an order book from cached messages or from a streaming reader.
• Export top-of-book and market-depth snapshots.
• Construct standard feed file paths for NSE CM and FO streams.

---

2. Architecture overview

The library follows this flow:

Binary feed file
    -> Rust binary parser
    -> Python message dictionaries
    -> optional SymbolMaster enrichment
    -> OrderbookBuilder
    -> snapshot / full-depth output

Main internal components:

Component	Purpose
Rust parser	Reads binary files and decodes order/trade packets.
PyO3 bindings	Exposes Rust classes and methods to Python.
MessageCacheReader	Loads all decoded messages into memory for repeated analysis.
StreamingBinaryLoader	Opens a file and reads messages sequentially using get_next_msg().
SymbolMaster	Loads NSE contract master CSV and maps token to contract metadata.
OrderbookBuilder	Processes order/trade messages and builds token-wise depth.
FeedPathBuilder	Creates standard NSE feed file paths from segment/date/stream inputs.

---

3. Public API

The Python module exports these classes:

from fastreader import (
    MessageCacheReader,
    StreamingBinaryLoader,
    OrderbookBuilder,
    SymbolMaster,
    FeedPathBuilder,
)

---

4. Installation

From PyPI

pip install fastreader

From a local source checkout

pip install maturin
maturin develop --release

For a production wheel build:

maturin build --release
pip install target/wheels/*.whl

Verify installation

import fastreader
from fastreader import MessageCacheReader, StreamingBinaryLoader, OrderbookBuilder, SymbolMaster, FeedPathBuilder

print("fastreader imported successfully")

Expected output:

fastreader imported successfully

---

5. Required input files

5.1 Binary feed file

The binary file must contain supported NSE feed messages. The parser recognizes these message types:

Message type	Meaning in this library
N	New order message
M	Modify order message
X	Cancel/delete order message
T	Trade message

The binary header must be valid. If the file is empty, truncated, or starts with an unsupported first message type, the loader raises a runtime error.

5.2 Symbol master CSV file

SymbolMaster expects a CSV with these columns:

Required column	Used as
FinInstrmId	Token / instrument id
TckrSymb	Symbol, for example NIFTY
XpryDt	Expiry timestamp, converted to readable date
StrkPric	Strike price; divided by 100
OptnTp	Option type, for example CE, PE, or XX
StockNm	Full contract name

Optional lot-size column:

• NewBrdLotQty, preferred
• MinLot, fallback

---

6. Message dictionary format

Order messages returned by get_order_message() or get_next_msg() look like this:

{
    "message_kind": "order",
    "seq_no": 42,
    "msg_len": 48,
    "stream_id": 2,
    "msg_type": "N",
    "exch_ts": 1716269400123456789,
    "local_ts": 1716269401123456789,
    "order_id": 12345678901,
    "token": 40434,
    "order_type": "B",
    "price": 2195050,
    "quantity": 50,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}

Trade messages look like this:

{
    "message_kind": "trade",
    "seq_no": 99,
    "msg_len": 48,
    "stream_id": 2,
    "msg_type": "T",
    "exch_ts": 1716269402123456789,
    "local_ts": 1716269403123456789,
    "buy_order_id": 111111111,
    "sell_order_id": 222222222,
    "token": 40434,
    "trade_price": 2195100,
    "trade_quantity": 50,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}

After symbol enrichment, additional keys can be available:

{
    "token_symbol": "NIFTY",
    "strike_price": 21950,
    "option_type": "CE",
    "expiry": "30-May-2024",
    "lot_size": 50,
    "name": "NIFTY24MAY21950CE",
}

---

7. Typical workflow

from fastreader import MessageCacheReader, SymbolMaster, OrderbookBuilder

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

def show(value):
    print(value)

# 1. Load symbol metadata
sm = SymbolMaster()
sm_count = sm.load(symbol_csv)
show(sm_count)

# 2. Load binary messages into cache
reader = MessageCacheReader()
message_count = reader.load_to_cache(feed_path)
show(reader.get_cache_summary())

# 3. Build order book
builder = OrderbookBuilder()
processed = builder.build_from_list(reader)
show(processed)

# 4. Inspect active tokens and snapshot
active_tokens = builder.get_active_tokens()
show(active_tokens[:5])

if active_tokens:
    token = active_tokens[0]
    show(builder.get_snapshot(token, levels=5))

Expected output:

95632
{'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}
1180000
[40434, 40435, 40436, 50120, 50121]
{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), ...], 'asks': [(2195100, 50), ...]}

---

8. MessageCacheReader

MessageCacheReader loads the entire binary feed into memory. Use it when the file is small enough to fit comfortably in RAM or when you need to query the same file multiple times.

8.1 Create a reader

from fastreader import MessageCacheReader

reader = MessageCacheReader()
print(reader.get_cache_summary())

Expected output:

{'file_source': None, 'total_messages': 0, 'total_orders': 0, 'total_trades': 0, 'memory_usage_bytes': 0}

8.2 load_to_cache(file_path)

Loads a binary feed file into memory.

Parameters:

Parameter	Type	Required	Description
file_path	str	Yes	Path to the NSE binary feed file.

Returns:

• int: number of decoded messages loaded.

Possible errors:

• File does not exist.
• File is empty.
• Invalid first message type.
• Truncated header or payload.

Example:

from fastreader import MessageCacheReader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(feed_path)

print("Loaded messages:", count)

Expected output:

Loaded messages: 1250000

8.3 get_all_messages()

Returns every cached message as a human-readable string. Use this for quick inspection, logging, or debugging.

Parameters: none.

Returns:

• list[str]

Example:

messages = reader.get_all_messages()

print("Total formatted messages:", len(messages))
print(messages[0])

Expected output:

Total formatted messages: 1250000
Order Message: SeqNo 42, MsgLen 48, MsgType 'N', ExchTs 1716269400123456789, LocalTs 1716269401123456789, OrderId 12345678901, Token 40434, Side 'B', Price 2195050, Quantity 50, Missed 0

8.4 get_order_message()

Returns only order messages as Python dictionaries.

Parameters: none.

Returns:

• list[dict]

Example:

orders = reader.get_order_message()

print("Order count:", len(orders))
print(orders[0])

Expected output:

Order count: 1180000
{'message_kind': 'order', 'seq_no': 42, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'N', 'exch_ts': 1716269400123456789, 'local_ts': 1716269401123456789, 'order_id': 12345678901, 'token': 40434, 'order_type': 'B', 'price': 2195050, 'quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}

8.5 get_trade_message()

Returns only trade messages as Python dictionaries.

Parameters: none.

Returns:

• list[dict]

Example:

trades = reader.get_trade_message()

print("Trade count:", len(trades))
print(trades[0])

Expected output:

Trade count: 70000
{'message_kind': 'trade', 'seq_no': 99, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'T', 'exch_ts': 1716269402123456789, 'local_ts': 1716269403123456789, 'buy_order_id': 111111111, 'sell_order_id': 222222222, 'token': 40434, 'trade_price': 2195100, 'trade_quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}

8.6 get_all_trade_message()

Alias for get_trade_message().

Example:

trades = reader.get_all_trade_message()
print(len(trades))

Expected output:

70000

8.7 get_cache_summary()

Returns a summary of the currently cached file.

Parameters: none.

Returns:

• dict with file_source, total_messages, total_orders, total_trades, and memory_usage_bytes.

Example:

summary = reader.get_cache_summary()
print(summary)

Expected output:

{'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}

---

9. StreamingBinaryLoader

StreamingBinaryLoader reads one message at a time. Use it for large files, real-time-style processing, or when you do not want to hold the complete file in memory.

9.1 Create a streaming reader

from fastreader import StreamingBinaryLoader

stream = StreamingBinaryLoader()
print(stream.is_end_of_msg())

Expected output:

True

A new unopened stream reports end-of-message as True because no file is currently open.

9.2 open_stream(file_path, count_messages=True)

Opens a binary feed file for sequential reading.

Parameters:

Parameter	Type	Required	Default	Description
file_path	str	Yes	-	Binary feed path.
count_messages	bool	No	True	If true, scans the file to count messages before reading. For very large files, use False for faster startup.

Returns:

• int: total message count if count_messages=True; otherwise 0.

Example:

from fastreader import StreamingBinaryLoader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

stream = StreamingBinaryLoader()
count = stream.open_stream(feed_path, count_messages=True)
print("Messages in stream:", count)

Expected output:

Messages in stream: 1250000

Fast open without counting:

stream = StreamingBinaryLoader()
count = stream.open_stream(feed_path, count_messages=False)
print(count)

Expected output:

0

9.3 get_next_msg()

Reads the next available message and returns it as a Python dictionary. Returns None at EOF.

Parameters: none.

Returns:

• dict | None

Example:

msg = stream.get_next_msg()
print(msg)

Expected output:

{'message_kind': 'order', 'seq_no': 42, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'N', 'exch_ts': 1716269400123456789, 'local_ts': 1716269401123456789, 'order_id': 12345678901, 'token': 40434, 'order_type': 'B', 'price': 2195050, 'quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}

Read in a loop:

stream.reset_cursor()

while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    print(msg["seq_no"], msg["msg_type"], msg["token"])

Expected output:

42 N 40434
43 M 40434
44 T 40434

9.4 is_end_of_msg()

Checks whether the stream cursor is at the end of the file. It peeks ahead and then restores the current cursor position.

Parameters: none.

Returns:

• bool: True if there is no next message, otherwise False.

Example:

stream.reset_cursor()

print("Before reading:", stream.is_end_of_msg())
first = stream.get_next_msg()
print("After one read:", stream.is_end_of_msg())

Expected output:

Before reading: False
After one read: False

At the end:

while stream.get_next_msg() is not None:
    pass

print(stream.is_end_of_msg())

Expected output:

True

9.5 reset_cursor()

Moves the stream cursor back to the beginning of the opened file.

Parameters: none.

Returns:

• None

Example:

stream.get_next_msg()
stream.get_next_msg()

stream.reset_cursor()
first_msg_again = stream.get_next_msg()

print(first_msg_again["seq_no"])

Expected output:

42

9.6 attach_symbol_master(master)

Attaches a loaded SymbolMaster. After this, every get_next_msg() result is automatically enriched if the token exists in the symbol master.

Parameters:

Parameter	Type	Required	Description
master	SymbolMaster	Yes	Loaded symbol master object.

Returns:

• None

Example:

from fastreader import StreamingBinaryLoader, SymbolMaster

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
sm.load(symbol_csv)

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

msg = stream.get_next_msg()
print(msg["token"], msg["token_symbol"], msg["strike_price"], msg["option_type"], msg["expiry"], msg["lot_size"], msg["name"])

Expected output:

40434 NIFTY 21950 CE 30-May-2024 50 NIFTY24MAY21950CE

9.7 detach_symbol_master()

Removes the attached symbol master. Future messages will again have token_symbol, strike_price, and option_type as None unless manually enriched.

Example:

stream.detach_symbol_master()
msg = stream.get_next_msg()
print(msg.get("token_symbol"), msg.get("strike_price"), msg.get("option_type"))

Expected output:

None None None

---

10. SymbolMaster

SymbolMaster loads NSE FO/CM contract master CSV data and gives fast token-to-contract lookups.

10.1 Create a symbol master

from fastreader import SymbolMaster

sm = SymbolMaster()
print(sm)
print(len(sm))

Expected output:

SymbolMaster(contracts=0)
0

10.2 load(csv_path)

Loads an explicit contract master CSV file.

Parameters:

Parameter	Type	Required	Description
csv_path	str	Yes	Full path to the NSE contract master CSV.

Returns:

• int: number of contracts loaded.

Possible errors:

• CSV file does not exist.
• Required columns are missing.
• File cannot be read.

Example:

from fastreader import SymbolMaster

symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
count = sm.load(symbol_csv)

print("Contracts loaded:", count)
print(sm)

Expected output:

Contracts loaded: 95632
SymbolMaster(contracts=95632)

10.3 load_for_date(segment, day, month, year, base_path=None)

Builds the standard NSE contract master path and loads the CSV.

Path pattern:

{base_path}/CONTRACT/{DD}_{MM}_{YYYY}/NSE_{FO|CM}_contract_{DD}{MM}{YYYY}.csv

Parameters:

Parameter	Type	Required	Default	Description
segment	str	Yes	-	NSE_FO, FO, NSE_CM, or CM.
day	int	Yes	-	Day of month.
month	int	Yes	-	Month number.
year	int	Yes	-	Four-digit year.
base_path	`str	None`	No	/nas/50.30

Returns:

• int: number of contracts loaded.

Example:

sm = SymbolMaster()
count = sm.load_for_date("NSE_FO", day=21, month=5, year=2026, base_path="/data")
print(count)

Expected output:

95632

Possible error:

RuntimeError: unknown segment 'BSE_FO' - expected NSE_FO, FO, NSE_CM, or CM

10.4 lookup(token)

Looks up a single token.

Parameters:

Parameter	Type	Required	Description
token	int	Yes	Instrument token.

Returns:

• dict with found=True and metadata if token exists.
• dict with found=False and None fields if token is unknown.

Example:

info = sm.lookup(40434)
print(info)

Expected output:

{'token': 40434, 'found': True, 'symbol': 'NIFTY', 'name': 'NIFTY24MAY21950CE', 'option_type': 'CE', 'strike': 21950, 'expiry': '30-May-2024', 'lot_size': 50}

Unknown token:

print(sm.lookup(999999999))

Expected output:

{'token': 999999999, 'found': False, 'symbol': None, 'name': None, 'option_type': None, 'strike': None, 'expiry': None, 'lot_size': None}

10.5 enrich(msg)

Enriches a message dictionary in place. Use it when you already have a message from get_next_msg() and want to add contract metadata manually.

Parameters:

Parameter	Type	Required	Description
msg	dict	Yes	Message dictionary returned by StreamingBinaryLoader.get_next_msg().

Returns:

• bool: True if token was found and metadata was added, otherwise False.

Example:

stream.reset_cursor()
msg = stream.get_next_msg()

found = sm.enrich(msg)

print("Found:", found)
print(msg["token_symbol"], msg["strike_price"], msg["option_type"], msg["expiry"], msg["lot_size"], msg["name"])

Expected output:

Found: True
NIFTY 21950 CE 30-May-2024 50 NIFTY24MAY21950CE

---

11. OrderbookBuilder

OrderbookBuilder builds and queries token-wise order book state from decoded feed messages.

Use it when you want:

• Active tokens.
• Full bid/ask depth.
• Top N levels.
• CSV snapshot rows.
• Message type filters.
• Incremental streaming order-book updates.

11.1 Create an order book builder

from fastreader import OrderbookBuilder

builder = OrderbookBuilder()
print(builder.get_active_tokens())

Expected output:

[]

11.2 apply_filter(logic_criteria=None)

Restricts which message types are processed by the builder.

Parameters:

Parameter	Type	Required	Description
logic_criteria	`list[str]	None`	No

Returns:

• None

Example: process only new and modify order messages.

builder = OrderbookBuilder()
builder.apply_filter(["N", "M"])

processed = builder.build_from_list(reader)
print(processed)

Expected output:

950000

Clear filter:

builder.apply_filter(None)

11.3 build_from_list(source)

Builds the order book from either:

• a MessageCacheReader, or
• a list[dict] of decoded messages.

Parameters:

Parameter	Type	Required	Description
source	`MessageCacheReader	list[dict]`	Yes

Returns:

• int: number of messages accepted and processed.

Example using MessageCacheReader:

builder = OrderbookBuilder()
processed = builder.build_from_list(reader)
print("Processed:", processed)

Expected output:

Processed: 1180000

Example using a message list:

orders = reader.get_order_message()

builder = OrderbookBuilder()
processed = builder.build_from_list(orders)

print(processed)

Expected output:

1180000

Possible errors:

• Source is neither MessageCacheReader nor list[dict].
• Message dictionary is missing required keys.
• Unsupported msg_type.

11.4 build_from_source(source, limit=None)

Builds the order book from either a cached reader or a streaming reader.

Parameters:

Parameter	Type	Required	Default	Description
source	`MessageCacheReader	StreamingBinaryLoader`	Yes	Source reader.
limit	`int	None`	No	None

Returns:

• int: number of accepted messages processed.

Example from cache:

builder = OrderbookBuilder()
processed = builder.build_from_source(reader)
print(processed)

Expected output:

1180000

Example from stream:

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(stream, limit=10000)

print(processed)

Expected output:

10000

11.5 orderbook_add_msg(msg)

Adds one decoded message dictionary into the order book. Use this for step-by-step streaming processing.

Parameters:

Parameter	Type	Required	Description
msg	dict	Yes	One message returned by get_next_msg().

Returns:

• bool: True if accepted and applied; False if skipped by filters or business rules.

Example:

stream.reset_cursor()
builder = OrderbookBuilder()

while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    accepted = builder.orderbook_add_msg(msg)
    if accepted:
        token = msg["token"]
        snapshot = builder.get_snapshot(token, levels=5)
        print(snapshot)
        break

Expected output:

{'token': 40434, 'found': True, 'mid_price': 2195050, 'best_bid': (2195050, 50), 'best_ask': None, 'spread': None, 'bids': [(2195050, 50)], 'asks': []}

11.6 get_active_tokens()

Returns tokens currently present in the order book.

Parameters: none.

Returns:

• list[int]

Example:

active_tokens = builder.get_active_tokens()
print(active_tokens[:10])

Expected output:

[40434, 40435, 40436, 50120, 50121]

11.7 get_full_depth(token)

Returns complete bid/ask depth for a token.

Parameters:

Parameter	Type	Required	Description
token	int	Yes	Instrument token.

Returns:

• dict with token, found flag, best bid, best ask, spread, bids, and asks.

Example:

depth = builder.get_full_depth(40434)
print(depth)

Expected output:

{'token': 40434, 'found': True, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), (2195000, 250)], 'asks': [(2195100, 50), (2195150, 300)]}

Unknown token:

print(builder.get_full_depth(999999999))

Expected output:

{'token': 999999999, 'found': False, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}

11.8 get_snapshot(token, levels=None)

Returns top N bid/ask levels for one token. Default depth is 5 levels.

Parameters:

Parameter	Type	Required	Default	Description
token	int	Yes	-	Instrument token.
levels	`int	None`	No	5

Returns:

• dict with token, found flag, mid price, best bid, best ask, spread, bids, and asks.

Example:

snapshot = builder.get_snapshot(40434, levels=5)
print(snapshot)

Expected output:

{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), (2195000, 250)], 'asks': [(2195100, 50), (2195150, 300)]}

Unknown token:

print(builder.get_snapshot(999999999, levels=5))

Expected output:

{'token': 999999999, 'found': False, 'mid_price': 0, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}

11.9 snapshot_header()

Returns the CSV header for a 5-level snapshot row.

Parameters: none.

Returns:

• str

Example:

header = builder.snapshot_header()
print(header)

Expected output:

local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4

11.10 get_snapshot_row(token, levels=None)

Returns one CSV-formatted snapshot row. The current implementation emits 5 bid/ask levels and fills missing levels with zeros.

Parameters:

Parameter	Type	Required	Default	Description
token	int	Yes	-	Instrument token.
levels	`int	None`	No	5

Returns:

• str: CSV row.

Example:

print(builder.snapshot_header())
print(builder.get_snapshot_row(40434, levels=5))

Expected output:

local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
0,0,2195075,2195050,100,2195100,50,2195000,250,2195150,300,0,0,0,0,0,0,0,0,0,0,0,0

---

12. FeedPathBuilder

FeedPathBuilder creates standard NSE feed file paths.

12.1 Create a builder

from fastreader import FeedPathBuilder

path_builder = FeedPathBuilder()
print(path_builder)

Expected output:

FeedPathBuilder()

12.2 build(segment, stream_id, day, month, year, base_path=None)

Builds a path string without checking whether the file exists.

Parameters:

Parameter	Type	Required	Default	Description
segment	str	Yes	-	NSE_CM, CM, NSE_FO, or FO.
stream_id	int	Yes	-	Positive stream id.
day	int	Yes	-	Day of month.
month	int	Yes	-	Month number.
year	int	Yes	-	Four-digit year.
base_path	`str	None`	No	/nas/50.30

Returns:

• str: constructed file path.

Example:

builder = FeedPathBuilder()
path = builder.build("NSE_CM", stream_id=2, day=29, month=12, year=2025)
print(path)

Expected output:

/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin

Custom base path:

path = builder.build("NSE_FO", stream_id=1, day=1, month=6, year=2026, base_path="/mnt/data")
print(path)

Expected output:

/mnt/data/NSE_FO/Feed_FO_StreamID_1_1_6_2026.bin

12.3 build_and_verify(segment, stream_id, day, month, year, base_path=None)

Builds the path and verifies that the file exists on disk.

Returns:

• str: constructed file path if it exists.

Possible errors:

• Segment is unknown.
• Stream id or date is invalid.
• File does not exist.

Example:

path = builder.build_and_verify("NSE_CM", stream_id=2, day=29, month=12, year=2025)
print(path)

Expected output:

/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin

---

13. Complete Jupyter examples

13.1 Load a file, inspect orders and trades

from fastreader import MessageCacheReader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(feed_path)

orders = reader.get_order_message()
trades = reader.get_trade_message()
summary = reader.get_cache_summary()

print("Total:", count)
print("Orders:", len(orders))
print("Trades:", len(trades))
print("Summary:", summary)
print("First order:", orders[0] if orders else None)
print("First trade:", trades[0] if trades else None)

Expected output:

Total: 1250000
Orders: 1180000
Trades: 70000
Summary: {'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}
First order: {'message_kind': 'order', 'seq_no': 42, ...}
First trade: {'message_kind': 'trade', 'seq_no': 99, ...}

13.2 Stream messages one by one and detect EOF

from fastreader import StreamingBinaryLoader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

read_count = 0
while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    read_count += 1
    if read_count <= 3:
        print(msg)

print("Read:", read_count)
print("EOF:", stream.is_end_of_msg())

Expected output:

{'message_kind': 'order', 'seq_no': 42, ...}
{'message_kind': 'order', 'seq_no': 43, ...}
{'message_kind': 'trade', 'seq_no': 44, ...}
Read: 1250000
EOF: True

13.3 Enrich streamed messages with token metadata

from fastreader import StreamingBinaryLoader, SymbolMaster

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
sm.load(symbol_csv)

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

msg = stream.get_next_msg()
print({
    "token": msg["token"],
    "token_symbol": msg.get("token_symbol"),
    "strike_price": msg.get("strike_price"),
    "option_type": msg.get("option_type"),
    "expiry": msg.get("expiry"),
    "lot_size": msg.get("lot_size"),
    "name": msg.get("name"),
})

Expected output:

{'token': 40434, 'token_symbol': 'NIFTY', 'strike_price': 21950, 'option_type': 'CE', 'expiry': '30-May-2024', 'lot_size': 50, 'name': 'NIFTY24MAY21950CE'}

13.4 Build order book from cached messages

from fastreader import MessageCacheReader, OrderbookBuilder

reader = MessageCacheReader()
reader.load_to_cache(feed_path)

builder = OrderbookBuilder()
processed = builder.build_from_list(reader)

print("Processed:", processed)
print("Active tokens:", builder.get_active_tokens()[:10])

Expected output:

Processed: 1180000
Active tokens: [40434, 40435, 40436, 50120, 50121]

13.5 Build order book from a streaming reader

from fastreader import StreamingBinaryLoader, OrderbookBuilder

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(stream, limit=50000)

print("Processed:", processed)
print("Active tokens:", builder.get_active_tokens()[:10])

Expected output:

Processed: 50000
Active tokens: [40434, 40435, 40436]

13.6 Build incrementally and print snapshot after each accepted message

stream.reset_cursor()
builder = OrderbookBuilder()

for _ in range(5):
    msg = stream.get_next_msg()
    if msg is None:
        break

    accepted = builder.orderbook_add_msg(msg)
    print("accepted:", accepted, "msg_type:", msg["msg_type"], "token:", msg["token"])

    if accepted:
        print(builder.get_snapshot(msg["token"], levels=5))

Expected output:

accepted: True msg_type: N token: 40434
{'token': 40434, 'found': True, 'mid_price': 2195050, 'best_bid': (2195050, 50), 'best_ask': None, 'spread': None, 'bids': [(2195050, 50)], 'asks': []}
accepted: True msg_type: M token: 40434
{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 50), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 50)], 'asks': [(2195100, 50)]}

---

14. Troubleshooting

14.1 Missing file path

Symptom:

RuntimeError: No such file or directory

Cause:

• The binary feed path or CSV path does not exist.

Fix:

from pathlib import Path

path = Path(feed_path)
print(path.exists(), path)

Use FeedPathBuilder.build_and_verify() when you want the library to validate the path for you.

14.2 Invalid binary header

Symptom:

RuntimeError: invalid first message type: 65

Cause:

• The file is not a supported NSE binary feed file.
• The file starts with a message type other than N, M, X, or T.
• The file is compressed, text, or otherwise not raw binary feed data.

Fix:

• Confirm the file source.
• Confirm it is not zipped or partially copied.
• Test another known-good feed file.

14.3 Empty or incomplete file

Symptom:

RuntimeError: file is empty or missing a complete message header

Cause:

• Empty file.
• File copy did not finish.
• Wrong path points to a placeholder file.

Fix:

from pathlib import Path

p = Path(feed_path)
print(p.exists(), p.stat().st_size)

14.4 Truncated message header

Symptom:

RuntimeError: truncated message header

Cause:

• The file ended before a full header could be read.

Fix:

• Recopy the source file.
• Compare file size with the original location.

14.5 Truncated message payload

Symptom:

RuntimeError: truncated message payload

Cause:

• Header was present, but the complete packet body was missing.
• File is corrupted or partially copied.

Fix:

• Recopy the feed file.
• Validate the file was completely written before reading.

14.6 Missing symbol master CSV columns

Symptom:

RuntimeError: column 'FinInstrmId' not found in /path/to/file.csv

Cause:

• The CSV is not the expected contract master file.
• Required columns are missing or renamed.

Required columns:

FinInstrmId,TckrSymb,XpryDt,StrkPric,OptnTp,StockNm

Optional columns:

NewBrdLotQty,MinLot

Fix:

import pandas as pd

cols = pd.read_csv(symbol_csv, nrows=0).columns.tolist()
print(cols)

14.7 Unknown segment name

Symptom:

RuntimeError: unknown segment 'BSE_FO' - expected NSE_FO, FO, NSE_CM, or CM

Cause:

• load_for_date() or FeedPathBuilder received an unsupported segment.

Fix:

Use one of:

NSE_FO, FO, NSE_CM, CM

14.8 Unsupported message type

Symptom:

TypeError: unsupported msg_type: A

Cause:

• You passed a custom dictionary into OrderbookBuilder with an unsupported msg_type.

Supported message types:

N, M, X, T

Fix:

for msg in messages:
    if msg["msg_type"] not in {"N", "M", "X", "T"}:
        print("Unsupported:", msg)

14.9 Empty order book snapshot

Symptom:

{'token': 40434, 'found': False, 'mid_price': 0, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}

Possible causes:

• The token has not received any accepted order-book messages yet.
• The builder was not populated.
• A filter skipped all relevant messages.
• You queried the wrong token.
• Only trade messages were processed, and no book levels were available.

Fix:

print("Active tokens:", builder.get_active_tokens()[:20])
print("Cache summary:", reader.get_cache_summary())

builder.apply_filter(None)
processed = builder.build_from_list(reader)
print("Processed:", processed)

---

15. Best practices

1. Use MessageCacheReader for smaller files and repeated analysis.
2. Use StreamingBinaryLoader for very large files or incremental processing.
3. Use count_messages=False for faster stream startup on large files.
4. Always check get_cache_summary() after loading a file.
5. Use SymbolMaster when users need readable symbol, strike, option type, expiry, lot size, and contract name.
6. Attach SymbolMaster to StreamingBinaryLoader for automatic enrichment.
7. Use apply_filter(["N", "M", "X"]) when you want to exclude trades from order-book processing.
8. Use get_active_tokens() before querying snapshots.
9. Treat prices as feed-scaled integer prices unless your application converts them to rupees/points.
10. Keep feed binary and contract master CSV from the same trading date whenever possible.

---

16. Quick API reference

MessageCacheReader

Method	Returns	Purpose
load_to_cache(file_path)	int	Load all feed messages into memory.
get_all_messages()	list[str]	Get formatted string messages.
get_order_message()	list[dict]	Get only order messages.
get_trade_message()	list[dict]	Get only trade messages.
get_all_trade_message()	list[dict]	Alias for trade messages.
get_cache_summary()	dict	Get file source, counts, and memory usage.

StreamingBinaryLoader

Method	Returns	Purpose
open_stream(file_path, count_messages=True)	int	Open a file for sequential reading.
get_next_msg()	`dict	None`
is_end_of_msg()	bool	Check EOF without advancing cursor.
reset_cursor()	None	Seek back to file start.
attach_symbol_master(master)	None	Enable auto-enrichment.
detach_symbol_master()	None	Disable auto-enrichment.

SymbolMaster

Method	Returns	Purpose
load(csv_path)	int	Load contract master from explicit path.
load_for_date(segment, day, month, year, base_path=None)	int	Build standard contract path and load.
lookup(token)	dict	Get metadata for token.
enrich(msg)	bool	Enrich one message dictionary in place.
len(sm)	int	Number of loaded contracts.
repr(sm)	str	Display contract count.

OrderbookBuilder

Method	Returns	Purpose
apply_filter(logic_criteria=None)	None	Restrict processed message types.
build_from_list(source)	int	Build from cache or list of dicts.
build_from_source(source, limit=None)	int	Build from cache or stream.
orderbook_add_msg(msg)	bool	Add one decoded message.
get_active_tokens()	list[int]	Tokens with active book state.
get_full_depth(token)	dict	Full bid/ask depth.
get_snapshot(token, levels=None)	dict	Top N levels.
snapshot_header()	str	CSV snapshot header.
get_snapshot_row(token, levels=None)	str	CSV snapshot row.

FeedPathBuilder

Method	Returns	Purpose
build(segment, stream_id, day, month, year, base_path=None)	str	Build feed file path.
build_and_verify(segment, stream_id, day, month, year, base_path=None)	str	Build path and verify file exists.

---

17. Minimal end-to-end example

from fastreader import FeedPathBuilder, SymbolMaster, StreamingBinaryLoader, OrderbookBuilder

base_path = "/data"
day, month, year = 21, 5, 2026
stream_id = 1
segment = "NSE_FO"

# Build paths
path_builder = FeedPathBuilder()
feed_path = path_builder.build(segment, stream_id, day, month, year, base_path=base_path)

# Load symbol master
sm = SymbolMaster()
sm.load_for_date(segment, day, month, year, base_path=base_path)

# Stream binary feed
stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

# Build order book from first 100000 accepted messages
book = OrderbookBuilder()
processed = book.build_from_source(stream, limit=100000)

print("Processed:", processed)
print("Active tokens:", book.get_active_tokens()[:10])

# Query first active token
active = book.get_active_tokens()
if active:
    token = active[0]
    print("Token info:", sm.lookup(token))
    print("Snapshot:", book.get_snapshot(token, levels=5))
    print("CSV header:", book.snapshot_header())
    print("CSV row:", book.get_snapshot_row(token, levels=5))
else:
    print("No active tokens found. Check feed file, message filters, and token date.")

Expected output:

Processed: 100000
Active tokens: [40434, 40435, 40436, 50120, 50121]
Token info: {'token': 40434, 'found': True, 'symbol': 'NIFTY', 'name': 'NIFTY24MAY21950CE', 'option_type': 'CE', 'strike': 21950, 'expiry': '30-May-2024', 'lot_size': 50}
Snapshot: {'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100)], 'asks': [(2195100, 50)]}
CSV header: local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,...
CSV row: 0,0,2195075,2195050,100,2195100,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

---