hft-lob 0.2.6

Python wrapper for a high-performance Rust orderbook CLI

hft-lob

High-performance Python library for reading NSE binary market feed files and reconstructing a full 5-level Limit Order Book (LOB). Powered by a compiled Rust binary — zero Python overhead on the critical path.

---

Features

• Reconstruct LOB from NSE CM binary feed files
• Support for single or multiple instrument tokens
• Three simple CLI commands — no arguments, no paths, no tokens on the command line
• Clean Python Reader API for scripting and backtesting
• 23-field CSV output per tick (timestamps, mid-price, 5-level bid/ask)
• No dependencies — Rust binary is bundled

---

Architecture

hft-lob
├── hft_lob/
│   ├── cli.py            # Reader class + CLI entry point
│   └── bin/
│       └── orderbook-linux-x86_64   # Compiled Rust binary
~/.hft_lob                # User config  (FILE + TOKEN)
~/.hft_lob_cache          # Message cache (built once, reused)
~/.hft_lob_state          # Read cursor   (INDEX + FILE + TOKEN)

Data flow:

NSE .bin feed file
       │
       ▼
 Rust binary (subprocess)          ← runs ONCE, then result is cached
  orderbook-linux-x86_64
       │
       ▼
  ~/.hft_lob_cache                 ← all CSV rows persisted to disk
       │
       ├──── ~/.hft_lob_state      ← tracks current INDEX
       │
    ┌──┴──────────────────────┐
    ▼                         ▼
 hft-lob get_next        hft-lob get_all
 (reads 1 row,           (streams full
  advances INDEX)         cache to stdout)

The Rust binary runs once per unique FILE+TOKEN combination. Subsequent get_next calls read directly from the disk cache — microseconds instead of seconds.

---

Install

pip install hft-lob

---

Configure (once)

Create ~/.hft_lob — this is the only setup you ever need to do:

cat > ~/.hft_lob << 'EOF'
FILE=/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
TOKEN=1333,2885,5900
EOF

Key	Description
FILE	Absolute path to the NSE binary feed file
TOKEN	Instrument token(s). Comma-separated for multiple.

---

CLI Usage

No arguments. No file path. No token. Just run:

hft-lob get_next

Prints the next LOB tick as a single CSV row.

hft-lob get_all

Prints every LOB tick, one CSV row per line (includes header on first line).

hft-lob eof

Prints True if all messages consumed, False if more remain.

hft-lob reset

Resets the read cursor back to the first message (cache is kept).

Pipe examples:

# Count total messages
hft-lob get_all | wc -l

# Preview first 5 rows
hft-lob get_all | head -6

# Save to CSV
hft-lob get_all > lob_data.csv

---

Python API

from hft_lob.cli import Reader

# Single token
r = Reader("/path/to/feed.bin", tokens=1333)

# Multiple tokens — merged into one stream
r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])

Method / Attribute	Returns	Description
r.get_next_message()	str | None	Next CSV row, or None at EOF
r.get_all_messages()	list[str]	All CSV rows as a list
r.is_end_of_file()	bool	True after all messages are consumed
r.header	str	Comma-separated column names

Streaming pattern:

r = Reader("/path/to/feed.bin", tokens=1333)
while not r.is_end_of_file():
    row = r.get_next_message()
    if row:
        print(row)

---

CSV Output Format

23 fields per row:

Field	Description
local_ts	Local timestamp (nanoseconds epoch)
exch_ts	Exchange timestamp (nanoseconds epoch)
mid_price	(best_bid + best_ask) / 2
bid_price_0 – bid_price_4	Bid price at depth levels 0–4
bid_qty_0 – bid_qty_4	Bid quantity at depth levels 0–4
ask_price_0 – ask_price_4	Ask price at depth levels 0–4
ask_qty_0 – ask_qty_4	Ask quantity at depth levels 0–4

---

Load into pandas

import io
import pandas as pd
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
msgs = r.get_all_messages()

df = pd.read_csv(io.StringIO(r.header + "\n" + "\n".join(msgs)))
df["exch_ts"] = pd.to_datetime(df["exch_ts"], unit="ns")
print(df.head())
print(f"Total rows: {len(df)}")

---

Requirements

Item	Requirement
OS	Linux x86_64
Python	3.7+
Dependencies	None

The Rust binary (orderbook-linux-x86_64) is bundled inside the package — no separate install, no Rust toolchain needed.

---

License

MIT