hft-lob 0.2.9

Python wrapper for a high-performance Rust orderbook CLI

hft-lob

High-performance Python library for reading NSE binary market feed files and reconstructing a full 5-level Limit Order Book (LOB). Powered by a compiled Rust binary — zero Python overhead on the critical path.

---

Features

• Reconstruct LOB from NSE CM binary feed files
• Support for single or multiple instrument tokens
• CLI commands — no arguments, no paths, no tokens on the command line
• get_bid / get_ask commands — instantly see bid/ask levels of the current row
• LOBRow class — named field access in Python (row.bid(0), row.all_asks())
• Cache system — Rust binary runs once, every subsequent call reads from disk instantly
• 23-field CSV output per tick (timestamps, mid-price, 5-level bid/ask)
• No dependencies — Rust binary is bundled

---

Architecture

hft-lob
├── hft_lob/
│   ├── cli.py            # LOBRow class + Reader class + CLI entry point
│   └── bin/
│       └── orderbook-linux-x86_64   # Compiled Rust binary
~/.hft_lob                # User config  (FILE + TOKEN)
~/.hft_lob_cache          # Message cache (built once, reused)
~/.hft_lob_state          # Read cursor   (INDEX + FILE + TOKEN)

Data flow:

NSE .bin feed file
       │
       ▼
 Rust binary (subprocess)          ← runs ONCE, then result is cached
  orderbook-linux-x86_64
       │
       ▼
  ~/.hft_lob_cache                 ← all CSV rows persisted to disk
       │
       ├──── ~/.hft_lob_state      ← tracks current INDEX
       │
    ┌──┴───────────────────────────────────────┐
    ▼                                          ▼
hft-lob get_next                   hft-lob get_all
(reads 1 row, advances INDEX)      (streams full cache to stdout)
    │
    ├── hft-lob get_bid [level]    ← bid price+qty of current row
    └── hft-lob get_ask [level]    ← ask price+qty of current row

The Rust binary runs once per unique FILE+TOKEN combination. All subsequent calls read directly from ~/.hft_lob_cache — microseconds instead of seconds.

---

Install

pip install hft-lob

---

Configure (once)

Create ~/.hft_lob — the only setup you ever need:

printf 'FILE=/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin\nTOKEN=1333,2885,5900\n' > ~/.hft_lob

Key	Description
FILE	Absolute path to the NSE binary feed file
TOKEN	Instrument token(s). Comma-separated for multiple.

---

CLI Usage

Navigate rows

hft-lob get_next

Advances to the next row and prints all 23 fields with names:

--- row 0 ---
  local_ts: 1766979473076306016
  exch_ts: 1451466473069645718
  mid_price: 0
  bid_price_0: 99310
  bid_qty_0: 13456
  ask_price_0: 0
  ask_qty_0: 0
  ...

Inspect bid levels

hft-lob get_bid          # all 5 bid levels of current row

bid levels:
  [0]  price=99310  qty=13456
  [1]  price=99300  qty=5200
  [2]  price=99290  qty=8100
  [3]  price=0      qty=0
  [4]  price=0      qty=0

hft-lob get_bid 0        # best bid only

bid level 0:  price=99310  qty=13456

Inspect ask levels

hft-lob get_ask          # all 5 ask levels
hft-lob get_ask 0        # best ask only
hft-lob get_ask 2        # ask level 2 only

Other commands

hft-lob get_all          # print every row as raw CSV (for piping/saving)
hft-lob eof              # True if all rows consumed, False otherwise
hft-lob reset            # go back to row 0 (cache is kept)

Pipe examples:

hft-lob get_all | wc -l          # count total rows
hft-lob get_all | head -6        # preview first 5 rows
hft-lob get_all > lob_data.csv   # save to file

---

Python API

LOBRow — named field access

from hft_lob.cli import LOBRow

row = LOBRow("1766979473076306016,1451466473069645718,0,99310,13456,...")

# Best bid and ask
price, qty = row.bid(0)     # bid level 0
price, qty = row.ask(0)     # ask level 0

# All 5 levels
row.all_bids()   # [(price, qty), (price, qty), ...]
row.all_asks()   # [(price, qty), (price, qty), ...]

# Direct field access
row.mid_price
row.bid_price_0
row.ask_qty_2
row.local_ts
row.exch_ts

# As dict
row.to_dict()    # {'local_ts': '...', 'bid_price_0': '...', ...}

Reader — bulk processing

from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])

r.get_next_message()    # returns one CSV string, or None
r.get_all_messages()    # returns list of all CSV strings
r.is_end_of_file()      # returns True / False
r.header                # CSV column names string

---

CSV Output Format

23 fields per row:

Field	Description
local_ts	Local timestamp (nanoseconds epoch)
exch_ts	Exchange timestamp (nanoseconds epoch)
mid_price	(best_bid + best_ask) / 2
bid_price_0 – bid_price_4	Bid price at depth levels 0–4
bid_qty_0 – bid_qty_4	Bid quantity at depth levels 0–4
ask_price_0 – ask_price_4	Ask price at depth levels 0–4
ask_qty_0 – ask_qty_4	Ask quantity at depth levels 0–4

---

Load into pandas

import io
import pandas as pd
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
msgs = r.get_all_messages()

df = pd.read_csv(io.StringIO(r.header + "\n" + "\n".join(msgs)))
df["exch_ts"] = pd.to_datetime(df["exch_ts"], unit="ns")
print(df.head())
print(f"Total rows: {len(df)}")

---

Requirements

Item	Requirement
OS	Linux x86_64
Python	3.7+
Dependencies	None

The Rust binary (orderbook-linux-x86_64) is bundled — no separate install, no Rust toolchain needed.

---

License

MIT

---

Features

• Reconstruct LOB from NSE CM binary feed files
• Support for single or multiple instrument tokens
• Three simple CLI commands — no arguments, no paths, no tokens on the command line
• Clean Python Reader API for scripting and backtesting
• 23-field CSV output per tick (timestamps, mid-price, 5-level bid/ask)
• No dependencies — Rust binary is bundled

---

Architecture

hft-lob
├── hft_lob/
│   ├── cli.py            # Reader class + CLI entry point
│   └── bin/
│       └── orderbook-linux-x86_64   # Compiled Rust binary
~/.hft_lob                # User config  (FILE + TOKEN)
~/.hft_lob_cache          # Message cache (built once, reused)
~/.hft_lob_state          # Read cursor   (INDEX + FILE + TOKEN)

Data flow:

NSE .bin feed file
       │
       ▼
 Rust binary (subprocess)          ← runs ONCE, then result is cached
  orderbook-linux-x86_64
       │
       ▼
  ~/.hft_lob_cache                 ← all CSV rows persisted to disk
       │
       ├──── ~/.hft_lob_state      ← tracks current INDEX
       │
    ┌──┴──────────────────────┐
    ▼                         ▼
 hft-lob get_next        hft-lob get_all
 (reads 1 row,           (streams full
  advances INDEX)         cache to stdout)

The Rust binary runs once per unique FILE+TOKEN combination. Subsequent get_next calls read directly from the disk cache — microseconds instead of seconds.

---

Install

pip install hft-lob

---

Configure (once)

Create ~/.hft_lob — this is the only setup you ever need to do:

cat > ~/.hft_lob << 'EOF'
FILE=/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
TOKEN=1333,2885,5900
EOF

Key	Description
FILE	Absolute path to the NSE binary feed file
TOKEN	Instrument token(s). Comma-separated for multiple.

---

CLI Usage

No arguments. No file path. No token. Just run:

hft-lob get_next

Prints the next LOB tick as a single CSV row.

hft-lob get_all

Prints every LOB tick, one CSV row per line (includes header on first line).

hft-lob eof

Prints True if all messages consumed, False if more remain.

hft-lob reset

Resets the read cursor back to the first message (cache is kept).

Pipe examples:

# Count total messages
hft-lob get_all | wc -l

# Preview first 5 rows
hft-lob get_all | head -6

# Save to CSV
hft-lob get_all > lob_data.csv

---

Python API

from hft_lob.cli import Reader

# Single token
r = Reader("/path/to/feed.bin", tokens=1333)

# Multiple tokens — merged into one stream
r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])

Method / Attribute	Returns	Description
r.get_next_message()	str | None	Next CSV row, or None at EOF
r.get_all_messages()	list[str]	All CSV rows as a list
r.is_end_of_file()	bool	True after all messages are consumed
r.header	str	Comma-separated column names

Streaming pattern:

r = Reader("/path/to/feed.bin", tokens=1333)
while not r.is_end_of_file():
    row = r.get_next_message()
    if row:
        print(row)

---

CSV Output Format

23 fields per row:

Field	Description
local_ts	Local timestamp (nanoseconds epoch)
exch_ts	Exchange timestamp (nanoseconds epoch)
mid_price	(best_bid + best_ask) / 2
bid_price_0 – bid_price_4	Bid price at depth levels 0–4
bid_qty_0 – bid_qty_4	Bid quantity at depth levels 0–4
ask_price_0 – ask_price_4	Ask price at depth levels 0–4
ask_qty_0 – ask_qty_4	Ask quantity at depth levels 0–4

---

Load into pandas

import io
import pandas as pd
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
msgs = r.get_all_messages()

df = pd.read_csv(io.StringIO(r.header + "\n" + "\n".join(msgs)))
df["exch_ts"] = pd.to_datetime(df["exch_ts"], unit="ns")
print(df.head())
print(f"Total rows: {len(df)}")

---

Requirements

Item	Requirement
OS	Linux x86_64
Python	3.7+
Dependencies	None

The Rust binary (orderbook-linux-x86_64) is bundled inside the package — no separate install, no Rust toolchain needed.

---

License

MIT