themis-llm 2.0.1

THEMIS — Retrieval-grounded LLM for Indian statutory law (BNS, BNSS, BSA, IPC, RTI)

⭐ If THEMIS sparked ideas about fine-tuning LLMs on domain-specific law — a star helps other researchers find it. Takes 2 seconds.

████████╗██╗  ██╗███████╗███╗   ███╗██╗███████╗
╚══██╔══╝██║  ██║██╔════╝████╗ ████║██║██╔════╝
   ██║   ███████║█████╗  ██╔████╔██║██║███████╗
   ██║   ██╔══██║██╔══╝  ██║╚██╔╝██║██║╚════██║
   ██║   ██║  ██║███████╗██║ ╚═╝ ██║██║███████║
   ╚═╝   ╚═╝  ╚═╝╚══════╝╚═╝     ╚═╝╚═╝╚══════╝

THEMIS — Retrieval-Grounded LLM for Indian Statutory Law

"Not retrieval. Not lookup. Baked into weights, grounded by retrieval."

HuggingFace:

• v5: Daniel2503/themis-mistral-7b-lora-v5

---

What is THEMIS?

THEMIS is a domain-specific large language model fine-tuned on Indian statutory law, with retrieval-grounding built in as a first-class feature. It is not a retrieval system, a search engine, or a chatbot wrapper. It is a parametric knowledge model with retrieval-grounding — legal understanding is baked into the model weights through supervised fine-tuning, and answers are grounded by retrieving verified statute text from anchor tables.

Where HECTOR retrieves — THEMIS reasons. But THEMIS also verifies.

---

v5 Results

Training:

• 52,170 training examples (BNS, BNSS, BSA, IPC, RTI, Constitution)
• 1,549 training steps, 2 epochs on Kaggle T4
• Final training loss: 0.1314, validation loss: 0.9808

The overfitting diagnosis → retrieval-grounding fix:

• v5 showed a train/val loss gap indicating overfitting
• Manual testing confirmed the model sometimes fabricates plausible but incorrect legal content
• Fix: a retrieval-grounding layer that looks up actual section text and feeds it as context before generating
• This measurably fixed 2/3 manual test cases

Key insight: Fine-tuning teaches the model how to reason about law. Retrieval-grounding ensures it reasons about correct law, not memorized approximations.

---

Quick Start

Install

pip install themis-llm

Basic Usage

from themis import ThemisModel

# Load model (downloads from HuggingFace on first run)
model = ThemisModel.from_pretrained()

# Ask a grounded question
response = model.ask("What does Section 302 of the BNS say about murder?")

print(response.text)        # The answer
print(response.grounded)    # True — retrieval found a matching section
print(response.section)     # "302"
print(response.act)         # "The Bharatiya Nyaya Sanhita, 2023"
print(response.confidence)  # 0.95

When Retrieval Finds Nothing

response = model.ask("What is the limitation period for filing a suit?")

print(response.grounded)    # False
print(response.warning)     # "No specific legal section found..."
# Answer is from unguided model memory — use with caution

Override Decoding

response = model.ask(
    "Explain Section 63 of BSA.",
    temperature=0.3,
    max_new_tokens=256,
)

Raw Mode (No Grounding)

response = model.ask("What is law?", grounded=False)

---

CLI Usage

Install

pip install themis-llm[cli]

Single-Shot Q&A

themis ask "What does Section 302 of the BNS say about murder?"

Interactive Chat

themis chat

Then type questions interactively. Type exit or quit to leave.

Other Commands

# View model info
themis info

# View version
themis version

# Run evaluation harness
themis eval

# Preprocess datasets
themis preprocess

# Scrape legal data (advanced)
themis scrape --law bns
themis scrape --law bnss
themis scrape --law cpa

# Generate synthetic Q&A pairs (advanced)
themis generate --no-api

CLI Help

themis --help
themis ask --help

---

Architecture

themis/
├── model.py          # ThemisModel — main public API
├── grounding.py      # Retrieval-grounding engine
├── exceptions.py     # Custom exceptions
├── cli.py            # CLI entry point
├── config.py         # Configuration
├── infer.py          # Legacy inference engine
├── data/
│   ├── anchors/      # Ground-truth anchor tables (bns, bnss, bsa, rti, cpa)
│   ├── kaggle/       # Raw training data sources
│   └── ...           # Data pipeline tools
├── eval/             # Evaluation harness
├── training/         # Training config and scripts
└── tests/            # Unit tests

---

Tech Stack

Layer	Technology	Purpose
Base Model	Mistral 7B Instruct v0.3	Foundation — strong instruction following
Fine-tuning	LoRA (r=16, alpha=32)	Parameter-efficient training
Quantization	4-bit NF4	VRAM efficiency (13GB on T4)
Training	Unsloth + TRL	2x faster LoRA training
Grounding	Anchor table retrieval	Section text verification
Platform	Kaggle T4 (free tier)	Training compute

---

Training

v5 Configuration

base_model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
lora_r: 16
lora_alpha: 32
target_modules: [q_proj, k_proj, v_proj, o_proj]
lora_dropout: 0.10
epochs: 2
batch_size: 4
gradient_accumulation: 4
learning_rate: 1e-4
lr_scheduler: cosine
warmup_steps: 100
max_seq_length: 2048
platform: Kaggle T4

Training Notebook

See notebooks/THEMIS_v5_Training.ipynb for the full training pipeline with:

• Checkpoint saving every 100 steps
• Loss monitoring with overfitting/underfitting indicators
• Resume training support
• HuggingFace Hub backup

---

Dataset

Component	Examples	Source
BNS Sections	3,938	358 sections × 11 templates
BSA Sections	1,837	167 sections × 11 templates
BNSS Sections	5,841	531 sections × 11 templates
IPC Sections	4,884	444 sections × 11 templates
GSMS-B QA	6,354	BNS/BNSS/BSA reasoning Q&A
IndicLegalQA	10,002	Supreme Court judgment Q&A
RTI Cases	1,218	CIC decisions
Constitution	870	Constitutional provisions
Total	52,170

---

Evaluation

Metric	v5 Result	Target
Citation accuracy (parseable)	98.6%	>95%
Mismatch rate	1.4%	<2%
Train loss	0.1314	0.3-0.5
Val loss	0.9808	<0.7

---

The Journey

Version	Data	Issue	Fix
v1	1,939	BNS hallucination	20k training pairs
v2	20,909	Overfitting (loss 0.06)	Reduced epochs 3→2
v3	20,909	Still overfitting	Reduced data, added eval
v4	20,909	Mismatch bug (21.6%)	Anchor validation
v5	52,170	Train/val gap (0.13 vs 0.98)	Retrieval-grounding

See THEMIS_finetuning_journey.md for the full post-mortem.

---

Relationship to HECTOR

	THEMIS	HECTOR
Architecture	Parametric fine-tune + retrieval grounding	RAG (Qdrant + Chain-of-Verification)
Knowledge	Model weights + anchor tables	External vector database
Best for	Citizen Q&A	Deep legal research
Citations	Grounded (retrieval-verified)	Source-grounded (verified)
Status	v5 trained	Production-ready

---

License

MIT License

---

Citation

@misc{themis2026,
  author = {Daniel Deshmukh},
  title = {THEMIS: Retrieval-Grounded LLM for Indian Statutory Law},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/Daniel2503/themis-mistral-7b-lora-v5}
}

---

THEMIS — Greek goddess of law, justice, and order. Because justice should not require a law degree to understand.