llm4smeft 1.1.1

LLM-driven interactive hypothesis generation for SMEFT global fits

LLM4SMEFT

LLM-driven interactive hypothesis generation for SMEFT global fits

---

What is this?

llm4smeft turns a fine-tuned LLM into a collaborator for Standard Model Effective Field Theory (SMEFT) global-fit interpretation. Point it at a handful of observables — m_W, Gamma_Z, Higgs signal strengths, whatever — and it will:

1. Retrieve the relevant stored SMEFiT fit summary (or ask the model to normalize your observable names and retry, or build a fresh case from scratch if nothing matches);
2. Ask the model to propose a small set of sparse Wilson-coefficient hypotheses, each capped at a chosen number of active operators;
3. Show you why — the chain-of-thought, the Fisher information driving each operator, and the PCA directions behind the dominant tensions;
4. Let you push back in plain English ("drop OpWB, try the leptonic direction instead") and watch it revise, turn after turn;
5. Save the accepted session back into the data store as a new record, so the next retrieval has more to work with.

It's built for the loop a physicist actually works in: propose, inspect, argue, refine — not a single black-box inference call.

---

Table of contents

• Install
• Quickstart
• How it works
• Interactive commands
• Configuration
• The data store
• Project layout
• Testing
• Known limitations
• License

---

Install

pip install llm4smeft

That's it — the 29 bundled SMEFiT summary records (llm4smeft/data/*.json) and the prompt template ship inside the package, resolved relative to the installed files rather than your working directory. Run llm4smeft from anywhere; no repo checkout, no cd, no copying folders around.

GPU users: torch is a plain PyPI dependency, so the default install pulls a CPU-only wheel. For CUDA, install the matching PyTorch build from pytorch.org before pip install llm4smeft, or reinstall torch afterward with the right --index-url.

---

Quickstart

llm4smeft

--- Case setup ------------------------------------
  Observables (comma-separated) Example - m_W, Gamma_Z, Higgs_signal_strength: m_W, Gamma_Z
  Number of hypotheses Example - 3: 3
  Sparsity cap (max operators per hypothesis) Example - 3: 3

  Case retrieved from stored summary: EWPO
  Matched observable groups: EWPO
  Fit-selected operators: Ope, Opta, Ole1111, OpWB, Opl1, O3pl1, O3pl2, Oll1221, O3pq, O3pQ3

-- Turn 1 — generating -------------------------------

  Hypotheses:

  1. [OpWB, O3pl2, Oll1221]  tag=leptonic_universality_break
     Chain of tensions in the Z-pole asymmetries points to a shared direction...

  Your feedback (or 'done' / 'quit'):

Type feedback to steer the next turn, show to redisplay, history for the full transcript, or done to accept and persist the session.

Trying it without a GPU or model download

LLM4SMEFT_PROPOSER_MODE=mock llm4smeft

Mock mode swaps the LLM for a deterministic stand-in so you can exercise the full retrieval → hypothesis → evaluation → save-back loop offline. This is exactly what the test suite runs in CI.

---

How it works

observables ──▶ retrieval (regex/alias match against stored SMEFiT fits)
                     │
                     ├─ hit ──────────────────────────────▶ BenchmarkCase
                     │
                     └─ miss ─▶ LLM normalizes names ─▶ retry retrieval
                                      

BenchmarkCase ──▶ LLMProposer ──▶ sentinel-formatted response
                                     <<<REASONING>>>   step-by-step, bullet form
                                     <<<HYPOTHESES>>>  strict JSON, sparse operator sets
                                     <<<END_HYPOTHESES>>>
                                        │
                     ┌──────────────────┼───────────────────┐
                     ▼                  ▼                    ▼
              toy chi²/AIC/BIC   Fisher-information    reconciliation against
                 scoring          + PCA context           reasoning text
                     │                  │                    │
                     └──────────────────┴────────────────────┘
                                        ▼
                          you review, give feedback, repeat
                                        │
                                     'done'
                                        ▼
                        session saved back into the data store

A few things worth knowing about the internals:

• Retrieval first, generation second. If your observables match a stored SMEFiT summary, the pipeline reuses its Fisher information and PCA decomposition rather than asking the model to invent one — grounding proposals in an actual fit rather than free-associated physics.
• Strict output contract. The model is prompted to answer inside four sentinel markers (<<<REASONING>>> / <<<HYPOTHESES>>> / their END_ counterparts); the parser is tolerant of malformed JSON and can rescue hypotheses straight out of unstructured reasoning text if the strict format slips.
• Reasoning/answer reconciliation. If the model commits to an operator set in its chain-of-thought but drops it from the final JSON, the pipeline notices and restores it (tagged reasoning_alignment) rather than silently losing it.
• Toy scoring, not a real fit. The chi²/AIC/BIC shown per hypothesis are a lightweight overlap heuristic against the retrieved case's known support — useful for ranking proposals during a session, not a substitute for an actual SMEFiT likelihood evaluation.

---

Interactive commands

Command	Effect
(any other text)	Sent as feedback; the model revises on the next turn
show	Redisplay the current hypotheses + Fisher/PCA context
history	Print the full turn-by-turn conversation so far
done / accept / ok / yes	Accept the session and write it back to the data store
quit / exit / q	Exit without saving

---

Configuration

All configuration is via environment variables — no config file needed.

Variable	Default	Purpose
LLM4SMEFT_PROPOSER_MODE	local	local runs the real HF model; mock uses a deterministic offline stand-in
LLM4SMEFT_HF_MODEL	ahammad115566/smeft-qwen-7b	Hugging Face model id used for hypothesis proposals
LLM4SMEFT_CASE_MODEL	same as above	Optional separate model for LLM-generated (non-retrieved) cases
LLM4SMEFT_DEVICE_MAP	cuda	Passed to transformers as the device map ("auto" for multi-GPU)
LLM4SMEFT_LOAD_IN_4BIT	0	Set to 1 to load the model 4-bit quantized
LLM4SMEFT_FLATTEN_HISTORY	1	Flatten multi-turn history into a single user message per call
LLM4SMEFT_DATA_DIR	(package data/ dir)	Override where summaries are read from / written to
LLM4SMEFT_DEBUG	0	Set to 1 for full tracebacks on proposer errors

---

The data store

The package ships 29 pre-computed SMEFiT fit summaries spanning four sectors:

Sector	Records
Electroweak precision	EWPO, EW_VV, EW_all, LEP_WW, LHC_VV, VV_all, W_helicity
Higgs	higgs_SS, higgs_STXS, higgs_hh, higgs_VV, higgs_EW, higgs_all
Top quark	tt_incl, ttZ, ttW, tta, ttbb, tttt, tW, tZ, single_top, top_4heavy, top_EW, top_all, top_higgs, ttV, singletop_all
Cross-sector	global

Each record carries fit_metadata (fitted operators, priors, dataset groups), fit_results (95% CL bounds, per-dataset χ² residuals), correlations, pca (principal components with singular values), and fisher_information (which dataset group dominates the sensitivity to each operator). Retrieval matches your observable strings to a record by regex/alias, preferring the tightest match with the least "extra" scope.

Every done-accepted session is folded back into this store as a new smefit-<sector>-<timestamp>.json record, so repeated sessions on the same observables accumulate prior context (_previous_session_notes) rather than starting cold each time.

---

Project layout

llm4smeft/
├── benchmarks/     BenchmarkCase — the ground-truth container passed to the model
├── chat/           ChatSession (turn tracking) + ChatHistoryStore (JSON persistence)
├── data/           bundled SMEFiT summary records (package data)
├── evaluator/      toy chi²/AIC/BIC scoring against known support
├── experiments/    run_interactive.py — the CLI entry point
├── hypotheses/      Hypothesis schema
├── prompts/        base_prompt.txt — the sentinel-format system prompt template
├── proposers/      LLMProposer, prompt rendering, response parsing
└── summaries/      Summary schema + retrieval/matching logic

---

Testing

git clone <your fork>
cd llm4smeft
pip install -e .
python tests/test_pipeline.py

The suite runs entirely offline (LLM4SMEFT_PROPOSER_MODE=mock for the end-to-end check) and covers observable→summary matching, Fisher-ranked operator menus, sentinel-format parsing, reasoning rescue/reconciliation, and the full accept-and-save round trip — 50+ checks, no GPU required.

---

License

MIT