search-expert 0.1.2

Parse natural language search queries into structured fields using fine-tuned Qwen3.5-0.8B LoRA adapters.

 search-expert

Natural language → structured search queries, instantly.
Fine-tuned Qwen3.5-0.8B LoRA adapters for search query parsing across 10 domains.

---

What it does

"Non-stop business class from JFK to Tokyo under $3,000"

{
  "domain":      "flights",
  "origin":      "JFK",
  "destination": "Tokyo",
  "cabin_class": "business",
  "stops":       "lte:0",
  "price":       "lt:3000"
}

Search Expert uses a custom fine-tuned Small Language Model to understand natural language queries and extract only the fields explicitly mentioned — never hallucinating values that aren't there. It works across 10 search verticals out of the box.

Install

pip install search-expert

Usage

Basic

from search_expert import SearchExpert, ModelFormat, ParseResult

expert = SearchExpert()  # loads the JSON adapter by default

result = expert.parse("noise cancelling headphones any colour but red or green, under $200")
print(result.fields)

{
    'domain':  'ecommerce',
    'product': 'headphones',
    'feature': 'noise cancelling',
    'color':   ['ne:red', 'ne:green'],
    'price':   'lt:200'
}

Why hybrid search beats the alternatives

	Text-to-SQL	Pure vector search	Hybrid search (this pipeline)
Hard constraints (price, brand, color)	✅	❌	✅
Semantic intent ("good for travel")	❌	✅	✅
Ranked results by relevance	❌	✅	✅
Works on unstructured descriptions	❌	✅	✅
Respects exclusions ("not black")	✅	❌	✅
Price is a hard cutoff, not a soft signal	✅	❌	✅

Text-to-SQL is a lookup tool — it returns rows that match, but can't rank by relevance or understand semantics.
Pure vector search is a semantic tool — it understands meaning, but treats "$200" as a soft hint, not a hard rule. A $350 product can rank above a $180 one if its description is more similar to the query.
This pipeline is a retrieval tool — structured filters enforce the hard constraints first, then vector search ranks the surviving candidates by semantic relevance.

In production, we generally use a version of this pattern:
structured pre-filtering → ANN (approximate nearest neighbour) vector search → learning-to-rank re-ranker.

search-expert makes step 1 trivial with a tiny, fast, locally-runnable model.

Operator reference

All numeric and exclusion constraints use a consistent prefix so downstream filters need zero NLP — just parse the string.

Query phrase	Output value
"under $200", "below $200"	lt:200
"up to $200", "max $200"	lte:200
"over $150k", "above $150k"	gt:150000
"at least $150k", "$150k+"	gte:150000
"around $200", "~$200"	approx:200
"$100–$200", "between $100 and $200"	between:100:200
"any colour but red or green"	["ne:red", "ne:green"]

Applying a filter in one line:

result = expert.parse("apartments under $2,500/month in Austin")
salary = result.get_numeric_constraint("price")
# {'operator': 'lt', 'value': 2500.0, 'value_hi': None}

filtered = [l for l in listings if l["price"] < salary["value"]]

Domains present in the training data

Domain	Example query
real_estate	"2BR apartment in Austin under $1,500/month"
ecommerce	"Sony noise cancelling headphones under $300"
jobs	"Remote senior ML engineer paying over $150k"
flights	"Non-stop business class JFK to Tokyo under $3,000"
hotels	"5-star hotel in Paris with breakfast under $400/night"
cars	"Electric SUV with 300+ mile range under $50k"
restaurants	"Vegan Italian in NYC with outdoor seating under $40"
movies	"Thriller on Netflix with 8+ IMDB rating"
healthcare	"Female therapist in Chicago accepting Aetna"
courses	"Python ML course for beginners under $30"
events	"Taylor Swift concert in London in July"

The model

Each adapter is a LoRA fine-tune of Qwen3.5-0.8B trained on 100,000 (query, structured output) pairs spanning all 10 domains above.

Adapter	HuggingFace	Format
JSON (default)	sarthakrastogi/search-expert-json-0.8b	JSON
YAML	sarthakrastogi/search-expert-yaml-0.8b	YAML

Format leaderboard (held-out test set, 300 samples per format):

Rank	Format	Key F1	Value Acc	Parse Rate
🥇	JSON	0.913	0.874	98.2%
🥈	YAML	0.901	0.861	97.6%
🥉	TOML	0.887	0.843	96.1%
4.	XML	0.871	0.829	94.8%
5.	CSV key=value	0.856	0.812	93.3%

Both public adapters return the same Python dict — the format only affects the model's internal generation language.

Benchmarking against Amazon's search

I evaluated search-expert's hybrid pipeline against Amazon's native search and pure vector search on 25 complex, multi-constraint ecommerce queries (e.g. "Sony noise cancelling headphones, not black, under $250").

Each pipeline's top-6 results were scored on whether they actually satisfied the hard constraints in the query — price limits, color exclusions, and color requirements. No relevance proxies; just "did the result follow the rules?"

Metric	Amazon	Pure Vector	Hybrid (search-expert)
Price satisfaction	0.95	0.76	1.00
Color exclusion	0.66	0.78	1.00
Color match	0.62	0.47	1.00
Overall (all constraints)	0.76	0.56	1.00
Perfect@6 (all 6 correct)	0.42	0.21	1.00

Perfect@6 is the strictest metric: 1.0 only if every result in the top 6 satisfies every constraint simultaneously.

The hybrid pipeline scores 1.0 across the board. Amazon's search respects price constraints reasonably well but frequently returns wrong-colored results (only 62% color match on average). Pure vector search is the weakest on color — it understands what kind of thing you want, but treats "not black" as a soft suggestion rather than a hard rule.

Full benchmark code and dataset: benchmarks/amazon/ · Dataset on HuggingFace

Comparison: Why not just use spaCy/NER instead?

Capability	Description	spaCy / NER	search-expert
Named entity extraction	Pulls out cities, brands, product names, job titles from free text	✅ good	✅
Numeric value extraction	Detects numbers like $200, 5 stars, 2 bedrooms	✅ good	✅
Comparison operator inference	Maps phrases like "under", "at least", "starting from" to lt: / gte: operators	⚠️ partial, brittle rules	✅
Range constraints	Understands "between $100 and $300" or "$100–$300" as between:100:300	⚠️ regex only	✅
Approximation operator	Maps "around $200", "roughly $200", "~$200" to approx:200	❌	✅
Exclusion constraints	Maps "any colour but red or green" to ["ne:red", "ne:green"]	❌ needs custom logic	✅
Multi-domain field assignment	Correctly assigns "noise cancelling" to feature vs. a brand or city depending on domain	❌ explodes in rules	✅
Novel phrasing generalisation	Handles phrasings not seen during development without rule updates	❌ misses anything not in rules	✅ generalises
Inference speed	Time to parse one query	✅ ~1 ms (CPU)	⚠️ ~400 ms (GPU) / ~3 s (CPU)
GPU requirement	Whether a GPU is needed for production-speed inference	✅ none	⚠️ optional
Maintenance cost	Effort required to extend coverage to new domains, fields, or phrasings	❌ high — rules accumulate and conflict	✅ low — add training data

Repo structure

search-expert/
├── search_expert/        # Library source
│   ├── __init__.py
│   ├── expert.py         # SearchExpert class (main API)
│   ├── config.py         # Model IDs, prompts, format enum
│   ├── loader.py         # HF model loading (unsloth / peft / plain)
│   ├── parser.py         # Raw output → dict parsers
│   ├── result.py         # ParseResult dataclass
│   └── exceptions.py     # Custom exceptions
├── training/             # Fine-tuning pipeline
│   ├── finetune.py       # Training script
│   └── evaluate.py       # Format comparison leaderboard
├── tests/
│   └── test_search_expert.py
├── examples/
│   └── search_expert_colab.ipynb
├── pyproject.toml
└── README.md

Development

git clone https://github.com/sarthakrastogi/search-expert
cd search-expert
pip install -e ".[dev]"

pytest tests/ -v                                          # unit tests (no GPU needed)
SEARCH_EXPERT_RUN_MODEL_TESTS=1 pytest tests/ -v          # includes model inference tests

License

MIT © Sarthak Rastogi

Contributing

Contributions are very welcome! Please open an issue or submit a pull request with any improvements.

Contact

For questions, feedback, or just to say hi, you can reach me at:

Email

LinkedIn