onet-match

Map free text about work to O*NET occupations. Job titles, resume bullets, job postings, project descriptions, interest statements and skill lists go in. Ranked O*NET-SOC occupations come out, each with key tasks, skills, knowledge, RIASEC interests and a calibrated confidence score.

• Open source (MIT) and runs locally. No API key or account needed to match.
• Fine-tuned bi-encoder + cross-encoder reranker over all 923 O*NET 31.0 occupations. Top-1 accuracy is 51% on held-out O*NET text and 65% on messy human-style text, vs 21% and 17% for a TF-IDF baseline.
• Python API, CLI, REST API (FastAPI) and Docker. About 47 ms per query on a laptop.
• Also includes the O*NET data pipeline it's built on: bulk database download, a Web Services client and analysis tools.

Real outputs of the served model, saved in outputs/examples/readme_examples.json.

Quickstart

pip install "onet-match[matching] @ git+https://github.com/Benjamin-V-Chan/onet-match"

from onet_match import match

result = match("ran the robotics club's CAD team", top_k=5)
for m in result.matches:
    print(m.soc, m.title, m.confidence)
print(result.profile["riasec_code"])      # aggregate interest code, e.g. "RIC"

The package imports as onet_match and provides the onet-match command. The first call downloads the pretrained model bundle (about 0.5 GB) from the Hugging Face Hub (BenjaminChan12/onet-match) and caches it.

onet-match match "Excel, QuickBooks, payroll, AP/AR"          # CLI (add --json for JSON)
pip install "onet-match[matching,serve] @ git+https://github.com/Benjamin-V-Chan/onet-match"
onet-match serve --port 8000                                   # REST API
curl -s localhost:8000/match -H 'content-type: application/json' \
     -d '{"text": "ran the robotics club'\''s CAD team", "top_k": 3}'

REST routes: POST /match, POST /match/batch, GET /occupation/{soc} and GET /health. The full request and response contract and the Docker setup are in docs/API.md.

What each match contains

soc, title, confidence (calibrated, 0–1), score, major_group and its title, job_zone, riasec_code, description, top_tasks, top_skills and top_knowledge (with O*NET importance and level). Each result also carries a profile that aggregates skills, knowledge and all six RIASEC interests across the matches, weighted by confidence.

Use cases

• Career exploration (EdTech). Map what students say they did or enjoy to occupations and skills.
• Resume and profile parsing. Code experience bullets and skill lists to SOC codes.
• Job-posting normalization. Turn informal titles and descriptions into standard occupations.
• Workforce and labor-market analytics. Batch-code large text collections.
• Career counseling tools. Show the tasks and skills behind a suggestion.
• LLM agents. Ground occupation questions in O*NET instead of free generation (see llms.txt).

Details and integration patterns: docs/APPLICATIONS.md.

Accuracy

All models are evaluated on the same held-out splits. Every number traces to a saved run in outputs/runs/.

• O*NET test: 6,792 held-out titles, task statements, work activities and descriptions.
• Messy-text test: 4,703 informal snippets written by a different LLM than the one that wrote the training text.
• Confidence: temperature-calibrated, ECE 0.037.
• Full results: per-input-type tables, calibration, ablations, learning curves and error analysis are in docs/RESULTS.md.

How it works

flowchart LR
    B["O*NET 31.0 database<br/>(bulk download)"] --> C["Occupation catalog<br/>923 occupations"]
    C --> P["Labeled pairs<br/>titles · tasks · activities"]
    C --> S["LLM-written messy text<br/>train: model A · test: model B"]
    P --> T["Fine-tune bi-encoder<br/>+ cross-encoder reranker"]
    S --> T
    T --> BU["Model bundle<br/>(Hugging Face Hub)"]
    BU --> API["match() · onet-match CLI · REST API"]

1. A fine-tuned bi-encoder (bge-base) embeds the input and retrieves the 20 nearest occupations.
2. A fine-tuned cross-encoder rescores those 20.
3. The two scores are blended with a weight chosen on validation data.
4. A temperature-scaled softmax turns the blended scores into calibrated confidence.

The validation and test tasks are removed from the occupation documents the models search, so no test text leaks into retrieval. Synthetic training and test text come from different LLMs. Details are in docs/SYNTHETIC_DATA.md.

O*NET data pipeline

The onet_data_collector package (onet-collect) downloads and verifies O*NET database releases, which needs no credentials. It also wraps O*NET Web Services: keyword search, occupation details and flattened profiles. See docs/COLLECTOR.md.

onet-collect bulk download                         # download + verify the latest O*NET database
onet-match collect "healthcare" "technology"       # Web Services search (free O*NET account, see .env.example)
onet-match eda | keywords | cluster                # classic analysis: coverage, keyword attribution, TF-IDF clusters

Train your own

git clone https://github.com/Benjamin-V-Chan/onet-match && cd onet-match
pip install -e ".[dev]"
onet-match matching build-data          # O*NET catalog, labeled pairs, splits
scripts/matching/reproduce.sh           # every model, ablation and the results report
onet-match matching bundle "<spec>" --run <run>   # package a model for serving

• Synthetic text: WITH_SYNTH=1 TRAIN_LLM=<model> EVAL_PROVIDER=<provider> EVAL_LLM=<model> scripts/matching/reproduce.sh also generates the messy training and test text, using a different LLM for each. It needs LLM API keys; see docs/SYNTHETIC_DATA.md.
• Training time: bge-base took 58 minutes for one epoch on a laptop GPU (Apple Silicon, MPS). The exact scripts behind every saved run are in scripts/matching/.
• Tests: pytest runs fully offline, using tiny local models and a mocked LLM.

Set ONET_MATCH_BUNDLE=/path/to/bundle to serve your own model, or ONET_MATCH_HF_REPO=owner/repo to load one from the Hub.

Background

This project was originally built in 2024 as an open-source edtech tool for analyzing O*NET occupational data at scale, and as part of my capstone project for an engineering course. Its analytics pipeline (EDA, occupation clustering, and keyword attribution) was later integrated into production products at Project Leo, and has inspired related work at other companies and projects.

In 2026 it was rebuilt around semantic occupation matching, and the former ONET-Data-Collector repository was merged in.

Citation

@software{chan_onet_match,
  author = {Chan, Benjamin},
  title  = {onet-match: mapping free text to O*NET occupations},
  url    = {https://github.com/Benjamin-V-Chan/onet-match},
  year   = {2026}
}

License and data attribution

The code is MIT-licensed; see LICENSE.

This product includes information from the O*NET 31.0 Database by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA), used under the CC BY 4.0 license. O*NET® is a trademark of USDOL/ETA. USDOL/ETA has not approved, endorsed or tested this product.