hugpy_core
hugpy central, minimal. Workers (hugpy_worker, one per box, over fitevict)
register and heartbeat here; the registry is live in memory. /v1/chat/completions
picks an online worker that has the model (resident first; alloc.worker pins one)
and relays to its /infer/stream. No evict-to-fit, discovery, admission or DB
state — each box's fitevict owns placement and the call log.
hugpy-core --host 127.0.0.1 --port 7020
Benchmark call-log history
POST /llm/benchmark/run opens a core call-log row before validation or planning.
Accepted runs keep that row pending until their asynchronous outcome is known.
Rejected plans retain their constraint reasons in the core response and error.
The response and X-Hugpy-Request-Id expose its request ID.
Benchmark history/status read call_log; the JSON state and run archives are no
longer read or written. Existing files are left intact as legacy evidence and
are not converted into invented historical call records. Grading entries store
call pointers and verdicts. History resolves answers and measurements from the
referenced execution records. Each benchmark inference core row points back to
the orchestration row, and its execution pointer identifies the wrapper row.
Pending benchmark rows from a previous core process are marked interrupted.
The brain (failure auto-evaluation)
Ruling: toolserver instructions/hugpy/brain/brain-established-2026-10-06.
A brain lives in core: always present, callable, and it evaluates every
model-load failure automatically. hugpy-agent, when installed, adds features and
access on top (the agent-station TUI); without it, this brain does the work.
- Event: a
hugpy_wrapper.residentsrow whoseload_errorchanges firespg_notify('model_rows');brain.ensure()(started byapp.main) listens and evaluates it. No polling. - Classify: rules first, from the legacy taxonomy (
hard_load_failure,vram_fit,engine_unavailable,unreachable,vision_needs_slot,missing_dependency,other), plus pip-shaped signals (pip install X,No module named 'x', transformers' "requires the following packages"). - Infer: the brain model is asked only when the rules leave something open
(class
other, or an import name with no package name, e.g.cv2→opencv-python). The call goes through core's own/v1/chat/completionswithmetadata.intent = "brain-evaluate", so it is acall_logrow. The brain never overrules a class the rules determined. - Record: every evaluation is a Help ticket (
help_tickets, kindload-failure), de-duplicated per model/worker/error viaoccurrences. - Call it:
POST /llm/brain/evaluate {error, worker?, model_key?, file?};GET /llm/brainshows the model, policy and listener state.
Which model is the brain. The static hugpy-brain model group (see
Model groups below) holds the models the brain may run on, in priority
order. When the group is empty it fills itself from the grades: every model
whose latest verdicts pass the threshold in each listed grader category
(default coding 3/3 + logic 3/3 of the hugpy-native-v2 suite's
easy/medium/hard tiers), best first. Resolution: the Settings selection
(brain.model) → HUGPY_BRAIN_MODEL → the group's first servable member →
the best-graded servable model (flagged below threshold). The Help chat uses
the same brain (after HUGPY_HELP_MODEL).
Auto-implement gate. Every evaluation carries auto: {eligible, why}.
A fix may be implemented without approval only when brain.auto_fix is
ticked and it is a pip fix with named packages; a fix read literally off the
error needs nothing more, a fix a model inferred also needs that model to meet
the threshold. Otherwise it waits for operator approval.
Settings → Policies → Brain (settings_policies):
| key | meaning |
|---|---|
brain.model |
the brain's model; unset = automatic (first servable hugpy-brain member) |
brain.auto |
auto-evaluate load failures (default on) |
brain.auto_fix |
the auto-implement tick box (default on) |
brain.threshold |
{grader category: tiers that must pass}, default {"coding": 3, "logic": 3} |
Not automated yet (same ruling, next step): the venv ladder for
missing_dependency — existing env profiles → a new overlay profile → an
isolated profile — each candidate test-loaded, then designated
(model_workers.env_profile); auto decides whether that happens on its own.
Never pip-install into a shared venv.
Model groups (Priority groups panel)
model_groups.py + sql/019_model_groups.sql: the panel's groups are rows in
hugpy_core.model_groups (record shape lifted from legacy
comms/priority_groups.py: ordered members, ordered workers, enabled),
plus static (cannot be deleted or renamed), role and source
(operator | auto). GET/POST /llm/model-groups, PUT/PATCH/DELETE /llm/model-groups/<id>. One structural rule, as before: a model key may be in
at most one enabled routing group. Role groups (hugpy-brain, role brain)
never route, may be empty, and sit outside that rule. Core's request routing
does not walk priority groups yet; the list payload says so (routing: false)
and resolve answers that honestly. allocate returns 501 (not lifted).
Verdicts priced at the pair's ctx and KV type
planner.refresh_worker passes each pair's user_settings to
measured_verdict.price: the verdict is priced at the pair's ctx_pct
(k% of the trained window, floored to 1024) and kv_cache_type
(f16 / q8_0 / q4_0). A knob write that touches either re-plans that worker
at once (modeldb.pair_knobs → planner.replan), so the console's Memory
distribution follows the knobs.
The fix ladder (missing dependencies) — fix_ladder.py
Operator design: try every environment on the box that may already fit, else
the box's worker base + the new pip install, else keep that new (broken)
install attributed and leave the ticket for the operator. Candidates, best
first: BUILT profiles whose frozen lock holds every needed package, then
DECLARED-but-unbuilt ones whose manifest holds them (the box builds on the test
load). Each candidate is attributed (model_workers.env_profile) and
test-loaded (envprofiles.run_test, an env-test call, 10 min cap); a failure
is reverted, and its log tail (call_log.response_meta.load_log_tail) is read
for the next missing package (transformers' "requires the X library" wording
included). The new environment is auto-<model> (worker base + packages, up to
3 rounds). Starts on its own when the evaluation is auto-eligible, else on ✓
approve; a core restart resumes it once its worker re-registers. Progress lives
on the ticket (detail.ladder). First real fix: Qwen3.5-0.8B on computron →
its vision profile (pillow + the box-pinned torchvision).
Allocation determinations
A benchmark lane refused because the worker does not hold the model goes to
the brain (brain.constraint_and_file): on another online worker → reroute;
in central's catalog → pull (ACTIVATE, automatic when the gate allows, else
"✓ pull" on the allocation ticket); nowhere → not available.
Grader tok/s
/llm/model-metrics2 rows carry a throughput block per worker × quant ×
alloc and /llm/models/<m>/metrics carries worker_averages (Σ tokens /
Σ generation seconds over every recorded call), both from the call log. The
Grader tab showed "no calls recorded" everywhere because the rows had only a
bare tok/s number and worker_averages was a fixed [].
1 maintainer
Metadata
1 maintainer
Release files for hugpy-core 0.1.62
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hugpy_core-0.1.62.tar.gz | 195.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| hugpy_core-0.1.62-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 390.4 kB
Release files / hugpy_core-0.1.62.tar.gz
| Download URL | hugpy_core-0.1.62.tar.gz |
|---|---|
| Size | 195.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5f8e402edc779df5884a3a449562b4d62f37f1ffc7cf835fd92c2fcb02861e5c
|
|
BLAKE2b-256 checksum How to use checksums |
3d37526633afdc5beec97de99ace1be5942dbdb9b0ffebb5de9c31ead114842c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / hugpy_core-0.1.62-py3-none-any.whl
| Download URL | hugpy_core-0.1.62-py3-none-any.whl |
|---|---|
| Size | 195.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
352f25323d16b4aa027133d0f026b54f22a5cac84167e1288f881b6a99a6f34d
|
|
BLAKE2b-256 checksum How to use checksums |
840b72adb431fdceafb2cb94eb636eb4bcbf7ef9bf1058bf17d2da11db17723d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
