hugpy-media

The Comfy renderer recovery records the restored battery profiles, current route connections, exact PNG-derived settings, and rollout requirements. The backend changes are staged; original battery data is preserved.

The media add-on arm for hugpy-core (distribution hugpy-media, import name hugpy_media). The Python wheel includes its /media/ UI and backend category tree. Installed into Core's Python environment, it runs as an embedded addon. The Debian package also provides a standalone service and discovery descriptor for installations using a separate process. The package does two jobs:

1. It is a self-contained arm: the /media media-intelligence console (a chat-first SPA) and its toolset API (/api/media/*), embedded in Core or served separately on 127.0.0.1:7340. Core proxies the separate deployment.
2. It provides engine task plugins: runners and request builders for speech recognition, TTS, embeddings and similarity, keywords, summaries, vision chat, vision analysis (depth, detection, classification, segmentation), image generation, ComfyUI, and document/URL extraction. They register with hugpy_engine.tasks.

The backend contract (category tree → Blueprint → standalone server → arms.d → Core proxy) is documented in ROUTES-CONVENTION.md. How the model routes pick and fall back between models is in docs/execute_prompt.md.

The shared media-intelligence[documents]>=0.1.3,<0.3 dependency owns native document parsing, persisted source-bound artifacts, page retrieval and search. OCR requests go through Hugpy workers. It remains a separate distribution; the hugpy-media add-on provides the hosted upload, authorization and API layer.

Running inside Core

Install the Media wheel and its base dependencies into the Python environment used by the updated Core service. Core then mounts hugpy_media.routes.media_bp at /api/media/*, retains /api/session/*, and loads the hugpy.arm_prepare and hugpy.arm_ui hooks. The packaged UI runs at /media/ on Core's port; there is no second Media HTTP process. Core continues to own /v1/*, Help, and /api/prompt (including the /api/prompt/execute spelling).

Media is optional: Core can run without the package installed. No model stack is loaded by the startup hook. Inference still dispatches through the existing Engine/Wrapper path; installing Media's base wheel does not install GPU extras.

Configuration belongs to the Core service environment. Embedded hosting does not read /etc/hugpy/media.env, since its global values could alter Core. Set HUGPY_MEDIA_UPLOADS_HOME to a dedicated directory if desired; otherwise uploads use the host's hugpy_platform.constants.UPLOADS_HOME. Document artifacts use its .documents child unless HUGPY_DOCUMENTS_HOME is configured. The Core service user needs write access to these directories. To retain existing upload sessions, use their existing directory and grant Core access before switching.

An existing arms.d/media.json does not override an embedded Media UI. Named API URLs such as /api/arms/media/media/health remain local too. External addons continue through the descriptor proxy. The standalone CLI and Debian service remain available as a separate deployment profile; installing that Debian package alone does not install Media into Core's virtual environment.

For a source build, install the wheel from python -m build --wheel package into Core's environment, with its base dependencies, and restart Core when applying the change. Avoid leaving the separate hugpy-media.service enabled once the embedded deployment is active. No UI rebuild is needed when using the wheel's existing ui_dist.

Layout

The package follows the shared imports/ convention. Cross-cutting primitives live in one cheap layer that never imports a model stack. The model stacks are cohesive subpackages imported lazily.

src/hugpy_media/
  imports/            shared primitives; importing this pulls in no model stack
    constants.py      ARM_NAME="media", UI_MOUNT="/media/", ML_TASKS (route -> task),
                      env-var NAMES, standalone host/port/env-file, LLM upstream default
    functions.py      have() (find_spec probe), lazy()/first_lazy(), dist_version(), env_str()
    classes.py        MediaError (hosts turn it into {"error": ...})
    init_imports.py   explicit stdlib + shared re-exports, C = constants
  toolset.py          get_toolset() -> {"media": {route: callable}}, get_manifest()
  routes.py           media_bp = register_categories(get_bp("media"), get_toolset())
  serve/__init__.py   standalone server: `hugpy-media serve`
  ui.py               dist_dir() -> the wheel's built SPA (ui_dist/)
  ui_dist/            built @hugpy/media-intelligence-ui (see ui_dist/PROVENANCE.md)
  api.py              back-compat shim: media_bp, ML_TASKS, normalize_ml_kwargs
  plugin.py           register(): the hugpy_engine.tasks entry point (LazyRunner proxies)
  plugin_builders.py  pydantic request builders per task
  extract.py          deterministic document/URL text ingest (path-jailed, SSRF-guarded)
  hooks.py            fleet-worker process/telemetry hooks
  model_battery.py    battery recorder
  comfy/ embed/ imagegen/ keywords/ pdfs/ schemas/ seo/ summarizers/
  text/ tts/ vision/ vision_analysis/ whisper_model/        the model stacks
  chunking.py         shim -> text/chunking.py

Modules import shared pieces through the layer: from hugpy_media.imports import constants as C, functions as F, MediaError. tests/test_import_policy.py forbids wildcard imports, so init_imports re-exports explicitly.

The toolset (media/*, 18 routes)

Every route is GET+POST at /api/media/<route>, with a trailing-slash variant. Request data (JSON body, else form, else query string) is pruned to the callable's parameters. The return value comes back as {"result": ...}, and an exception as 500 {"error": "<message>"}. Append ?help to any route for its parameters. Source: toolset.get_toolset(), verified against the live GET /api/media/manifest.

The model routes call hugpy_engine.execute_prompt with the task fixed by the route. Any task the client sends is ignored. model becomes model_key, and for vision image_b64 becomes images=[...]. The reserved worker pool HUGPY_ML_POOL is applied unless the task is listed in HUGPY_ML_GENERAL_ROUTE_TASKS (default image-text-to-text). The result is model_dump()ed, so it includes ok and the engine's resolution metadata (task, source, model_key, attempts).

Standalone vs needs core/GPU

• Works with nothing else running: health, capabilities, manifest, upload, extract, fetch. analyze works too for documents and URLs, with its enrich stages recorded as errors when no model answers.
• Needs a reachable core (HUGPY_CORE_URL, default http://127.0.0.1:7020) and a fleet worker that serves the task: transcribe, summarize, keywords, embed, similarity, imagine, and vision for GGUF vision models. The GPU work happens on the worker. The media box needs no torch or whisper for these.
• Needs local extras in this venv: depth, detect, classify, segment and transformers-framework vision. The deb's venv ships none of those extras, so on this host these routes return an {"error": ...}.
• capabilities reports local extras only. A route shown ready: false can still answer through core when the fleet serves its task.

Chat narration (/v1)

The SPA's chat narration does not use the toolset. It calls the OpenAI-compatible surface at the origin root: POST /v1/chat/completions (streamed over SSE) and GET /v1/models.

• Behind core (http://<core>/media/), /v1/* is core's own surface.
• Standalone, the server forwards /v1/* (streamed) to an upstream. With no reachable upstream it answers 503 {"error": {"code": "no_model_backend", ...}}.
  • 0.3.4 (source): the upstream is HUGPY_MEDIA_LLM_URL, else HUGPY_CORE_URL, else http://127.0.0.1:7020, core's loopback relay. Core serves /v1/chat/completions and /v1/models there.
  • 0.3.3 (installed here): the upstream is HUGPY_MEDIA_LLM_URL, default http://127.0.0.1:52017, the retired local wrapper port. Nothing listens on 52017 on this host, so a standalone /v1/models on :7340 returns that 503, while the same call through core succeeds. Set HUGPY_MEDIA_LLM_URL=http://127.0.0.1:7020 in /etc/hugpy/media.env, or upgrade to 0.3.4.

Standalone server and CLI

hugpy-media                     # same as `hugpy-media serve`
hugpy-media serve [--host H] [--port P] [--env-file /etc/hugpy/media.env]

serve loads the env file without overriding the environment. It then builds abstract_flask.get_Flask_app(name="hugpy-media", routes=hugpy_media.routes, url_prefix="api") and adds:

• the SPA at /media/ with history fallback (/ → 302 /media/). A build without ui_dist answers 500 with the reason.
• the /v1/* forwarder.

It then runs Flask's threaded server. get_Flask_app also adds /api/endpoints, /endpoints and /prefixes, and writes its audit log hugpy-media.log to the working directory.

curl -s 127.0.0.1:7340/api/media/health
curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com"}' 127.0.0.1:7340/api/media/fetch
curl -s -F file=@notes.pdf 127.0.0.1:7340/api/media/upload      # -> {"result": {"path": ...}}
curl -s '127.0.0.1:7340/api/media/vision?help'

Environment

The file /etc/hugpy/media.env (KEY=VALUE) is optional. The deb does not create it, and the unit reads it through EnvironmentFile=-.

Debian package

BUILD_PYTHON=/usr/bin/python3 packaging/debian/build.sh     # -> build/debian/hugpy-media_<ver>_amd64.deb (Python 3.12)
sudo apt install ./build/debian/hugpy-media_<ver>_amd64.deb      # 0.3.3 is the newest deb built so far

build.sh bundles offline wheels for hugpy-media and its private siblings (hugpy-platform and hugpy-engine, built from the local source tree) and their public dependencies. Since 0.3.4, hugpy-storage is no longer bundled. The 0.3.3 deb bundled it as a base dependency. It then asserts that the wheel contains ui_dist/index.html and that the abstract_flask wheel is in the bundle.

postinst:

• creates the hugpy-media system user and /var/lib/hugpy-media;
• builds /opt/hugpy-media/venv offline;
• writes /usr/share/hugpy/arms.d/media.json;
• writes and starts hugpy-media.service, with WorkingDirectory= and DEFAULT_ROOT= set to /var/lib/hugpy-media.

Core then proxies /api/media/* and /media/* to the service. The arm is not installed into core's venv. postrm removes the unit and the arms.d file. On purge it also removes the venv, the state, /etc/hugpy/media.env and the user. For host-level steps see docs/install.md and docs/operations.md.

Dependencies

Runtime (pyproject.toml, 0.3.4): hugpy-platform>=0.3.0, hugpy-engine>=0.3.2, flask>=2, abstract_flask, abstract_essentials, pydantic>=2, requests, beautifulsoup4, PyPDF2, pdfplumber, pillow.

hugpy-storage is not a base dependency since 0.3.4. It moved to the local extra. The local in-process inference runners (transformers, embed, keywords, vision, imagegen) resolve weights lazily through hugpy_storage.download_models.ensure_model, and those runners need the torch extras anyway. import hugpy_media never imports hugpy_storage, and the arm deb (fleet mode, where the GPU work happens on workers) does not need it.

Extras (each model stack is imported only inside its runner): audio (openai-whisper), imagegen, embed, transformers, vision, extract (python-docx, openpyxl, ...), keywords (keybert), comfy, local (hugpy-storage), tts (chatterbox-tts; opt-in only, because it conflicts with the rest), all (every extra except tts, including local), and test.

Entry points

Engine plugin

hugpy_media.plugin.register() registers these (task, framework) pairs with hugpy_engine.tasks, using LazyRunner proxies so registration imports no model stack:

• automatic-speech-recognition, text-to-speech, text-summarization, text2text-generation, keyword-extraction, feature-extraction, sentence-similarity, image-text-to-text, text-to-image and image-to-image on transformers;
• text-to-image and image-to-image on comfy;
• document-extraction and url-extraction on any framework;
• the four vision-analysis tasks on transformers.

Tasks in the engine's TASK_ROUTES table (speech recognition, TTS, summarization, keywords, embeddings, image generation) are relayed to core, not run by these plugin runners, in an arm process. The runners serve in-process callers such as a worker.

UI

The SPA source is ui/media_intelligence_ui. Its build is copied into src/hugpy_media/ui_dist/, and the provenance is in ui_dist/PROVENANCE.md.

Downloads and feature gallery

Get the Debian packages, architecture details, SHA-256 checksums and installation commands from Hugpy Downloads. The feature gallery shows the console, Agent, Station, Media and Video interfaces. Screenshots reflect the versions captured; the current interface may differ.