hugpy-media

The media arm of Hugpy (import name hugpy_media). The source tree is at 0.3.4; the installed deb and running service on this host are 0.3.3. The package does two jobs:

1. It is a self-contained arm: the /media media-intelligence console (a chat-first SPA) and its toolset API (/api/media/*), served by its own service on 127.0.0.1:7340. Hugpy Core proxies both.
2. It provides engine task plugins: runners and request builders for speech recognition, TTS, embeddings and similarity, keywords, summaries, vision chat, vision analysis (depth, detection, classification, segmentation), image generation, ComfyUI, and document/URL extraction. They register with hugpy_engine.tasks.

The arm contract every arm follows (toolset → Blueprint → standalone server → arms.d → core proxy) is documented in docs/arms.md. How the model routes pick and fall back between models is in docs/execute_prompt.md.

Layout

The package follows the shared imports/ convention. Cross-cutting primitives live in one cheap layer that never imports a model stack. The model stacks are cohesive subpackages imported lazily.

src/hugpy_media/
  imports/            shared primitives; importing this pulls in no model stack
    constants.py      ARM_NAME="media", UI_MOUNT="/media/", ML_TASKS (route -> task),
                      env-var NAMES, standalone host/port/env-file, LLM upstream default
    functions.py      have() (find_spec probe), lazy()/first_lazy(), dist_version(), env_str()
    classes.py        MediaError (hosts turn it into {"error": ...})
    init_imports.py   explicit stdlib + shared re-exports, C = constants
  toolset.py          get_toolset() -> {"media": {route: callable}}, get_manifest()
  routes.py           media_bp = register_categories(get_bp("media"), get_toolset())
  serve/__init__.py   standalone server: `hugpy-media serve`
  ui.py               dist_dir() -> the wheel's built SPA (ui_dist/)
  ui_dist/            built @hugpy/media-intelligence-ui (see ui_dist/PROVENANCE.md)
  api.py              back-compat shim: media_bp, ML_TASKS, normalize_ml_kwargs
  plugin.py           register(): the hugpy_engine.tasks entry point (LazyRunner proxies)
  plugin_builders.py  pydantic request builders per task
  extract.py          deterministic document/URL text ingest (path-jailed, SSRF-guarded)
  hooks.py            fleet-worker process/telemetry hooks
  model_battery.py    battery recorder
  comfy/ embed/ imagegen/ keywords/ pdfs/ schemas/ seo/ summarizers/
  text/ tts/ vision/ vision_analysis/ whisper_model/        the model stacks
  chunking.py         shim -> text/chunking.py

Modules import shared pieces through the layer: from hugpy_media.imports import constants as C, functions as F, MediaError. tests/test_import_policy.py forbids wildcard imports, so init_imports re-exports explicitly.

The toolset (media/*, 18 routes)

Every route is GET+POST at /api/media/<route>, with a trailing-slash variant. Request data (JSON body, else form, else query string) is pruned to the callable's parameters. The return value comes back as {"result": ...}, and an exception as 500 {"error": "<message>"}. Append ?help to any route for its parameters. Source: toolset.get_toolset(), verified against the live GET /api/media/manifest.

The model routes call hugpy_engine.execute_prompt with the task fixed by the route. Any task the client sends is ignored. model becomes model_key, and for vision image_b64 becomes images=[...]. The reserved worker pool HUGPY_ML_POOL is applied unless the task is listed in HUGPY_ML_GENERAL_ROUTE_TASKS (default image-text-to-text). The result is model_dump()ed, so it includes ok and the engine's resolution metadata (task, source, model_key, attempts).

Standalone vs needs core/GPU

• Works with nothing else running: health, capabilities, manifest, upload, extract, fetch. analyze works too for documents and URLs, with its enrich stages recorded as errors when no model answers.
• Needs a reachable core (HUGPY_CORE_URL, default http://127.0.0.1:7020) and a fleet worker that serves the task: transcribe, summarize, keywords, embed, similarity, imagine, and vision for GGUF vision models. The GPU work happens on the worker. The media box needs no torch or whisper for these.
• Needs local extras in this venv: depth, detect, classify, segment and transformers-framework vision. The deb's venv ships none of those extras, so on this host these routes return an {"error": ...}.
• capabilities reports local extras only. A route shown ready: false can still answer through core when the fleet serves its task.

Chat narration (/v1)

The SPA's chat narration does not use the toolset. It calls the OpenAI-compatible surface at the origin root: POST /v1/chat/completions (streamed over SSE) and GET /v1/models.

• Behind core (http://<core>/media/), /v1/* is core's own surface.
• Standalone, the server forwards /v1/* (streamed) to an upstream. With no reachable upstream it answers 503 {"error": {"code": "no_model_backend", ...}}.
  • 0.3.4 (source): the upstream is HUGPY_MEDIA_LLM_URL, else HUGPY_CORE_URL, else http://127.0.0.1:7020, core's loopback relay. Core serves /v1/chat/completions and /v1/models there.
  • 0.3.3 (installed here): the upstream is HUGPY_MEDIA_LLM_URL, default http://127.0.0.1:52017, the retired local wrapper port. Nothing listens on 52017 on this host, so a standalone /v1/models on :7340 returns that 503, while the same call through core succeeds. Set HUGPY_MEDIA_LLM_URL=http://127.0.0.1:7020 in /etc/hugpy/media.env, or upgrade to 0.3.4.

Standalone server and CLI

hugpy-media                     # same as `hugpy-media serve`
hugpy-media serve [--host H] [--port P] [--env-file /etc/hugpy/media.env]

serve loads the env file without overriding the environment. It then builds abstract_flask.get_Flask_app(name="hugpy-media", routes=hugpy_media.routes, url_prefix="api") and adds:

• the SPA at /media/ with history fallback (/ → 302 /media/). A build without ui_dist answers 500 with the reason.
• the /v1/* forwarder.

It then runs Flask's threaded server. get_Flask_app also adds /api/endpoints, /endpoints and /prefixes, and writes its audit log hugpy-media.log to the working directory.

curl -s 127.0.0.1:7340/api/media/health
curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com"}' 127.0.0.1:7340/api/media/fetch
curl -s -F file=@notes.pdf 127.0.0.1:7340/api/media/upload      # -> {"result": {"path": ...}}
curl -s '127.0.0.1:7340/api/media/vision?help'

Environment

The file /etc/hugpy/media.env (KEY=VALUE) is optional. The deb does not create it, and the unit reads it through EnvironmentFile=-.

Debian package

BUILD_PYTHON=/usr/bin/python3 packaging/debian/build.sh     # -> build/debian/hugpy-media_<ver>_amd64.deb (Python 3.12)
sudo apt install ./build/debian/hugpy-media_<ver>_amd64.deb      # 0.3.3 is the newest deb built so far

build.sh bundles offline wheels for hugpy-media and its private siblings (hugpy-platform and hugpy-engine, built from the local source tree) and their public dependencies. Since 0.3.4, hugpy-storage is no longer bundled. The 0.3.3 deb bundled it as a base dependency. It then asserts that the wheel contains ui_dist/index.html and that the abstract_flask wheel is in the bundle.

postinst:

• creates the hugpy-media system user and /var/lib/hugpy-media;
• builds /opt/hugpy-media/venv offline;
• writes /usr/share/hugpy/arms.d/media.json;
• writes and starts hugpy-media.service, with WorkingDirectory= and DEFAULT_ROOT= set to /var/lib/hugpy-media.

Core then proxies /api/media/* and /media/* to the service. The arm is not installed into core's venv. postrm removes the unit and the arms.d file. On purge it also removes the venv, the state, /etc/hugpy/media.env and the user. For host-level steps see docs/install.md and docs/operations.md.

Dependencies

Runtime (pyproject.toml, 0.3.4): hugpy-platform>=0.3.0, hugpy-engine>=0.3.2, flask>=2, abstract_flask, abstract_essentials, pydantic>=2, requests, beautifulsoup4, PyPDF2, pdfplumber, pillow.

hugpy-storage is not a base dependency since 0.3.4. It moved to the local extra. The local in-process inference runners (transformers, embed, keywords, vision, imagegen) resolve weights lazily through hugpy_storage.download_models.ensure_model, and those runners need the torch extras anyway. import hugpy_media never imports hugpy_storage, and the arm deb (fleet mode, where the GPU work happens on workers) does not need it.

Extras (each model stack is imported only inside its runner): audio (openai-whisper), imagegen, embed, transformers, vision, extract (python-docx, openpyxl, ...), keywords (keybert), comfy, local (hugpy-storage), tts (chatterbox-tts; opt-in only, because it conflicts with the rest), all (every extra except tts, including local), and test.

Entry points

Engine plugin

hugpy_media.plugin.register() registers these (task, framework) pairs with hugpy_engine.tasks, using LazyRunner proxies so registration imports no model stack:

• automatic-speech-recognition, text-to-speech, text-summarization, text2text-generation, keyword-extraction, feature-extraction, sentence-similarity, image-text-to-text, text-to-image and image-to-image on transformers;
• text-to-image and image-to-image on comfy;
• document-extraction and url-extraction on any framework;
• the four vision-analysis tasks on transformers.

Tasks in the engine's TASK_ROUTES table (speech recognition, TTS, summarization, keywords, embeddings, image generation) are relayed to core, not run by these plugin runners, in an arm process. The runners serve in-process callers such as a worker.

UI

The SPA source is ui/media_intelligence_ui. Its build is copied into src/hugpy_media/ui_dist/, and the provenance is in ui_dist/PROVENANCE.md.

Downloads and feature gallery

Get the Debian packages, architecture details, SHA-256 checksums and installation commands from Hugpy Downloads. The feature gallery shows the console, Agent, Station, Media and Video interfaces. Screenshots reflect the versions captured; the current interface may differ.