dc43-contracts-app

A FastAPI application that surfaces the dc43 governance experience. It relies on shared service clients to interact with contract, governance, and data product backends, and bundles HTML templates plus static assets for local demos and packaged deployments.

Features

• Browse and edit contracts, datasets, and data products backed by any dc43 service implementation.
• Inspect governance metrics alongside dataset and contract records with interactive trend charts so operators can review historical observations without leaving the UI.
• Export integration helper bundles to bootstrap Spark or Delta pipelines.
• Embed a documentation-driven chat assistant powered by LangChain and Gradio so teams can query the Markdown guides that ship with dc43.

Documentation chat assistant

The docs chat surface reuses off-the-shelf components—LangChain for retrieval augmented generation and Gradio for the UI—so the repository does not have to maintain bespoke chat widgets. To enable it:

1. Install the optional extra:

   pip install "dc43-contracts-app[docs-chat]"
   
   # Working from a source checkout? the root demo extra already pulls the
   # assistant stack – run `pip install --no-cache-dir -e ".[demo]"` once and
   # skip additional `dc43-contracts-app[docs-chat]` installs to avoid pip
   # dependency conflicts.
   

   Mixing both commands in the same environment (for example running the editable install and then invoking the PyPI extra) causes pip to report conflicting requirements because they reference the same local package.

2. Provide an API key via the configured environment variable (defaults to OPENAI_API_KEY) or set an inline secret with docs_chat.api_key in a private TOML file. When you rely on embedding_provider = "huggingface" the same OpenAI key is still used for chat completions, but embeddings no longer require an external service.

3. Toggle the feature in contracts-app.toml:

   [docs_chat]
   enabled = true
   provider = "openai"
   model = "gpt-4o-mini"
   embedding_provider = "huggingface" # Opt into "openai" to reuse hosted embeddings.
   embedding_model = "" # Defaults to sentence-transformers/all-MiniLM-L6-v2.
   api_key_env = "OPENAI_API_KEY"
   # api_key = "sk-your-api-key" # optional inline secret stored outside git
   # code_paths = ["~/project/sibling-module"] # optional extra source directories
   # reasoning_effort = "medium" # use with OpenAI `o4-*` models
   

   api_key_env records the name of the variable that stores your API key. Load the secret separately (for example via direnv, dotenv, a .env file passed to dc43-demo --env-file, or a shell export OPENAI_API_KEY=...).

4. Pass the config path to the demo launcher so the loader picks up your changes:

   dc43-demo --config /path/to/contracts-app.toml
   

   The legacy export DC43_CONTRACTS_APP_CONFIG=/path/to/contracts-app.toml workflow still works when you prefer a global environment variable.

5. Restart the dc43 app. The assistant indexes Markdown under docs/ and the source trees in src//packages/ from your dc43 checkout by default and ignores parent directories (for example ~/src). Override docs_chat.docs_path, docs_chat.code_paths, or docs_chat.index_path when the repository lives elsewhere.

   Embeddings are requested in small batches during the initial index build so the default Hugging Face workflow runs locally without tripping OpenAI's token limits. Point the assistant at large documentation or source directories without juggling manual chunk sizes. Prefer managed embeddings? set embedding_provider = "openai" and specify a compatible embedding_model. The docs-chat extra already installs langchain-huggingface and sentence-transformers, so leaving embedding_model empty keeps the sentence-transformers/all-MiniLM-L6-v2 default.

   The FastAPI application now kicks off the documentation index warm-up as it loads the configuration so the one-off downloads and FAISS persistence happen in the background while the UI comes online. Cached manifests are reused across restarts until the docs or code change, and if a prompt arrives mid warm-up the chat surface explains that it is waiting for the cached index to finish building before continuing.

   While users wait for an answer the chat UI streams progress updates—loading documentation, embedding batches, and querying OpenAI—before presenting the final, cited response. Programmatic callers receive the same step list in the JSON payload under a new steps field.

   For deployments that want the index ready before the app starts, run dc43-docs-chat-index --config /path/to/contracts-app.toml as part of your build pipeline. The CLI shares the same configuration loader and writes the FAISS cache to docs_chat.index_path (or the workspace default) so runtime warm-ups immediately reuse the persisted manifest.

The contracts setup wizard mirrors these settings via the Documentation assistant module. Pick the Gradio option under the User experience group to populate [docs_chat] in the exported dc43-contracts-app.toml and surface the assistant alongside other deployment assets.

The assistant is purposefully constrained to dc43 setup and usage. When a prompt strays outside that scope the response reminds the requester to come back with a dc43-specific question so the chat surface stays focused on project guidance.

Programmatic callers can POST to /api/docs-chat/messages with a JSON payload ({"message": "...", "history": [...]}) and receive answers plus cited sources. The embedded Gradio UI is mounted at /docs-chat/assistant and the HTML entry point lives at /docs-chat.

Environment variables

Combine these overrides with existing workspace and backend settings to tailor the dc43 app to your deployment environment.