veri-agent

Template architecture for an agent based system. The main componenent is an agent service providing various (LangGraph defined) workflows in a chat context, persisting states per workflow and user. The second significant component is a knowledgebase service, providing secure access to various data sources and indexing capabilities (vector db, graph db).

Agent architecture overview Note: The knowledgebase service is not yet implemented as a separate service but part of the agents service.

Run service

Dev

cd agents/deployment/dev
docker compose up

Prod:

cd agents/deployment/prod
docker compose up

Python API example

Python API examples can be found in the examples folder.

Example to run the extract workflow:

source /opt/veri-agent-dev/env
uv run python -m examples.run_extract.py --kb bluedragon --llm nova_lite

Run ingestion

Example to run Veritone Support ingestion:

export QDRANT_URL=http://localhost:6335  # or 6333 for prod
source /opt/veri-agent-dev/env            # source /opt/ver-agent/env   # for prod
cd agents
puv run python ingest.py --kb veritone_support --dsn veritone_support --dsl /opt/veri-agent-dev/data/raw/veritone_support/

Example to run BlueDragon ingestion:

export QDRANT_URL=http://localhost:6335  # or 6333 for prod
source /opt/veri-agent-dev/env            # source /opt/ver-agent/env   # for prod
cd agents
uv run python ingest.py --kb bluedragon --dsn lanl --dsl /opt/veri-agent-dev/data/raw/lanl/defenses/ --tags defenses
uv run python ingest.py --kb bluedragon --dsn lanl --dsl /opt/veri-agent-dev/data/raw/lanl/performance/ --tags performance

Example to run Real-Time Crime ingestion:

export QDRANT_URL=http://localhost:6335  # or 6333 for prod
source /opt/veri-agent-dev/env            # source /opt/ver-agent/env   # for prod
uv run python ingest.py --kb realtimecrime

Roadmap

Agent service

Hydra configuration
API via FastAPI
Streamlit UI for quick testing
LangGraph workflow management
Provide LLM access to workflows
Provide tool access to workflows
Persist user state per workflow
Persist user state in a remote database/knowledge service?
Provide knowledge base access to workflows

Knowledgebases

Hydra configuration
Access to local or remote vector DB (configurable if running with knowledge service)
Docling integration
Access to local or remote graph DB (configurabel if running with knowledge service)
Service regularly indexing configured data sources
API to query data sources (with restrictions)
API to push data to knowledge base

Additional capabilities

LangFuse connection
Connect to AWS Bedrock knowledge base

API Notes

Artifacts

Tools can return artifacts that are not passed onwards to the agent but can be used by the frontend. Whenever we return an artifact, it shall contain a 'source' field to specify the origin that defines the schema for the rest of the data. Typically it then also holds a list of 'items' where each item shall contain a type field that can be used to determine how to render the item.

Example output for the dmh_video_search tool

{
  "type": "tool",
  "data": {
    "content": "Title: Jordan - Tourism - ancient city of Petra 2 | Description: Aerial views of the ancient city of Petra, one of the most important archeological sites in the world.\nTitle: Jordan - Tourism - ancient city of Petra 1 | Description: Aerial views of the ancient city of Petra, one of the most important archeological sites in the world.\nTitle: Jordan - Tourism - ancient city of Petra 2 | Description: Aerial views of the ancient city of Petra, one of the most important archeological sites in the world.",
    "additional_kwargs": {},
    "response_metadata": {},
    "type": "tool",
    "name": "dmh_video_search",
    "id": "6fa5261b-7c07-49d6-bd8b-e688eae5f0d9",
    "tool_call_id": "toolu_bdrk_018FX42xZqPkcxDhWudhA1Ud",
    "artifact": {
      "items": [
        {
          "assetId": 51947623,
          "name": "1SMBC5_CJF3SLGXRR",
          "metaData": [
            {
              "name": "Title",
              "value": "Jordan - Tourism - ancient city of Petra 2"
            },
            {
              "name": "Description",
              "value": "Aerial views of the ancient city of Petra, one of the most important archeological sites in the world."
            },
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:13"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Keywords",
              "value": "Architecture Art Monument Museum Ruins UNESCO Town"
            },
            {
              "name": "Format.FrameRate",
              "value": "25 fps"
            },
            {
              "name": "Format.BroadcastStandard",
              "value": ""
            }
          ],
          "metaDataDefault": [
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:13"
            },
            {
              "name": "Rights.Reproduction",
              "value": "Rights Managed"
            },
            {
              "name": "clipIngested",
              "value": "2023-02-09T17:42:52Z"
            },
            {
              "name": "Format.Duration.Display",
              "value": ""
            }
          ],
          "thumbnail": {
            "name": "thumbnail",
            "urls": {
              "https": "//cdnt3m-a.akamaihd.net/tem/warehouse/1SM/BC5/1SMBC5_CJF3SLGXRR_lt.jpg"
            }
          },
          "hasDownloadableComp": false,
          "url": "https://cdnt3mt-a.akamaihd.net/1SM/BC5/1SMBC5_CJF3SLGXRR_xp.f4v?__gda__=1725017594_4821e1524d4d4ae95bd5cb43d9c7d4a0"
        },
        {
          "assetId": 51947731,
          "name": "1SMBC5_CLGJL2JLBZ",
          "metaData": [
            {
              "name": "Title",
              "value": "Jordan - Tourism - ancient city of Petra 1"
            },
            {
              "name": "Description",
              "value": "Aerial views of the ancient city of Petra, one of the most important archeological sites in the world."
            },
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:58"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Keywords",
              "value": "Architecture Art Monument Museum Ruins UNESCO Town"
            },
            {
              "name": "Format.FrameRate",
              "value": "25 fps"
            },
            {
              "name": "Format.BroadcastStandard",
              "value": ""
            }
          ],
          "metaDataDefault": [
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:58"
            },
            {
              "name": "Rights.Reproduction",
              "value": "Rights Managed"
            },
            {
              "name": "clipIngested",
              "value": "2023-02-09T17:45:32Z"
            },
            {
              "name": "Format.Duration.Display",
              "value": ""
            }
          ],
          "thumbnail": {
            "name": "thumbnail",
            "urls": {
              "https": "//cdnt3m-a.akamaihd.net/tem/warehouse/1SM/BC5/1SMBC5_CLGJL2JLBZ_lt.jpg"
            }
          },
          "hasDownloadableComp": false,
          "url": "https://cdnt3mt-a.akamaihd.net/1SM/BC5/1SMBC5_CLGJL2JLBZ_xp-wmorm.f4v?__gda__=1725017595_466391df9505e71bda54107f49dad198"
        },
        {
          "assetId": 51876092,
          "name": "1SMBC5_26UQCQ13KF",
          "metaData": [
            {
              "name": "Title",
              "value": "Jordan - Tourism - ancient city of Petra 2"
            },
            {
              "name": "Description",
              "value": "Aerial views of the ancient city of Petra, one of the most important archeological sites in the world."
            },
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:13"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Keywords",
              "value": "Architecture Art Monument Museum Ruins UNESCO Town"
            },
            {
              "name": "Format.FrameRate",
              "value": "25 fps"
            },
            {
              "name": "Format.BroadcastStandard",
              "value": ""
            }
          ],
          "metaDataDefault": [
            {
              "name": "TE.DigitalFormat",
              "value": "High Definition"
            },
            {
              "name": "Resource.Class",
              "value": "Footage"
            },
            {
              "name": "Format.Duration",
              "value": "00:00:13"
            },
            {
              "name": "Rights.Reproduction",
              "value": "Rights Managed"
            },
            {
              "name": "clipIngested",
              "value": "2023-02-09T10:02:13Z"
            },
            {
              "name": "Format.Duration.Display",
              "value": ""
            }
          ],
          "thumbnail": {
            "name": "thumbnail",
            "urls": {
              "https": "//cdnt3m-a.akamaihd.net/tem/warehouse/1SM/BC5/1SMBC5_26UQCQ13KF_lt.jpg"
            }
          },
          "hasDownloadableComp": false,
          "url": "https://cdnt3mt-a.akamaihd.net/1SM/BC5/1SMBC5_26UQCQ13KF_xp-wmorm.f4v?__gda__=1725017596_95bd019151028cd187838c0666cd8560"
        }
      ],
      "totalCount": 40,
      "currentPage": 0,
      "pageSize": 30,
      "hasNextPage": true,
      "numberOfPages": 2,
      "hasPreviousPage": false,
      "type": "dmh_search_results"
    },
    "status": "success"
  }
}

User interaction

This is still WIP. The most likely method to handle user interaction is the use of a 'UserInteraction' tool that can be called by the agent to request user input.

Potential options are to request textual user input or provide a list of options to choose from. This could also refer to a list of artifacts from a previous tool call. For example the DMH Video Search tool could return a list of videos and the user could be asked to select one of them.

WIP: we could use a conditional edge to decide if the ask human tool shall be called (https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/wait-user-input/#interacting-with-the-agent) and then interrupt before the tool, modify the state and then continue. The agent node would have to specify which information is needed from the user. Alternatively we could have the agent quit the current workflow and start with a conditional start node dependent on if user input to an ask human tool ws sent. Both feel a bit hacky.

Metadata

Release files for veri-agents-playground 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for veri-agents-playground 0.1.0
File	Interpreter	ABI	Platform	Reset
veri_agents_playground-0.1.0-py3-none-any.whl 270.8 kB May 8, 2025	Python 3	none	any	Details

Release files / veri_agents_playground-0.1.0-py3-none-any.whl

Download URL	veri_agents_playground-0.1.0-py3-none-any.whl
Size	270.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`3049f510ec89ceb3566556db1f92d39d9ac611c4f14eaefad859b179d762e1b7`
BLAKE2b-256 checksum How to use checksums	`dd6885f93a0c34aab7034bc8f4a79fc419c429497ff1c371c3bda2ce4132145d`
Upload date	May 8, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.6.3`

Release history Release notifications | RSS feed

This release

0.1.0 This release

May 8, 2025 1 release file

veri-agents-playground 0.1.0