vmi-llm-sdk 0.2.0

VMI LLM SDK — prompts, memory (vector + connected DB), transactions, multi-provider LLM calls

VMI LLM SDK (Python)

A configuration-driven SDK for building AI features on top of the VMI platform. It integrates two VMI backend services and calls LLM providers directly with your own API keys:

Service	Auth	Used for
vmi-backend	X-Access-Token header (or JWT)	prompt definitions, 2-phase call logging, connected-DB context, tagged memory data
vmi-orchestrator (vector DB)	Authorization: Bearer	semantic memory search/write, sessions, transactions
LLM providers (OpenAI, Anthropic/Claude, Gemini, DeepSeek)	your own API keys	the actual model call

This README documents every public function, method and class in the SDK — what it does, every parameter it accepts, what it returns, what it raises, and a runnable example. Use the table of contents to jump straight to what you need.

---

Table of contents

• Install
• Quick start
• Configuration — VmiConfig
• Module-level API (llm_sdk)
• VmiClient — the entry point
  • Health
  • Memory functions
  • LLM functions
  • Transaction functions
  • Session functions
• Data models
• Exceptions
• Internals reference
  • MemoryService
  • LLMService
  • VectorDBClient
  • BackendClient
  • HttpClient
  • PromptCache
  • resolvePrompt
  • Providers
• Transactions
• Sessions
• Architecture
• Test bench (web UI)
• Tests

---

Install

pip install -e .            # core (requests only)
pip install -e ".[all]"     # + openai, anthropic, google-generativeai

Provider keys can be supplied via VmiConfig fields or the matching environment variable: OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, DEEPSEEK_API_KEY. DeepSeek reuses the OpenAI SDK against a different base_url, so it needs no separate package.

---

Quick start

from llm_sdk import VmiClient, VmiConfig

# backendUrl / vectorDbUrl are NOT constructor arguments — they are
# hardcoded inside llm_sdk/config.py and always point at the fixed
# vmi-backend / vmi-orchestrator deployment this SDK talks to. You only
# supply credentials and identity.
client = VmiClient(VmiConfig(
    accessToken="<company access token>",
    vectorDbToken="<vector db token>",
    companyCode="WPCORP4812",
    background="You help engineers with the ACME order platform.",
))

Or the module-level API, which keeps a single default client for you:

import llm_sdk
llm_sdk.init(config)
llm_sdk.callLLM(promptId="PROMPT-XXXX", inputParams={"question": "..."})

---

Configuration — VmiConfig

llm_sdk.config.VmiConfig is a plain dataclass holding every setting the SDK needs. Every field has a sensible default ("", 0, etc.) except where noted, so you only set what your use case requires — but two "require" methods enforce the minimum needed for a given operation (see below).

backendUrl and vectorDbUrl are hardcoded, not configurable. They are module-level constants (_BACKEND_URL, _VECTOR_DB_URL) baked into llm_sdk/config.py, wired onto VmiConfig as init=False fields. Every consumer of this library talks to the same fixed vmi-backend / vmi-orchestrator deployment — these are infrastructure details, not per-consumer configuration. Passing backendUrl= or vectorDbUrl= to VmiConfig(...) raises a TypeError (they aren't constructor parameters at all); to point at a different deployment you edit the constants in config.py itself. You only ever supply credentials (accessToken/jwtToken/vectorDbToken) and identity (companyCode/tenantId).

Field	Type	Default	Description
backendUrl	str	(hardcoded, not settable)	Base URL of vmi-backend. Fixed in llm_sdk/config.py; not a constructor argument. Used for prompts, call logging, connected-DB context, tagged memory data.
accessToken	str	""	Company access token, sent as X-Access-Token. Required for vmi-backend's public endpoints (prompts, call logging, memory-data, extract-open).
jwtToken	str	""	Alternative/additional auth: sent as Authorization: Bearer <jwtToken> together with X-Tenant-ID. Required for the JWT-protected endpoints (/database-knowledge/extract, /memory-data/fetch).
tenantId	str	""	Tenant id sent as X-Tenant-ID alongside jwtToken. If left blank and companyCode is set, it is auto-filled from companyCode (and vice versa) in __post_init__.
vectorDbUrl	str	(hardcoded, not settable)	Base URL of vmi-orchestrator (the vector DB). Fixed in llm_sdk/config.py; not a constructor argument. Used for all memory operations (fetchMemory, addMemory, sessions, transactions).
vectorDbToken	str	""	Token sent as Authorization: Bearer <vectorDbToken> to vmi-orchestrator.
openaiApiKey	str	os.environ["OPENAI_API_KEY"]	OpenAI key. Falls back to the environment variable if not set explicitly. Also used as the summarizer model's key.
anthropicApiKey	str	os.environ["ANTHROPIC_API_KEY"]	Claude/Anthropic key.
googleApiKey	str	os.environ["GOOGLE_API_KEY"]	Gemini key.
deepseekApiKey	str	os.environ["DEEPSEEK_API_KEY"]	DeepSeek key (used with the OpenAI-compatible client).
companyCode	str	""	Identifies the calling company to vmi-backend; sent on initiateCall/completeCall and connected-DB requests. Same identifier as tenantId — whichever one you set fills the other.
environmentCode	str	""	Environment identifier (e.g. "prod", "staging") sent on call-logging requests.
companyProjectCode	str	""	Default memory isolation scope. When set, every vector write is tagged with metadata.companyProjectCode and every search is filtered by it, so multiple projects sharing one orchestrator never see each other's memories. Per-call companyProjectCode arguments to fetchMemory override this default.
background	str	""	Default "purpose" text injected into the system prompt when callLLM(passBackground=True) is called without an explicit string.
summarizerModel	str	"gpt-4o-mini"	Fixed internal OpenAI model used to condense fetched memory locally before the main LLM call, when the orchestrator doesn't return its own summary.
defaultCollection	str	"vectors"	Vector-DB collection used when a call doesn't specify one.
defaultTopK	int	3	Default number of results returned by fetchMemory/search when topK isn't specified.
connectionTimeoutSeconds	float	10.0	TCP connect timeout for every HTTP call the SDK makes.
readTimeoutSeconds	float	60.0	Read timeout for every HTTP call.
maxRetries	int	2	Number of retry attempts on transient HTTP failures (network errors, 502/503/504, and other configured retryable statuses).
promptCacheTtlSeconds	float	300.0	How long a prompt definition fetched from vmi-backend is cached before being re-fetched.

VmiConfig.__post_init__()

Runs automatically after construction. If only one of companyCode / tenantId is set, it copies that value into the other, since they identify the same tenant to the two different backends.

VmiConfig.requireBackend() -> None

Validates that the config has enough to talk to vmi-backend: backendUrl must be set (it always is, since it's hardcoded), and at least one of accessToken or jwtToken must be set. Raises VmiConfigError if the credential is missing. Called internally before any backend-dependent operation (callLLM with a promptId, fetchMemoryFromConnectedDB, fetchPrompt).

VmiConfig.requireVectorDb() -> None

Validates that vectorDbUrl is set — always true in practice since it's hardcoded; this check exists mainly as a defensive guard. Called internally before any vector-DB operation (fetchMemory, addMemory, addMemories, deleteMemory, getMemory, sessions, transactions, ping).

---

Module-level API (llm_sdk)

For scripts and simple apps that only need one client, llm_sdk exposes a default-client-backed functional API so you don't have to thread a VmiClient instance through your code.

llm_sdk.init(config: VmiConfig) -> VmiClient

Creates a new VmiClient from config and stores it as the module's default client (used by every function below). Returns the created client in case you want to keep a reference to it too. Calling init again replaces the previous default client.

import llm_sdk
client = llm_sdk.init(VmiConfig(vectorDbToken="..."))

llm_sdk.getClient() -> VmiClient

Returns the default client set by init.

• Raises: VmiConfigError if init() was never called.

llm_sdk.fetchMemory(query: str, **kwargs) -> MemoryData

Thin wrapper: calls getClient().fetchMemory(query, **kwargs). See VmiClient.fetchMemory for the full parameter list.

llm_sdk.addMemory(text: str, **kwargs) -> dict[str, Any]

Thin wrapper: calls getClient().addMemory(text, **kwargs). See VmiClient.addMemory.

llm_sdk.fetchMemoryFromConnectedDB(promptText: str, **kwargs) -> ConnectedDBContext

Thin wrapper: calls getClient().fetchMemoryFromConnectedDB(promptText, **kwargs). See VmiClient.fetchMemoryFromConnectedDB.

llm_sdk.callLLM(**kwargs) -> LLMCallResult

Thin wrapper: calls getClient().callLLM(**kwargs). See VmiClient.callLLM.

---

VmiClient — the entry point

llm_sdk.client.VmiClient is the object-oriented entry point of the SDK. Construct one per configuration/tenant; it wires up the backend client, vector-DB client, memory service, prompt cache and LLM service for you.

from llm_sdk import VmiClient, VmiConfig

# backendUrl / vectorDbUrl are hardcoded (see VmiConfig above) — only
# credentials and identity are passed in here.
client = VmiClient(VmiConfig(
    accessToken="...",
    vectorDbToken="...",
    companyCode="WPCORP4812",
))

VmiClient.__init__(config: VmiConfig)

Builds the client. Since config.backendUrl is always set (it's hardcoded), this always creates a BackendClient and a PromptCache on top of it; backend-dependent methods (fetchMemoryFromConnectedDB, callLLM with promptId, fetchPrompt) will still raise VmiConfigError at call time if you haven't supplied accessToken or jwtToken. The vector-DB client is likewise always created (config.vectorDbUrl is hardcoded too); the lazy config.requireVectorDb() check on each vector-DB call is a defensive no-op that only exists to fail loudly if that constant were ever cleared.

• Parameters: config (VmiConfig) — the SDK configuration described above.

Health

VmiClient.ping() -> dict[str, Any]

Liveness check against vmi-orchestrator (GET /health, unauthenticated, outside /api/v1).

• Returns: {"status": "ok"} (or whatever JSON body the orchestrator returns) when it's reachable.
• Raises: VmiConfigError if vectorDbUrl isn't configured; VmiApiError if the orchestrator is unreachable or returns a non-2xx status after retries.

client.ping()  # {"status": "ok"}

Memory functions

fetchMemory

VmiClient.fetchMemory(query: str, **kwargs) -> MemoryData

Semantic memory search against vmi-orchestrator's /api/v1/vectors/search. Delegates to MemoryService.fetchMemory (documented in full in Internals reference); this is the signature you actually call:

Parameter	Type	Default	Description
query	str	—	The natural-language text to search for (embedded and matched against stored vectors).
storage	str	"longterm"	"longterm" or "session". Session storage requires sessionId.
sessionId	Optional[str]	None	Session to search within, when storage="session".
collection	Optional[str]	None	Vector collection name; falls back to config.defaultCollection.
topK	Optional[int]	None	Max number of results; falls back to config.defaultTopK.
filters	Optional[dict]	None	Metadata filters passed straight through to the orchestrator (e.g. {"userId": "user_1"}).
tags	Optional[dict[str, str]]	None	Tag filters, ANDed together (e.g. {"projectCode": "PRJ1"}).
summarize	bool	True	Ask the orchestrator to also return a server-side summary of the hits.
companyProjectCode	Optional[str]	None	Overrides config.companyProjectCode for this call, scoping the search to one project.
rerank	bool	False	Fetch a wider candidate pool and re-rank it with a cross-encoder (longterm storage only).
rerankTopN	Optional[int]	None	Size of the candidate pool before reranking (orchestrator default is 20).
rerankMode	Optional[str]	None	Reranking strategy name, passed through to the orchestrator.
contentType	Optional[str]	None	One of CODING | CHAT | VIDEO | DOCUMENT | GENERAL — selects the orchestrator's retrieval strategy for that content type.
transaction	Optional[VmiTransaction]	None	If given, the search is scoped to the transaction's tx_id, so it can see staged/uncommitted writes.

• Returns: MemoryData — see Data models.
• Raises: VmiConfigError if vectorDbUrl isn't set; VmiApiError on unrecoverable API failures.
• Resilience: if summarize=True and the orchestrator returns a server error (≥500), the call is retried once with summarize=False rather than failing outright — the server-side summarizer is the flakiest part of search. Callers that need a summary (like callLLM) fall back to local summarization when this happens.

memory = client.fetchMemory(
    "what did we decide about caching?",
    storage="longterm",
    topK=5,
    filters={"userId": "user_1"},
    tags={"projectCode": "PRJ1"},
    summarize=True,
)
memory.memoryText   # numbered plain-text block, ready to paste into an LLM prompt
memory.dataNodes    # [DataNode(id, text, score, metadata), ...]
memory.summary      # server-side summary (present when summarize=True and it succeeded)

addMemory

VmiClient.addMemory(text: str, **kwargs) -> dict[str, Any]

Writes one memory entry to vector storage (POST /api/v1/vectors/write).

Parameter	Type	Default	Description
text	str	—	The text to embed and store.
vectorId	Optional[str]	None	Explicit id for the entry; auto-generated (mem_<uuid4>) if omitted.
metadata	Optional[dict]	None	Arbitrary metadata stored alongside the vector (used later for filters in fetchMemory).
tags	Optional[dict[str, str]]	None	Tags stored alongside the vector (used later for tags AND-filters in fetchMemory).
storage	str	"longterm"	"longterm" or "session".
sessionId	Optional[str]	None	Session to write into, when storage="session".
collection	Optional[str]	None	Vector collection; falls back to config.defaultCollection.
indexId	Optional[str]	None	Optional index selector passed through to the orchestrator.
isTransactional	bool	False	Stage the write server-side without a client-managed transaction object.
transaction	Optional[VmiTransaction]	None	If given, implies isTransactional=True and binds the write to this transaction's tx_id (commit/rollback as a group).

• Returns: the raw JSON response from the orchestrator's write endpoint (a dict).
• Raises: VmiConfigError if vectorDbUrl isn't set; VmiApiError on failure.

client.addMemory(
    "The user prefers Python over Java",
    metadata={"type": "preference"},
    tags={"userId": "user_1"},
    isTransactional=True,
)

addMemories

VmiClient.addMemories(items: list[dict[str, Any]], **kwargs) -> dict[str, Any]

Batch write (POST /api/v1/vectors/batch) — writes many memory entries in one HTTP call.

Parameter	Type	Default	Description
items	list[dict]	—	Each item is {"text": ..., "vector_id"? / "vectorId"?, "metadata"?, "tags"?}. text is required per item; if metadata doesn't already contain "text", it's added automatically (the batch endpoint, unlike single write, does not copy the text into metadata server-side, and without it the entries would be unsearchable/unreadable in results).
storage	str	"longterm"	"longterm" or "session".
sessionId	Optional[str]	None	Session to write into.
collection	Optional[str]	None	Vector collection; falls back to config.defaultCollection.
isTransactional	bool	False	Stage the whole batch server-side.
transaction	Optional[VmiTransaction]	None	Applies the transaction's tx_id to the batch and to every item in it.

• Returns: the raw JSON response from the batch endpoint (a dict).
• Raises: VmiConfigError if vectorDbUrl isn't set; VmiApiError on failure.

client.addMemories([
    {"text": "User likes dark mode", "metadata": {"type": "preference"}},
    {"text": "User's timezone is IST", "tags": {"userId": "user_1"}},
])

deleteMemory

VmiClient.deleteMemory(vectorId: str, **kwargs) -> dict[str, Any]

Deletes one memory entry by id (DELETE /api/v1/vectors/delete).

Parameter	Type	Default	Description
vectorId	str	—	The id of the entry to delete.
storage	str	"longterm"	"longterm" or "session".
sessionId	Optional[str]	None	Session the entry lives in, when storage="session".
collection	Optional[str]	None	Vector collection; falls back to config.defaultCollection.
isTransactional	bool	False	Stage the delete server-side.
transaction	Optional[VmiTransaction]	None	Binds the delete to this transaction's tx_id.

• Returns: the raw JSON response from the delete endpoint (a dict).
• Raises: VmiConfigError if vectorDbUrl isn't set; VmiApiError on failure (e.g. unknown vectorId).

client.deleteMemory("mem_abc123", storage="longterm")

getMemory

VmiClient.getMemory(vectorId: str, **kwargs) -> dict[str, Any]

Fetches a single memory entry by id (GET /api/v1/vectors/{vectorId}).

Parameter	Type	Default	Description
vectorId	str	—	The id of the entry to fetch.
storage	str	"longterm"	"longterm" or "session".
collection	Optional[str]	None	Vector collection; falls back to config.defaultCollection.
sessionId	Optional[str]	None	Session the entry lives in, when storage="session".

• Returns: the raw JSON response describing that entry (a dict).
• Raises: VmiConfigError if vectorDbUrl isn't set; VmiApiError if the id doesn't exist or the call fails.

entry = client.getMemory("mem_abc123")

fetchMemoryFromConnectedDB

VmiClient.fetchMemoryFromConnectedDB(promptText: str, **kwargs) -> ConnectedDBContext

Searches the company's registered/connected databases for context relevant to promptText and returns a consolidated summary (POST /api/v1/database-knowledge/extract-open, or /extract under JWT auth).

Parameter	Type	Default	Description
promptText	str	—	The text to find relevant database context for (typically the user's message).
connectedDatabaseCodes	Optional[list[str]]	None	Restrict the search to specific registered database codes; searches all connected databases if omitted.
entityIdentifier	Optional[dict[str, str]]	None	Identifies a specific entity to scope the search to, e.g. {"type": "user", "id": "user_123"}.
topK	int	5	Max number of facts/results to retrieve.
purpose	Optional[str]	None	Free-text purpose/intent hint passed to the extraction backend.
identifierText	Optional[str]	None	Free-text identifier hint (used when entityIdentifier isn't structured).
transaction	Optional[VmiTransaction]	None	Passes the transaction's tx_id in the request body.

• Returns: ConnectedDBContext — see Data models.
• Raises: VmiConfigError if backendUrl/auth aren't configured; VmiApiError on failure.

context = client.fetchMemoryFromConnectedDB(
    "recent orders for this customer",
    connectedDatabaseCodes=["DB-ORDERS"],
    entityIdentifier={"type": "user", "id": "user_123"},
)
context.consolidatedSummary

LLM functions

callLLM

VmiClient.callLLM(**kwargs) -> LLMCallResult

Executes an LLM call through the full pipeline that mirrors vmi-backend's CompanyProjectStepService.createStep (memory retrieval → connected-DB context → system-prompt assembly → provider call → call logging → memory write-back). This is the SDK's flagship function — see the pipeline diagram below for exactly what happens on each call.

Parameter	Type	Default	Description
promptId	Optional[str]	None	Id of a prompt definition stored in vmi-backend. Mutually exclusive with rawPrompt — pass exactly one. When used, the vendor/model default to the prompt definition's aiVendorCode/aiVendorVersionCode unless provider/model override them.
rawPrompt	Optional[str]	None	An inline prompt template (with {{placeholders}}) instead of a stored prompt. Requires provider and model to be set explicitly.
inputParams	Optional[dict[str, Any]]	None	Values substituted into {{name}} placeholders in the prompt text (see resolvePrompt). Non-string values are JSON-encoded.
provider	Optional[str]	None	Vendor code: OPENAI | CLAUDE | GEMINI | DEEPSEEK (case-insensitive). Overrides the prompt definition's vendor when both are given.
model	Optional[str]	None	Model/version code, e.g. "gpt-4o", "claude-sonnet-5". Overrides the prompt definition's model when both are given.
passMemory	bool	False	When True, fetches relevant memory for the user message and injects a summarized version into the system prompt. Non-fatal — a fetch/summarize failure degrades to no memory context, never blocks the call.
saveInMemory	bool	False	When True, instructs the model (via the system prompt) to return {"output": ..., "summary": ...} JSON, and writes summary back to vector storage after the call succeeds.
connectedDB	bool	False	When True, fetches connected-database context for the user message and injects its consolidated summary into the system prompt. Non-fatal.
connectedDatabaseCodes	Optional[list[str]]	None	Restricts the connected-DB fetch (when connectedDB=True) to specific database codes.
passBackground	bool | str	False	When True, injects config.background as the purpose/role grounding at the top of the system prompt. Pass a str instead to override the configured background for just this call.
storage	str	"longterm"	Memory storage backend used for passMemory fetches and saveInMemory writes: "longterm" or "session".
sessionId	Optional[str]	None	Session id, required when storage="session". See the "first message" note below.
collection	Optional[str]	None	Vector collection for memory operations; falls back to config.defaultCollection.
memoryTopK	Optional[int]	None	topK passed to the internal fetchMemory call.
memoryFilters	Optional[dict]	None	filters passed to the internal fetchMemory call.
memoryTags	Optional[dict[str, str]]	None	tags passed to the internal fetchMemory call.
temperature	float	0.7	Sampling temperature passed to the provider.
maxTokens	int	2048	Max output tokens requested from the provider.
transaction	Optional[VmiTransaction]	None	Propagated to every child call this pipeline makes (memory fetch, connected-DB fetch, call-log initiate/complete, memory write-back), so the whole call — including its memory writes — can be staged and committed/rolled back atomically.

• Returns: LLMCallResult — see Data models.
• Raises: VmiConfigError if neither/both of promptId/rawPrompt are given, or rawPrompt is given without provider+model, or backend config is missing when promptId is used; ProviderError if the underlying vendor call fails (memory/connected-DB/call-logging failures are swallowed and logged instead, per the non-fatal-periphery rule — only the LLM call itself can fail the request).
• Session first-message rule: mirrors createProjectStep's previousStepCount check — the first message sent to a given sessionId (tracked per VmiClient instance, in-memory) skips the passMemory fetch entirely, since a brand-new session has no memory yet.
• Output parsing: when saveInMemory=True, the system prompt asks for {"output": "...", "summary": "..."} JSON. If the model doesn't comply, the raw text is used as output and summary is None — the call never fails just because the model ignored the format instruction.

result = client.callLLM(
    promptId="PROMPT-XXXX",          # OR rawPrompt="..." + provider= + model=
    inputParams={"question": "..."}, # resolves {{question}} in the prompt

    passMemory=True,      # fetchMemory + gpt-4o-mini summarization → system prompt
    saveInMemory=True,    # LLM returns {"output","summary"}; summary written back
    connectedDB=True,     # inject connected-DB consolidated summary
    passBackground=True,  # inject purpose grounding (config.background or a str)

    storage="longterm", sessionId=None,
    temperature=0.7, maxTokens=2048,
)
result.output        # main response text
result.summary       # memory summary (present when saveInMemory=True)
result.callCode      # vmi-backend call-tracking code
result.totalTokens, result.durationMs

Pipeline executed per call (see LLMService for the full breakdown of each step):

resolve prompt (promptId → backend, TTL-cached | rawPrompt)
  → passMemory:    fetchMemory → summarize (gpt-4o-mini, non-fatal)
  → connectedDB:   extract-open → consolidatedSummary (non-fatal)
  → system prompt: background + summarized memory + DB context (+ JSON format if saveInMemory)
  → initiate call log (backend, non-fatal)
  → provider call (OPENAI | CLAUDE | GEMINI | DEEPSEEK)
  → complete call log (tokens, duration, status, memoryData)
  → saveInMemory:  write summary to vector DB

fetchPrompt

VmiClient.fetchPrompt(promptId: str) -> PromptDefinition

Fetches (and TTL-caches) a prompt definition from vmi-backend (GET /api/v1/public/ai-prompts/{promptId}) without executing it.

Parameter	Type	Description
promptId	str	The prompt's visibleId.

• Returns: PromptDefinition — see Data models.
• Raises: VmiConfigError if backendUrl/auth aren't configured; PromptNotFoundError if the id doesn't resolve.
• Caching: results are cached per promptId for config.promptCacheTtlSeconds (default 300s); repeated calls within the TTL don't hit the network.

prompt = client.fetchPrompt("PROMPT-XXXX")
prompt.promptContent, prompt.aiVendorCode, prompt.aiVendorVersionCode

invalidatePromptCache

VmiClient.invalidatePromptCache(promptId: Optional[str] = None) -> None

Forces the next fetchPrompt/callLLM(promptId=...) call to re-fetch from vmi-backend instead of using the cached definition.

Parameter	Type	Default	Description
promptId	Optional[str]	None	Invalidate only this prompt's cache entry. If None, clears the entire cache.

• Returns: None. No-op if the client has no backend configured.

client.invalidatePromptCache("PROMPT-XXXX")   # one entry
client.invalidatePromptCache()                # everything

Transaction functions

All vector-DB write/search operations in this SDK can be scoped to a transaction. See Transactions below for the concept; this section documents the individual methods.

beginTransaction

VmiClient.beginTransaction(
    backend: str = "longterm",
    collection: Optional[str] = None,
    sessionId: Optional[str] = None,
) -> VmiTransaction

Opens a new transaction on vmi-orchestrator (POST /api/v1/tx/begin).

Parameter	Type	Default	Description
backend	str	"longterm"	Which storage backend the transaction applies to: "longterm" or "session".
collection	Optional[str]	None	Vector collection; falls back to config.defaultCollection.
sessionId	Optional[str]	None	Session the transaction is scoped to, when relevant.

• Returns: VmiTransaction — see Data models. Pass it as the transaction= argument to any other SDK function.
• Raises: VmiConfigError if vectorDbUrl isn't set; TransactionError if the orchestrator rejects the request or returns no tx_id.

tx = client.beginTransaction()

commitTransaction

VmiClient.commitTransaction(transaction: VmiTransaction) -> dict[str, Any]

Commits all writes staged under transaction (POST /api/v1/tx/{txId}/commit).

• Parameters: transaction (VmiTransaction) — the transaction to commit.
• Returns: the raw JSON response from the commit endpoint.
• Raises: TransactionError if the commit fails.

rollbackTransaction

VmiClient.rollbackTransaction(transaction: VmiTransaction) -> dict[str, Any]

Discards all writes staged under transaction (POST /api/v1/tx/{txId}/rollback).

• Parameters: transaction (VmiTransaction) — the transaction to roll back.
• Returns: the raw JSON response from the rollback endpoint.
• Raises: TransactionError if the rollback fails.

transactionStatus

VmiClient.transactionStatus(transaction: VmiTransaction) -> dict[str, Any]

Reads the current status of a transaction (GET /api/v1/tx/{txId}/status) — e.g. whether it's open, committed or rolled back.

• Parameters: transaction (VmiTransaction).
• Returns: the raw JSON status body.

transaction (context manager)

with VmiClient.transaction(
    backend: str = "longterm",
    collection: Optional[str] = None,
    sessionId: Optional[str] = None,
) as tx:
    ...

Convenience wrapper around beginTransaction/commitTransaction/ rollbackTransaction: begins a transaction, yields it, commits on a clean exit, and rolls back automatically (re-raising the original exception) if the with block raises.

Parameter	Type	Default	Description
backend	str	"longterm"	Same as beginTransaction.
collection	Optional[str]	None	Same as beginTransaction.
sessionId	Optional[str]	None	Same as beginTransaction.

• Yields: VmiTransaction.

with client.transaction() as tx:                 # commit on exit, rollback on error
    client.addMemory("staged fact", transaction=tx)
    client.callLLM(rawPrompt="...", provider="OPENAI", model="gpt-4o",
                   saveInMemory=True, transaction=tx)

# equivalent, done manually:
tx = client.beginTransaction()
client.transactionStatus(tx)
client.commitTransaction(tx)      # or client.rollbackTransaction(tx)

Session functions

createSession

VmiClient.createSession(sessionId: str) -> None

Creates a session on vmi-orchestrator (POST /api/v1/session/create).

Parameter	Type	Description
sessionId	str	The id to create.

• Returns: None.
• ⚠️ Not idempotent: creating a session that already exists replaces it and wipes its stored vectors. Call this once per genuinely new session — writes and searches with storage="session" auto-create a missing session on demand, so calling createSession up front is optional and only useful when you want to guarantee a clean slate.

client.createSession("sess_001")

closeSession

VmiClient.closeSession(sessionId: str, flush: bool = True) -> dict[str, Any]

Closes a session (POST /api/v1/session/close).

Parameter	Type	Default	Description
sessionId	str	—	The session to close.
flush	bool	True	When True, flushes the session's memory into longterm storage before closing so it isn't lost.

• Returns: the raw JSON response from the close endpoint.

client.fetchMemory("...", storage="session", sessionId="sess_001")
client.closeSession("sess_001", flush=True)   # flush session memory to longterm

---

Data models

All models live in llm_sdk/models.py and are frozen dataclasses (immutable — set once at construction, never mutated afterward) unless noted.

MemorySource (str Enum)

Identifies where memory came from. Values: SESSION = "session", LONG_TERM = "longterm", CONNECTED_DB = "connected_db".

VmiTransaction

An open vector-DB transaction handle. Pass it as transaction= to fetchMemory/addMemory/callLLM/etc. so every child API call carries its tx_id and is staged rather than applied immediately.

Field	Type	Default	Description
txId	str	—	The transaction id returned by the orchestrator.
backend	str	"longterm"	"longterm" or "session".
collection	str	"vectors"	The vector collection the transaction applies to.
sessionId	Optional[str]	None	Associated session, if any.

PromptDefinition

A prompt fetched from vmi-backend.

Field	Type	Description
visibleId	str	The prompt's public id.
title	str	Human-readable title.
description	str	Human-readable description.
promptContent	str	The template text, containing {{placeholder}} variables.
aiVendorCode	str	Default vendor for this prompt: OPENAI | CLAUDE | GEMINI | DEEPSEEK.
aiVendorVersionCode	str	Default model code, e.g. "gpt-4o".
isLive	bool	Whether the prompt is currently active/published in vmi-backend.

DataNode

One individual hit from a vector search.

Field	Type	Default	Description
id	str	—	The vector's id.
text	str	—	The extracted text content of the hit (prefers chunkBody for code chunks, then text/content/metadata.text).
score	Optional[float]	None	Similarity score, when returned by the orchestrator.
metadata	dict[str, Any]	{}	The full metadata stored alongside the vector.

MemoryData

Result of fetchMemory.

Field	Type	Default	Description
query	str	—	The original query text.
memoryText	str	—	A numbered, plain-text block ("[1] ...\n[2] ...") ready to paste directly into an LLM prompt as context.
dataNodes	list[DataNode]	[]	The individual hits behind memoryText.
summary	Optional[str]	None	Server-side (orchestrator) summary of the hits, present when summarize=True succeeded.
raw	dict[str, Any]	{}	The full unprocessed response from the orchestrator.

ConnectedDBContext

Result of fetchMemoryFromConnectedDB.

Field	Type	Default	Description
promptText	str	—	The text the search was run for.
consolidatedSummary	str	—	The main, human-readable summary of what was found — this is what you typically inject into a prompt.
entityResults	list[dict[str, Any]]	[]	Per-entity raw result rows.
totalFactsFound	int	0	Total number of facts retrieved across all searched databases.
databasesSearched	int	0	How many connected databases were queried.
purpose	Optional[str]	None	Echoes the purpose argument, if the backend returned one.
raw	dict[str, Any]	{}	The full unprocessed response data.

LLMCallResult

Result of callLLM.

Field	Type	Default	Description
output	str	—	The model's main response text (extracted from the {"output", "summary"} JSON when saveInMemory=True, otherwise the raw response text).
summary	Optional[str]	—	The 2-3 sentence memory summary, present only when saveInMemory=True and the model complied with the requested JSON format.
provider	str	—	The resolved vendor code actually used (OPENAI | CLAUDE | GEMINI | DEEPSEEK).
model	str	—	The resolved model/version code actually used.
inputTokens	int	—	Prompt/input tokens consumed, per the provider's usage report.
outputTokens	int	—	Completion/output tokens consumed.
durationMs	int	—	Wall-clock time the provider call took, in milliseconds.
callCode	Optional[str]	None	vmi-backend's call-tracking code (only set when a promptId was used and call logging succeeded).
systemPrompt	str	""	The exact system prompt sent to the model (background + summarized memory + DB context [+ JSON-format instructions]) — useful for debugging/inspection.
memoryText	str	""	The raw (pre-summarization) memory text that was fetched, when passMemory=True.
connectedDBSummary	str	""	The connected-DB consolidated summary that was injected, when connectedDB=True.
raw	dict[str, Any]	{}	The full raw response object from the underlying provider SDK call.

Property:

• LLMCallResult.totalTokens -> int — computed as inputTokens + outputTokens.

ProviderResult

The contract every provider client (OpenAIProvider, AnthropicProvider, GeminiProvider) must return from call(...).

Field	Type	Default	Description
text	str	—	The raw text response from the model.
inputTokens	int	—	Input/prompt tokens used.
outputTokens	int	—	Output/completion tokens used.
raw	dict[str, Any]	{}	The vendor SDK's raw response object (or a compact dict for vendors like Gemini that don't serialize cleanly).

---

Exceptions

All in llm_sdk/exceptions.py. Every exception the SDK raises derives from VmiError, so except VmiError: catches everything the SDK can throw.

Exception	Extends	Extra attributes	Raised when
VmiError	Exception	—	Base class; never raised directly.
VmiConfigError	VmiError	—	The SDK is misconfigured: a required token/key is missing (e.g. calling fetchMemoryFromConnectedDB without accessToken/jwtToken, or callLLM with both/neither of promptId/rawPrompt). Not raised for the URLs — those are hardcoded and always present.
VmiApiError	VmiError	statusCode: int | None, responseBody: str | None	A vmi-backend or vmi-orchestrator HTTP call failed (non-2xx status after retries, or a network error).
PromptNotFoundError	VmiApiError	(inherited)	fetchPrompt/callLLM(promptId=...) referenced a promptId that vmi-backend doesn't recognize (HTTP 404, or an empty data payload).
ProviderError	VmiError	provider: str | None, errorCode: str | None	The underlying LLM vendor call failed — bad/missing API key, vendor API error, unknown aiVendorCode, or a required provider package (anthropic, google-generativeai) isn't installed.
TransactionError	VmiApiError	(inherited)	beginTransaction/commitTransaction/rollbackTransaction failed, or tx/begin returned no tx_id.

from llm_sdk import VmiError, VmiConfigError, ProviderError

try:
    client.callLLM(rawPrompt="Hi", provider="OPENAI", model="gpt-4o")
except ProviderError as e:
    print(e.provider, e.errorCode, e)
except VmiError as e:
    print("SDK error:", e)

---

Internals reference

You normally interact only with VmiClient / the module-level functions above. This section documents the internal service and client classes they delegate to, for anyone extending the SDK, debugging a call, or writing tests against a lower layer directly.

MemoryService (llm_sdk/memory_service.py)

Implements the memory-related business logic on top of VectorDBClient and BackendClient. VmiClient.fetchMemory/addMemory/fetchMemoryFromConnectedDB delegate straight to this class — its methods and parameters are exactly those documented under Memory functions above, constructed as:

MemoryService(vectorClient: VectorDBClient, backendClient: Optional[BackendClient])

• fetchMemory(...) -> MemoryData — runs VectorDBClient.search(...), converts the raw results list into MemoryData.memoryText (via VectorDBClient.toMemoryText) and MemoryData.dataNodes (via VectorDBClient.toDataNodes), and retries once without summarize on a server-side summarizer failure.
• addMemory(...) -> dict — thin pass-through to VectorDBClient.write(...).
• fetchMemoryFromConnectedDB(...) -> ConnectedDBContext — pass-through to BackendClient.extractConnectedDBContext(...); raises VmiConfigError if no backendClient was configured (i.e. VmiConfig.backendUrl is unset).

LLMService (llm_sdk/llm_service.py)

Implements callLLM's orchestration — a faithful Python port of vmi-backend's CompanyProjectStepService.createStep. Constructed as:

LLMService(config, providerRegistry, memoryService, backendClient, promptCache)

Public methods:

• callLLM(**kwargs) -> LLMCallResult — documented in full under VmiClient.callLLM. Internally it runs, in order:
  1. Resolve the prompt. promptId → _getPromptDefinition (cached) → resolvePrompt substitutes inputParams into {{placeholders}}, and vendor/model default to the definition unless overridden. rawPrompt → resolvePrompt directly, vendor/model must be supplied explicitly.
  2. Memory context (passMemory=True). Skipped entirely on the first message of a brand-new session (storage="session", tracked per client instance) — mirrors createProjectStep's previousStepCount check. Otherwise calls MemoryService.fetchMemory, then prefers the orchestrator's own summary, falling back to _summarizeMemory (a local call to the fixed summarizerModel, default gpt-4o-mini) when no summary was returned. Any exception here is caught and logged — memory context is optional, never fatal.
  3. Connected-DB context (connectedDB=True). Calls MemoryService.fetchMemoryFromConnectedDB; exceptions are caught and logged, non-fatal.
  4. Background/purpose grounding (passBackground). Uses the string directly if one was passed, otherwise config.background.
  5. Assemble the system prompt via _buildSystemPrompt (see below).
  6. Initiate call logging (BackendClient.initiateCall) — only when a promptId was used and a backend client exists; never blocks the LLM call even on failure (returns None instead of raising).
  7. Call the provider (ProviderRegistry.getProvider(vendorCode).call(...)), timing the call. A failure here is propagated (this is the one step allowed to fail the whole request) — but the failure is still logged to vmi-backend first (completeCall(status="ERROR", ...)) before re-raising.
  8. Parse the output via _parseOutput — extracts {"output","summary"} JSON when saveInMemory=True was requested; falls back to treating the whole response as plain output text if the model didn't comply.
  9. Complete call logging (BackendClient.completeCall) with token counts, duration, status and the summary.
  10. Write the summary back to memory (saveInMemory=True and a summary was extracted) via MemoryService.addMemory, tagged with metadata={"type": "summary", "promptId": ..., "callCode": ...}. Failure here is caught and logged, non-fatal.
• getPrompt(promptId: str) -> PromptDefinition — same as VmiClient.fetchPrompt; delegates to the internal cached loader.

Private helpers (useful to understand when debugging a callLLM call):

• _getPromptDefinition(promptId) -> PromptDefinition — reads through the PromptCache; raises VmiConfigError if no cache/backend was configured.

• _summarizeMemory(memoryText: str) -> str — sends memoryText to config.summarizerModel (via a throwaway OpenAIProvider) with a fixed system prompt asking for a 3-5 sentence plain-text condensation. temperature=0.3, maxTokens=512. Returns "" on empty input or on any failure (never blocks the main call).

• _buildSystemPrompt(background, summarizedMemory, connectedDBSummary, requireSummary) -> str (static) — assembles the final system prompt text in this order:

  1. Role line — "You are an AI assistant. Your purpose: {background}" or the generic "You are an AI assistant." if no background was given.
  2. "Relevant context from previous interactions:\n{summarizedMemory}" (if non-empty).
  3. "Relevant data from connected databases:\n{connectedDBSummary}" (if non-empty).
  4. The strict {"output": "...", "summary": "..."} JSON-format instruction (only when requireSummary=True, i.e. saveInMemory=True).

  Note: the user's actual message is deliberately not included here — it's passed separately as the provider's userMessage, keeping a clean role/context/user split.

• _parseOutput(text, expectSummary) -> tuple[str, Optional[str]] (static) — if expectSummary=False, returns (text, None) unchanged. Otherwise searches text for a {...} JSON block (via regex), parses it, and returns (parsed["output"], parsed.get("summary")) when valid; degrades to (text, None) with a warning log if parsing fails or the expected keys are missing.

VectorDBClient (llm_sdk/vector_client.py)

The low-level HTTP client for vmi-orchestrator. A Python port of vmi-backend's llmutils/VectorDBClient, extended with transaction and session endpoints. Constructed as VectorDBClient(config: VmiConfig).

• health() -> dict — GET /health (unauthenticated, outside /api/v1).
• createSession(sessionId: str) -> None — POST /api/v1/session/create. Not idempotent — see the createSession warning above.
• closeSession(sessionId: str, flush: bool = True) -> dict — POST /api/v1/session/close.
• beginTransaction(backend="longterm", collection=None, sessionId=None) -> VmiTransaction — POST /api/v1/tx/begin; raises TransactionError on failure or a missing tx_id in the response.
• commitTransaction(transaction) -> dict — POST /api/v1/tx/{txId}/commit.
• rollbackTransaction(transaction) -> dict — POST /api/v1/tx/{txId}/rollback.
• transactionStatus(transaction) -> dict — GET /api/v1/tx/{txId}/status.
• listOpenTransactions() -> dict — GET /api/v1/tx/open; lists all currently-open transactions (not exposed on VmiClient — call it directly on client._vectorClient if needed, or via your own VectorDBClient).
• search(queryText, storage="longterm", sessionId=None, collection=None, topK=None, filters=None, tags=None, summarize=True, indexId=None, companyProjectCode=None, rerank=False, rerankTopN=None, rerankMode=None, contentType=None, transaction=None) -> dict — POST /api/v1/vectors/search. Builds the request body from all the optional filters/tags/rerank/contentType parameters, includes tx_id when a transaction is given, and is read-safe to retry on transient 500s unless summarize=True (a deterministic summarizer failure would just delay MemoryService's fallback). Returns {"results": [...], "count": n, "summary"?: str}.
• write(text, vectorId=None, metadata=None, tags=None, storage="longterm", sessionId=None, collection=None, indexId=None, isTransactional=False, transaction=None, update=False) -> dict — POST /api/v1/vectors/write, or PUT /api/v1/vectors/update when update=True. Auto-generates vector_id as mem_<uuid4> if not given. A transaction implies is_transactional=True.
• batchWrite(items, storage="longterm", sessionId=None, collection=None, isTransactional=False, transaction=None) -> dict — POST /api/v1/vectors/batch. See addMemories above for the item shape and the metadata["text"] backfill behavior.
• delete(vectorId, storage="longterm", sessionId=None, collection=None, isTransactional=False, transaction=None) -> dict — DELETE /api/v1/vectors/delete.
• fetch(vectorId, storage="longterm", collection=None, sessionId=None) -> dict — GET /api/v1/vectors/{vectorId}.
• toDataNodes(results: list[dict]) -> list[DataNode] (static) — converts raw orchestrator result rows into DataNode objects.
• toMemoryText(results: list[dict]) -> str (static) — converts raw result rows into the numbered plain-text block used as MemoryData.memoryText. Keeps only the best-scoring chunk per artifactId (results arrive sorted by score) so multi-chunk hits from the same source file/document don't flood the context.
• _extractText(row: dict) -> str (static, internal) — for chunk results (metadata.isChunk is True), prefers metadata.chunkBody (the full code/text) over embed_text (signature + docstring only); otherwise falls back to row["text"], row["content"], or metadata["text"], in that order.
• _sessionExists(sessionId, collection="vectors") -> bool (internal) — non-destructive probe: fetches a made-up vector id scoped to the session; a 404 means the session exists (the vector just doesn't), a 500 means the session itself is missing. There's no dedicated session-status endpoint, so this is the only safe way to check.
• _callWithSessionRecovery(call, storage, sessionId) (internal) — wraps a vector operation; on a ≥500 error against storage="session", it checks _sessionExists and — only if the session is genuinely missing/expired — creates it and retries the call once. This is the only point where the SDK auto-creates a session, precisely because createSession up front would risk wiping an existing one.

BackendClient (llm_sdk/backend_client.py)

The low-level HTTP client for vmi-backend. Supports two auth modes sent together whenever both are configured: X-Access-Token for public endpoints, and Authorization: Bearer <jwt> + X-Tenant-ID for JWT-protected ones. Constructed as BackendClient(config: VmiConfig). Responses are all wrapped server-side as {"success": bool, "message": str, "data": ...}; _data(response, context) unwraps this and raises VmiApiError when success is falsy.

• getPrompt(promptId: str) -> PromptDefinition — GET /api/v1/public/ai-prompts/{promptId}. Raises PromptNotFoundError on HTTP 404 or an empty data payload.
• initiateCall(aiPromptCode, promptInText, aiVendorCode, aiVendorVersionCode, inputObjectJson=None, txId=None) -> Optional[str] — POST /api/v1/public/ai-prompt-calls/initiate, phase 1 of call logging. Returns the callCode to pass to completeCall, or None if the request fails — never raises, since call logging must never block the actual LLM call.
• completeCall(callCode, status, promptOutText, totalInputTokens, totalOutputTokens, durationInMilliseconds, errorMessage=None, errorCode=None, memoryData=None, txId=None) -> None — POST /api/v1/public/ai-prompt-calls/complete, phase 2. Also non-fatal: logs a warning on failure instead of raising.
• extractConnectedDBContext(promptText, connectedDatabaseCodes=None, entityIdentifier=None, topK=5, purpose=None, identifierText=None, txId=None) -> ConnectedDBContext — posts a PromptContextRequest to /api/v1/database-knowledge/extract-open (when config.accessToken is set) or /extract (JWT auth otherwise).
• pushMemoryData(dataType, jsonData, searchableTag=None, uniqueId=None, isForTesting=False) -> dict — POST /api/v1/public/memory-data. Pushes arbitrary tagged JSON data (not vector-embedded) for later retrieval by tag. Not wrapped by VmiClient — call it via your own BackendClient instance, or reach into client._backendClient.pushMemoryData(...) if needed.

HttpClient (llm_sdk/http_client.py)

Thin requests-based HTTP layer shared by BackendClient and VectorDBClient. Centralizes timeouts, retries and error mapping. Constructed as:

HttpClient(baseUrl, headers=None, connectionTimeout=10.0, readTimeout=60.0, maxRetries=2)

• request(method, path, json=None, params=None, headers=None, retryableStatuses=None) -> dict — issues one HTTP request, retrying up to maxRetries times on network errors or a response status in retryableStatuses | {502, 503, 504} (exponential backoff, capped at 4 seconds). Raises VmiApiError (with statusCode/responseBody set) on a non-2xx response after retries are exhausted, or after all retry attempts hit a network error. Returns {} for an empty body, or {"raw": <text>} if the body isn't valid JSON.
• get(path, **kwargs) / post(path, **kwargs) / put(path, **kwargs) / delete(path, **kwargs) — convenience wrappers around request() with the method fixed.
• Tracing: set logging.getLogger("llm_sdk.api").setLevel(logging.INFO) to log every request/response (bodies compact-truncated at 600 chars, so embeddings and long texts don't flood your logs).

PromptCache (llm_sdk/prompt_cache.py)

A small, thread-safe TTL cache for PromptDefinitions, replacing an older socket-push mechanism — prompts are pulled on first use and re-fetched after ttlSeconds.

PromptCache(loader: Callable[[str], PromptDefinition], ttlSeconds: float = 300.0)

• get(promptId: str) -> PromptDefinition — returns the cached definition if it's younger than ttlSeconds; otherwise calls loader(promptId), caches the result with the current timestamp, and returns it.
• invalidate(promptId: Optional[str] = None) -> None — removes one cached entry, or clears the whole cache if promptId is None.

resolvePrompt (llm_sdk/prompt_resolver.py)

resolvePrompt(promptContent: str, inputParams: dict[str, Any] | None = None) -> str

Substitutes {{name}} placeholders in promptContent with values from inputParams.

Parameter	Type	Description
promptContent	str	The template text, e.g. "Answer this: {{question}}".
inputParams	Optional[dict[str, Any]]	Values to substitute. Non-string values (numbers, lists, dicts, booleans) are JSON-encoded before substitution; string values are inserted verbatim.

• Returns: the template with all known placeholders replaced. Placeholders whose key isn't present in inputParams are left untouched ({{unknownVar}} stays as-is), so partially-resolved prompts remain visible in call logs for debugging rather than silently vanishing.

resolvePrompt("Hello {{name}}, your order {{orderId}} is {{status}}.",
              {"name": "Asha", "orderId": 4821, "status": "shipped"})
# "Hello Asha, your order 4821 is shipped."

Providers (llm_sdk/providers/)

ProviderClient (abstract base, providers/base.py)

The contract every vendor implementation follows. Providers do model invocation only — no memory, telemetry, or logging; those concerns live entirely in LLMService.

class ProviderClient(ABC):
    def __init__(self, apiKey: str): ...

    @abstractmethod
    def call(self, model: str, systemPrompt: str, userMessage: str,
              temperature: float = 0.7, maxTokens: int = 2048) -> ProviderResult: ...

Parameter	Type	Default	Description
model	str	—	Vendor-specific model id, e.g. "gpt-4o", "claude-sonnet-5", "gemini-1.5-pro".
systemPrompt	str	—	The full system/instruction text.
userMessage	str	—	The user's message/prompt.
temperature	float	0.7	Sampling temperature.
maxTokens	int	2048	Max tokens to generate.

• Returns: ProviderResult.
• _requireApiKey(providerName: str) -> str (protected helper) — returns the configured API key, or raises ProviderError if it's empty.

OpenAIProvider (providers/openai_provider.py)

OpenAIProvider(apiKey: str, baseUrl: str | None = None, name: str = "OPENAI")

Calls the OpenAI Chat Completions API. Also reused for any OpenAI-compatible vendor by passing a different baseUrl/name — this is how DeepSeek support works (baseUrl="https://api.deepseek.com").

• call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult — sends a two-message chat completion (system + user). Handles reasoning-style models (gpt-5*, o1/o3/o4...) that reject max_tokens and/or non-default temperature: on the corresponding BadRequestError, it automatically swaps max_tokens → max_completion_tokens, or drops temperature, and retries (up to 3 attempts total) — there's no reliable way to detect this by model name alone across OpenAI-compatible vendors. Raises ProviderError on any other failure.

AnthropicProvider (providers/anthropic_provider.py)

AnthropicProvider(apiKey: str)

Calls the Anthropic Messages API.

• call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult — sends system as a top-level parameter and the user message as the single message in messages. Concatenates all text-type content blocks in the response into ProviderResult.text. Raises ProviderError if the anthropic package isn't installed, or if the API call fails.

GeminiProvider (providers/gemini_provider.py)

GeminiProvider(apiKey: str)

Calls Google's Gemini API via google-generativeai.

• call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult — configures the client with system_instruction=systemPrompt and generation_config={"temperature", "max_output_tokens"}, then generates content from userMessage. Raises ProviderError if google-generativeai isn't installed, or if the API call fails.

ProviderRegistry (providers/registry.py)

ProviderRegistry(config: VmiConfig)

Maps vmi-backend's aiVendorCode values to the right provider client, with per-vendor instance caching.

• getProvider(aiVendorCode: str) -> ProviderClient — case-insensitive lookup. OPENAI → OpenAIProvider; CLAUDE or ANTHROPIC → AnthropicProvider; GEMINI or GOOGLE → GeminiProvider; DEEPSEEK → OpenAIProvider pointed at https://api.deepseek.com. Each provider instance is created once per vendor code and reused for subsequent calls. Raises ProviderError for any other code, listing the supported ones.

---

Transactions

Every SDK function that touches the vector DB accepts a transaction= argument. When set, the transaction's tx_id is included in the API hits of all child calls (search, write, extract, initiate, complete), so a whole callLLM — including any memory writes it performs — can be staged and committed or rolled back as a single atomic unit.

with client.transaction() as tx:                 # commit on exit, rollback on error
    client.addMemory("staged fact", transaction=tx)
    client.callLLM(rawPrompt="...", provider="OPENAI", model="gpt-4o",
                   saveInMemory=True, transaction=tx)

# or manually:
tx = client.beginTransaction()
client.transactionStatus(tx)
client.commitTransaction(tx)      # / client.rollbackTransaction(tx)

See Transaction functions for the full parameter reference of each method involved.

Sessions

client.createSession("sess_001")
client.fetchMemory("...", storage="session", sessionId="sess_001")
client.closeSession("sess_001", flush=True)   # flush session memory to longterm

See Session functions for full parameter details and the important non-idempotency warning on createSession.

---

Architecture

llm_sdk/
├── client.py           VmiClient facade (entry point)
├── config.py            VmiConfig
├── models.py             typed results (MemoryData, LLMCallResult, VmiTransaction, ...)
├── exceptions.py       VmiError hierarchy
├── http_client.py      retries / timeouts / error mapping
├── backend_client.py   vmi-backend API client
├── vector_client.py    vmi-orchestrator API client (port of Java VectorDBClient)
├── memory_service.py   fetchMemory / addMemory / fetchMemoryFromConnectedDB
├── llm_service.py      callLLM orchestration (port of createProjectStep)
├── prompt_cache.py     TTL cache for prompt definitions
├── prompt_resolver.py  {{variable}} substitution
└── providers/          OpenAI, Anthropic, Gemini, DeepSeek (+ registry)

Design rules (unchanged from v1):

• Providers do model invocation only — no memory, telemetry or logging inside them.
• Non-fatal periphery — memory fetch, summarization, connected-DB context and call logging degrade gracefully; only the LLM call itself can fail the request.
• Immutability — all result models are frozen dataclasses.
• Provider abstraction — vendor codes match vmi-backend's LLMClientFactory (OPENAI, CLAUDE, GEMINI, DEEPSEEK).

---

Test bench (web UI)

python -m sdk_test_app.cli          # opens http://127.0.0.1:7799

A browser console with four scenarios — Coding Assistant, Personal Educator, Chatbot, Enquiry System — each a memory-backed chat with its own background and storage mode. For every message the inspector shows the whole pipeline: what was retrieved from memory, the exact system prompt, the model output, the summary pushed back to memory, connected-DB context, and the raw API wire trace. A Function lab exposes every SDK function individually (memory CRUD, connected DB, backend prompts, transactions, sessions), a coverage board tracks which functions have passed/failed, and Run full test exercises all of them in order with a pass/fail report.

Configuration via sdk_test_app/.env. Note there's no backend/vector-DB URL variable — those are hardcoded in llm_sdk/config.py, not read from .env:

VMI_ACCESS_TOKEN=...
VMI_VECTOR_TOKEN=ABCDEFG1234
VMI_COMPANY_CODE=WPCORP4812
VMI_BACKGROUND=You are a helpful assistant.
OPENAI_API_KEY=sk-...

Tests

pip install pytest
python -m pytest tests/ -q