langchaint
langchaint is a provider-neutral Python client for LLM applications. It provides fully typed, asynchronous APIs for generation, streaming, embeddings, tools, retries, and billing. The application owns the agent loop.
Alpha: the API may change without notice.
Why langchaint
- Consistent API. Bind request fields once with
LLM.bind(), then callgenerate_one(),generate_many(), orstream_one()on the resultingBoundLLM. - Useful type annotations. For example,
llm.bind(response_format=Answer)typesgeneration.outputasAnswer. - Outcome variants with autocomplete. Match on
.kindwith editor autocomplete and no class imports. - Coordinated retries. Share concurrency limits, request-start pacing, and provider-directed pauses across models using one rate-limit quota.
- Complete billing.
GenerationandGenerationErrorvalues retain provider-reported usage from every recorded request, including billed retries. - Streaming.
stream_one()returns an async context manager and async iterator.final()returns the typed generation with its usage. - Agent loops in Python. Provider-neutral messages, typed tools with argument validation, concurrent dispatch, and explicit outcome variants support async control flow.
Install
langchaint requires Python 3.13 or newer.
Install the extra for each backend you use:
pip install "langchaint[openai]"
| Backend | Class | Install |
|---|---|---|
| Anthropic | Anthropic |
langchaint[anthropic] |
| Anthropic on Amazon Bedrock | AnthropicBedrock |
langchaint[anthropic-bedrock] |
| Cohere embeddings on Amazon Bedrock | CohereBedrock |
langchaint[cohere-bedrock] |
| DeepSeek | DeepSeek |
langchaint[deepseek] |
| Gemini | Gemini |
langchaint[gemini] |
| OpenAI | OpenAI |
langchaint[openai] |
| OpenAI embeddings | OpenAI |
langchaint[openai-embedding] |
| OpenAI on Amazon Bedrock | OpenAIBedrock |
langchaint[openai-bedrock] |
Install langchaint[tracing] for OpenTelemetry tracing.
Generate a typed output
import asyncio
from pydantic import BaseModel
from langchaint.openai import OpenAI
class Answer(BaseModel):
answer: str
confidence: float
async def main() -> None:
assistant = (
OpenAI()
.llm("gpt-5.6-terra")
.bind(
system_prompt="Answer clearly and concisely.",
response_format=Answer,
)
)
generation = await assistant.generate_one("Why is the sky blue?")
print(generation.output.answer)
print(generation.usage.cost_in_usd)
asyncio.run(main())
The Pydantic model validates the provider response.
generate_many() returns one outcome per input in input order.
A terminal failure becomes that input's GenerationError, so sibling outcomes remain available.
Coordinate retries across a rate-limit quota
Create one SharedBackoff for each rate-limit quota, and pass it to every backend that sends requests against that quota. Models from one OpenAI share its SharedBackoff:
from langchaint import SharedBackoff
openai = OpenAI(
shared_backoff=SharedBackoff(max_concurrent_requests=8, max_request_starts_per_second=50.0),
)
fast_model = openai.llm("gpt-5.6-luna")
strong_model = openai.llm("gpt-5.6-sol")
A rate-limit response pauses request starts across the shared quota. After a transient failure local to one request, langchaint waits and retries that request.
Stream with an explicit lifetime
text_assistant = OpenAI().llm("gpt-5.6-terra").bind()
async with text_assistant.stream_one("Explain photosynthesis.") as stream:
async for item in stream:
if isinstance(item, str):
print(item, end="", flush=True)
generation = await stream.final()
final() consumes the remaining stream and returns the assembled generation.
Build agent loops
The application controls turn limits, state, approvals, model changes, and persistence.
messages: list[Message] = [UserMessage(content=prompt)]
for _ in range(max_turns):
generation = await bound.generate_one(messages)
match generation.kind:
case "tool_call":
messages.append(generation.assistant_message)
outcomes = await bound.tool_manager.dispatch_many(generation.tool_calls)
messages.extend(outcome.tool_message for outcome in outcomes)
case "plain":
return generation.output
raise RuntimeError("model did not finish within max_turns")
ToolManager.dispatch_many() runs tool calls concurrently and preserves their order.
See examples/02_tool_loop.py for a complete typed tool loop.
Account for every request
generation.usage.cost_in_usd includes every billed retry recorded for the input.
GenerationError.usage preserves the recorded cost of a failed input.
More examples
See examples/README.md for complete examples.
License
1 maintainer
Metadata
1 maintainer
Release files for langchaint 0.26.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| langchaint-0.26.0.tar.gz | 175.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| langchaint-0.26.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 373.5 kB
Release files / langchaint-0.26.0.tar.gz
| Download URL | langchaint-0.26.0.tar.gz |
|---|---|
| Size | 175.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0538510f9c6635ff31f598c8d8d41d728a82b83789850ec16d131425bd5732d2
|
|
BLAKE2b-256 checksum How to use checksums |
f1097ae399be9e88093c58c62b1657a16ce90a8053968cc44a1eea7567ea09ff
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / langchaint-0.26.0-py3-none-any.whl
| Download URL | langchaint-0.26.0-py3-none-any.whl |
|---|---|
| Size | 197.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8a3b3b4d09972b41a643340b6bdd854937578752758ce2a9dfedad54e4089843
|
|
BLAKE2b-256 checksum How to use checksums |
8ec0f18b1707599840c020d9eb9c37635f7f5e454bc97eed7f61f46c4df9f10a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
