langchain-llama-server

This LangChain integration supports llama-server for remote inference over HTTP/S.

Normally you can use ChatOpenAI from langchain-openai for OpenAI compatible APIs. But, anything beyond the OpenAI spec is not supported. Hence why this integration exists.

This package was created to fill in the gaps between the OpenAI spec and extra fields llama-server returns:

Access reasoning_content on both streaming and non-streaming chat completions.
Capture timings, __verbose and other llama-server specific response fields.
Uses BaseChatOpenAI from langchain-openai to implement a chat_model called ChatLlamaServer

Similar to other server integrations:

ChatOllama
ChatXAI
- ChatXAI also works with llama-server for reasoning_content because they both implement the OpenAI spec closely
ChatOpenRouter

TODO

LlamaServerLLM ?
LlamaServerEmbedding ?
/slots endpoint -- introspect concurrent-request/context-slot usage
n_probs/logprobs support

What about ChatLlamaCpp?

The langchain_community package provides ChatLlamaCpp? This chat model targets in-process hosting using llama.cpp. It doesn't support llama-server for remote hosting.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

langchain_llama_server-0.2.0.tar.gz (136.7 kB view details)

Uploaded Jul 27, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

langchain_llama_server-0.2.0-py3-none-any.whl (5.5 kB view details)

Uploaded Jul 27, 2026 Python 3

File details

Details for the file langchain_llama_server-0.2.0.tar.gz.

File metadata

Download URL: langchain_llama_server-0.2.0.tar.gz
Upload date: Jul 27, 2026
Size: 136.7 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: uv/0.11.32 {"installer":{"name":"uv","version":"0.11.32","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for langchain_llama_server-0.2.0.tar.gz
Algorithm	Hash digest
SHA256	`5856b3591db43bf4c7186add8fb37a18d3b8d93127811f1a500739a355508aa4`
MD5	`dfa3d9c7ced325b2d51635453aa3d6ec`
BLAKE2b-256	`ac1ac09e77e8ed3e8ee08ed84fe96aae3662745a01a75f5b9b61ee9358948260`

See more details on using hashes here.

File details

Details for the file langchain_llama_server-0.2.0-py3-none-any.whl.

File metadata

Download URL: langchain_llama_server-0.2.0-py3-none-any.whl
Upload date: Jul 27, 2026
Size: 5.5 kB
Tags: Python 3
Uploaded using Trusted Publishing? Yes
Uploaded via: uv/0.11.32 {"installer":{"name":"uv","version":"0.11.32","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for langchain_llama_server-0.2.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`7a3fbc6578a654a2e531109b36ce1f8cd303e4a845828f2464f8f1537b0faf6b`
MD5	`76aa53ae82a1e6c93dd0f3c0fae967fb`
BLAKE2b-256	`767819b6999c9803d3387868ee0cf518ace27bd3ed09ad6dc9805aefcd21e821`