llcat 0.16.6

/usr/bin/cat for LLMs

Ever need to test if an inference endpoint is working or want to one-shot a model on a server?

Maybe you want to cycle through keys or models or benchmark a bank of IPs. Perhaps you want to orchestrate N queries across M models running on P servers and want to run the job in parallel without leaving any leaky state behind.

Existing tools require you to pick from a provider boutique and a small list of models shipped with the software then swap around credentials like you're Indiana Jones with a bag of sand.

llcat is a solution to these problems: a general-purpose CLI-based OpenAI-compatible /chat/completions caller (and also works with Ollama, OpenRouter, sglang, llama.cpp and more). It has a rich syntax and supports a sophisticated set of features while keeping simple things easy.

Example: Model List

Here's a screenshot from adding a custom model provider to goose. It asks you to manually supply the models as a comma separate list. What a pain!

Not a pain with llcat! It's easy!

Copy and paste this. Go ahead!

uvx llcat api.concentrate.ai -m | tr '\n' ','

Think of llcat like cURL or cat for LLMs: a stateless, transparent, explicit, low-level, composable tool for scripting and glue.

Conversations, which are regular JSON files, keys, servers and other configurations are explicitly specified each execution as command line arguments. There's a --curlify option as well.

This makes building things with llcat direct.

For instance, let's say I have a list of authentication tokens in some file, credentials.txt:

sk-or-v1-e1e5...
sk-or-v1-ej24...
sk-or-v1-ff24...

Here's how you do that with llcat:

Method 1:

llcat -k @credentials.txt:0
llcat -k @credentials.txt:1
llcat -k @credentials.txt:2

Method 2:

llcat -k sk-or-v1-e1e5...
llcat -k sk-or-v1-ej24...
llcat -k sk-or-v1-ff24...

You can do the same pattern with models, system prompts, queries, and servers. For instance:

llcat -k "@~/credentials.txt:12" \
      -u "@settings.json:.[3].host" \
      -s "@system_prompts:8" \
      -m "@settings.json:.[3].model" \
         "@query.txt:12" > output.txt

Wait wait wait, is that jq?

Yes! You can use normal strings (ex: "abc"), files (ex: @abc.txt) with line numbers (ex: @abc.txt:1) and even jq syntax (ex: @abc.json:.server[0].url). This makes parallel distributed execution painless.

Here's a pattern you might particularly like:

llcat -k @~/secrets.json:.openrouter

llcat is part of the DAY50 suite of open-source tools built for a future where AI workloads are split across devices, private servers, and cloud APIs prioritizing predictability, compatibility, coherency, transparency and functionality.

Very Quick Start

List the models on OpenRouter:

uvx llcat openrouter.ai/api -m

What about just the qwen ones?

uvx llcat openrouter.ai/api -m '*qwen3*'

What about their capabilities in JSON?

uvx llcat openrouter.ai/api -m '*qwen3*' --info | jq .

Sure. What about a different protocol, say ollama?

uvx llcat localhost:11434 -m '*qwen3*' --info | jq .

All the abstraction without those pesky leaks.

There's also support for schemas, dry-runs, expressing the calls as raw curls, adding body parameters (such as top_p or temperature), custom timeouts, and customizing thinking or streaming.

The basic CLI parameters are compatible with Simon Willison's llm which makes the transition a drop-in replacement. It's also faster than llm. Time it yourself. You'll see...

There's even an included tool for sanely manipulating the JSONs of the conversations for context engineering.

Examples

Here's some examples of how to use llcat as a building block for many common use-cases:

• Transferrable Conversations
• Stateful Interaction
• Interactive Chat
• Structured Output
• Evals
• Tool Calling

Example: Transferrable Conversations

Because conversations, models and servers are decoupled, you can mix and match them at any time.

Here's one conversation, hopping across models and servers.

Start a chat with Deepseek:

$ llcat -u https://openrouter.ai/api \
        -m deepseek/deepseek-r1-0528:free \
        -c /tmp/convo.txt \
        -k "$(cat openrouter.key)" \
        "What is the capital of France?"

Continue it with Qwen using MAS format and using the @ syntax for including the key by file:

$ llcat -u "https://openrouter.ai/api#m=qwen/qwen3-4b:free"
        -c /tmp/convo.txt \
        -k @openrouter.key \
        "And what about Canada?"

And finish on the local network:

$ llcat -u http://192.168.1.21:8080 \
        -c /tmp/convo.txt \
        "And what about Japan?"

Since the conversation goes to the filesystem as JSON you can use things like inotify or fuse and push it off to a vector search backend or modify the context window between calls.

Example: Adding State

llcat's explicit syntax means lots of things are within reach.

For instance wrappers can be made custom to your workflow.

Here's a way to store state with environment variables to make invocation more convenient:

llf()        { llc "$@" 2> >(jq . >&2) | examples/spinner sd }
llc()        { llcat -m "$LLC_MODEL" -u "$LLC_SERVER" -k "$LLC_KEY" "$@" }
llc-model()  { LLC_MODEL=$(llcat -m  -u "$LLC_SERVER" -k "$LLC_KEY" | fzf) }
llc-server() { LLC_SERVER=$1 }
llc-key()    { LLC_KEY=$1 }

And now you can do things like this:

$ llc-server http://192.168.1.21:8080
$ llc "write a diss track where the knapsack problem hates on the towers of hanoi"

And what's that llf at the top? That uses jq to pretty print the errors and streamdown to pretty print the output along with a program to display a spinner while you wait.

There's no configuration files to parse or implicit states to manage.

Example: Interactive Chat

A conversation interface is also quick:

#!/usr/bin/env bash

# We pick a file for the conversation or allow a user to pass it in with a CONV environment variable
conv=${CONV:-$(mktemp)}
echo -e "  Using: $conv\n"

# Show the previous conversation if there is any, stylize it with streamdown
jq -r '.[] | "\n**\(.role)**: \(.content)"' $conv | sd

# Read prompts in a loop
while read -E -p "  >> " query; do

    # Take the command line arguments of the shell script, pass them to llcat
    llcat -c $conv "$@" "$query" |& sd
    echo
done

So now instead of

llcat -u http://myserver -k mykey -m model

Our conversation loop can be invoked like

conversation.sh -u http://myserver -k mykey -m model

Adding additional features is trivial.

Example: Structured Output

Using the schema feature you can pass json in to enforce a schema. Try something like

$ llcat -u http://localhost:11434 -sc @examples/schema.json "give me a person"

Example: Evals

Running the same thing on multiple models and assessing the outcome is straight forward. Here we're using ollama

pre="llcat -u http://localhost:11434"
for model in $($pre -m); do
   $pre -m $model "translate 国際化がサポートされています。to english" > ${model}.outcome
done

You can use patterns like that also for testing tool calling completion. Here's a bigger example: a humor eval to see if models know a funny joke when they see one

If an error happens contacting the server, you get the request, response, and a non-zero exit.

Try this to see what that looks like

uvx llcat -u fakecomputer

Example: Tool calling

The examples directory contains this music playing tool listing the contents of this album:

$ llcat -u http://127.1:8080 -tf tool_file.json -tp tool_program.py "what mp3s do i have in my ~/mp3 directory"
{"level": "debug", "class": "toolcall", "message": "request", "obj": {"id": "iwCGjcRic8GAFB2jUvBUOeF9NNrldfxz", "type": "function", "function": {"name": "list_mp3s", "arguments": {"path":"~/mp3"}}}}
{"level": "debug", "class": "toolcall", "message": "result", "obj": ["Elektrobopacek - Towards the final Battle.mp3", "Elektrobopacek - Escape the Labyrinth.mp3", "Elektrobopacek - Journey to the misty Lands.mp3", "Elektrobopacek - Mistral Forte.mp3", "Elektrobopacek - Leaving Spaceport X-19.mp3", "Elektrobopacek - Dracula Rising.mp3"]}
Here are the MP3 files in your `~/mp3` directory:

1. **Elektrobopacek - Towards the final Battle.mp3**
2. **Elektrobopacek - Escape the Labyrinth.mp3**
3. **Elektrobopacek - Journey to the misty Lands.mp3**
4. **Elektrobopacek - Mistral Forte.mp3**
5. **Elektrobopacek - Leaving Spaceport X-19.mp3**
6. **Elektrobopacek - Dracula Rising.mp3**

Would you like to play any of these? Just share the filename, and I can play it for you! 🎵

In this example you can see how nothing is hidden so if the model makes a mistake it is immediately identifiable.

The debug JSON objects are sent to stderr so routing it separately is trivial.

MCP

MCPFile

This file is what you usually need to make for an mcp server definition:

{
  "mcpServers": {
    "<some_server>": {
      "command": "<some_command>",
      "args": ["<some>", "<args>"]
    }
    ...
  }
}

There's a basic extension on MCP here. You can explicity disable an MCP server by adding a flag "disabled": true like so:

{
  "mcpServers": {
    "<some_server>": {
      "command": "<some_command>",
      "disabled": true,
      "args": ["<some>", "<args>"]
    }
    ...
  }
}

MCPCat

MCP can be simple with simple tools. There's one included here. mcpcat is a 22 line Bash script.

Here is an example of it in use:

$ mcpcat init list | \
  uv run python -m my-server | \
  jq .

Let's say there's a calculator mcp, you can do something like

$ mcpcat init call calculate '{"expression":"2+2"}' | \
   uv run python -m mcp_server_calculator \
   jq .

The beauty here is you can see the Emperor's new clothes up close. Simply omit the pipe.

$ mcpcat init call calculate '{"expression":"2+2"}'
{"jsonrpc":"2.0","id":4,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"mcpcat","version":"1.0"}}}
{"jsonrpc":"2.0","method":"notifications/initialized"}
{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"calculate","arguments":{"expression":"2+2"}}}

That's all the STDIO Transport is.

There's ways of doing the network transports with this script as well. All you need is the appropriate network tools and compose away.

Usage

Now it's your turn.

usage: llcat [-h] [-su [@]SERVERURL] [-sk [@]SERVERKEY] [-to TIMEOUT]
             [-pr PROTO] [-m [[@]MODEL]] [-s [@]SYSTEM] [-a ATTACH]
             [-c CONVERSATION] [-cr CONVERSATIONRO] [-eb [@]EXTRABODY]
             [-sc [@]SCHEMA] [-mf MCP] [-tp TOOL_PROGRAM] [-tf TOOL_FILE]
             [-ps] [-bq BE_QUIET] [-nt] [-ns] [-nw] [-f] [--curlify]
             [--dry] [--version] [--info [INFO]] [--save SAVE]
             [[@]user_prompt ...]

llcat is /usr/bin/cat for LLMs. 

        🐱 Me-wow! 

https://github.com/day50-dev/llcat

Options with a [@] prefix can either be strings or paths to a file, curl style, @/like/this
They can also have line numbers @/like/this:0 or jq syntax @/like/this:.[0].field

positional arguments:
  [@]user_prompt        your prompt. If you omit the server_url, the first
                        argument will be the server

options:
  -h, --help            show this help message and exit
  -su, -u, --server_url [@]SERVERURL
                        server URL (e.g., http://::1:8080). Also supports MAS
                        format
  -sk, -k, --server_key [@]SERVERKEY
                        server API key for authorization
  -to, --timeout TIMEOUT
                        timeout in seconds for the read
  -pr, --proto PROTO    protocol to use (ollama, llama.cpp, openai, auto)
  -m, --model [[@]MODEL]
                        model to use (or list models if no value)
  -s, --system [@]SYSTEM
                        system prompt
  -a, --attach ATTACH   attach file(s)
  -c, --conversation CONVERSATION
                        conversation history file (r/w)
  -cr, --conversationro CONVERSATIONRO
                        the readonly conversation input (ro)
  -eb, --extra_body [@]EXTRABODY
                        JSON to add to the body, such as max_tokens or
                        temperature
  -sc, --schema [@]SCHEMA
                        set a schema to force structured output
  -mf, --mcp MCP        MCP file to use
  -tp, --tool_program TOOL_PROGRAM
                        program to execute tool calls
  -tf, --tool_file TOOL_FILE
                        JSON file with tool definitions
  -ps, --ps             currently running model (if supported)
  -bq, --be_quiet BE_QUIET
                        make it shutup about things
  -nt, --no_think       disable thinking
  -ns, --no_stream      disable streaming
  -nw, --no_wrap        do not wrap inputs in <xml-like-syntax>
  -f, --force           disable SSL verification
  --curlify             write curl equivalents of calls to stdout
  --dry                 dry run
  --version             show program's version number and exit
  --info [INFO]         get the info for a model
  --save SAVE           save an invocation to a reusable JSON file. Supply it
                        as a bare @argument to reuse

We're excited to see what you build.

Brought to you by DA`/50: Make the future obvious.