⚡ Quick Demo

Tailing an active LLM evaluation log with formatted JSON, metadata extraction, and safe handling of corrupt disk writes:

$ jsonl-tail examples/eval.log.jsonl

{
  "timestamp": "2026-09-03T10:00:03Z",
  "level": "INFO",
  "run_id": "eval-001",
  "model": "claude-3-5-sonnet",
  "step": 4,
  "score": 0.97,
  "meta": {
    "dataset": "human-eval",
    "temperature": 0.0
  }
}
[invalid json] [corrupt-disk-write-header] 0xDEADBEEF <<< corrupt log line pass-through >>>
{
  "timestamp": "2026-09-03T10:00:04Z",
  "level": "ERROR",
  "run_id": "eval-001",
  "model": "claude-3-5-sonnet",
  "step": 5,
  "score": 0.0,
  "meta": {
    "dataset": "human-eval",
    "error": "rate_limit_exceeded"
  }
}
{
  "timestamp": "2026-09-03T10:00:05Z",
  "level": "INFO",
  "run_id": "eval-001",
  "model": "gemini-2.5-pro",
  "step": 6,
  "score": 0.91,
  "meta": {
    "dataset": "math500",
    "i18n": "日本語テスト (evaluation successful)"
  }
}

Notice how the middle corrupt line is badged with [invalid json] and passes through cleanly without crashing the viewer.

💡 Why jsonl-tail?

In modern AI engineering, agent trajectories, eval benchmarks (SWE-bench, GSM8K), and model training traces all stream as append-only .jsonl files. Developers typically inspect these logs using traditional Unix pipelines:

$ tail -f run.jsonl | jq .

While convenient, this pipeline breaks down in production for three critical reasons:

1. The Fatal Syntax Crash: jq expects 100% syntactically valid JSON. If an external library prints a raw traceback, a progress bar, or an unformatted warning directly to stdout, jq throws parse error and immediately aborts the pipeline.
2. Standard Output Buffering: When jq is piped into downstream commands (or monitored in subshells), its output is block-buffered by default unless explicitly invoked with --unbuffered, delaying live step updates.
3. Partial Write Race Conditions: Active logging processes periodically flush byte chunks before the trailing newline \n hits disk. Standard tools attempt to parse the half-written line, corrupting terminal output.

jsonl-tail solves these edge cases out of the box. It is a single, zero-dependency CLI utility that gives you native tail -f semantics specifically engineered for JSON Lines.

🎯 Key Capabilities

🏗️ System Architecture

jsonl-tail follows a streaming, memory-bounded pipeline designed for continuous operation:

flowchart LR
    subgraph INGESTION["1. Ingestion & Buffering"]
        A["File Path(s) / Piped Stdin"] --> B["Bounded Ring Buffer (deque)"]
        A --> C["Atomic Follow Poller"]
    end

    subgraph RECOVERY["2. Fault-Tolerant Engine"]
        B --> D["Line Normalizer"]
        C --> D
        D --> E["JSON Decoder & Guard"]
        E -->|Valid Record| F["Filter & Key Projection"]
        E -->|Corrupt / Raw Text| G["Pass-Through Badging"]
    end

    subgraph EMISSION["3. Output Pipeline"]
        F --> H["Format Engine (Pretty / Compact)"]
        G --> I["ANSI Color Renderer"]
        H --> I
        I --> J["Immediate Unbuffered Flush (stdout)"]
    end

Architectural Highlights

• Bounded Memory Ingestion: Files and stdin streams are consumed through a bounded FIFO ring buffer (collections.deque(maxlen=N)). Memory footprint remains constant regardless of whether the log is 10 KB or 50 GB.
• Atomic Line Recovery: The follow engine inspects trailing byte offsets for line feeds (\n, \r). Incomplete writes are buffered in memory and the file pointer is preserved until complete records land on disk.
• Stream Auto-Reconfiguration: Standard output and error streams are dynamically reconfigured to UTF-8 with character replacement on startup, guaranteeing consistent rendering on Windows terminals and non-UTF8 locales.

🚀 Quick Start

Installation

Choose the installation method that fits your workflow:

# Method 1: Install as a standalone CLI tool via uv (Recommended)
uv tool install jsonl-tail

# Method 2: Install via standard pip
pip install jsonl-tail

# Method 3: Run instantly without installing
uvx jsonl-tail examples/eval.log.jsonl

# Method 4: Editable local development setup
git clone https://github.com/FreakyAdy/jsonl-tail.git
cd jsonl-tail
uv sync

Basic Commands

# 1. View last 10 records, formatted with syntax colors
jsonl-tail examples/eval.log.jsonl

# 2. View all records in compact one-line JSON
jsonl-tail -a -c examples/eval.log.jsonl

# 3. Follow live appended logs as they land
jsonl-tail -f examples/eval.log.jsonl

# 4. Filter for specific severity levels using regex
jsonl-tail --regex '"(WARN|ERROR)"' examples/eval.log.jsonl

# 5. Extract a specific nested key in compact format
jsonl-tail -k meta -c examples/eval.log.jsonl

# 6. Pipe input from stdin while respecting tail counts
cat examples/eval.log.jsonl | jsonl-tail -n 3 -c

💡 Terminal Tip: Set NO_COLOR=1 in your environment to disable ANSI color codes when piping to files or plain text pagers.

Runnable Examples

Explore the bundled examples/ directory to test jsonl-tail against real evaluation records:

# Inspect the sample evaluation log
jsonl-tail examples/eval.log.jsonl

# Filter for failed steps
jsonl-tail --filter "rate_limit_exceeded" examples/eval.log.jsonl

⚖️ Ecosystem Comparison

How jsonl-tail compares to existing log viewers and command-line JSON tools:

jsonl-tail is designed to be a lightweight, zero-dependency stream filter. When you need complex tree queries across deep schemas, use jq. When you need full-screen interactive log analysis, use lnav. When you want a crash-proof tail -f for JSON Lines, use jsonl-tail.

🤝 Contributing & Community

jsonl-tail is an open-source community project. Contributions, bug reports, and ideas are warmly welcome!

• CONTRIBUTING.md: Development setup, local testing, and PR guidelines.
• CHANGELOG.md: Release history and version notes.
• Issue Tracker: Report bugs or request enhancements.
• Pull Request Template: Standard PR review checklist.

📄 License

Distributed under the MIT License.

Copyright (c) 2026 Aditya Suryavanshi