narratoflow 0.4.0

Compress huge LLM context into dense intermediate representations. Provider-agnostic. Norwegian first-class.

narratoflow

Tags: llm · prompt-compression · token-optimization · cost-reduction · anthropic · openai · claude · gpt · pydantic · python · multilingual · narrative-generation · rag · context-window · apache-2.0

Compress huge LLM input context into dense intermediate representations. Pay fewer tokens, keep the meaning.

Docs: https://Mrrobi.github.io/narratoflow/ · PyPI: https://pypi.org/project/narratoflow/ · Source: https://github.com/Mrrobi/narratoflow

narratoflow (PyPI name; import as narrato) is an open-source Python library (Apache-2.0) for shrinking long source text before sending it to an LLM. It targets any workload where the input dwarfs the output and tokens are the dominant cost — RAG retrieval contexts, narrative generation, transcript summarisation, long-document QA.

The library has a generic, language- and domain-neutral core. Common starting points ship as named profiles (rag-en, narrative-no, news-en, …) so you do not have to choose every argument up front.

---

Highlights

• 43% token reduction on a real Norwegian narrative sample (gpt-4o-mini extractor → gpt-4o target), with 8/10 quality from an LLM judge
• Provider-agnostic — Anthropic + OpenAI ship out of the box; bring your own adapter for the rest
• Layered design — pick free deterministic layers, an LLM-backed semantic layer, or both
• Schema-driven — define a Pydantic model, get a dense JSON payload in return; 5 presets built in (narrative, qa, interview, dialogue, news)
• Long-document ready — automatic chunked map-reduce extraction with overlap-aware merging
• Anthropic prompt caching — opt-in via Compressor(cache=True). OpenAI's automatic prompt cache is reported on the response.
• Local models — OllamaProvider talks to a local Ollama daemon, no API key required
• Async API — Compressor.acompress(...) runs chunked extraction concurrently via asyncio.gather
• Typed — ships a py.typed marker (PEP 561); works directly with mypy / pyright / IDEs
• Multilingual — bundled stopwords for English, Norwegian, Swedish, Danish, German, French, Spanish, Italian, Portuguese, Dutch, Finnish, Polish
• Named profiles — Compressor.from_profile("rag-en") for one-line setup
• uv ready — pure-PEP-621 hatchling package; uv build, uvx, and uv add narratoflow all work

Provider-agnostic. Anthropic + OpenAI out of the box. Norwegian first-class. Stopword lists, lemma-friendly preprocessing, Norwegian benchmark samples bundled. Pluggable. Use any layer alone or stack them.

---

Why

LLM input is priced per token, and a long source document — say a 20-page Norwegian transcript that feeds a 200-word narrative — burns most of the budget before the model has written anything.

narrato lets you trade a tiny bit of fidelity for a large reduction in input tokens by passing your downstream LLM a dense, machine-friendly representation instead of the raw text.

The intermediate representation does not need to be human-readable. It just needs to be:

1. Cheap to produce.
2. Decodable by the downstream LLM into a faithful narrative.
3. Smaller in tokens than the original.

Architecture

                   ┌──────────────────────────────────────────────┐
   raw text  ──▶   │  L1 preprocess        (deterministic, free)   │
                   │  L2 codebook          (deterministic, free)   │
                   │  L3 semantic extract  (small LLM call)        │
                   │  L4 learned encoder   (optional, future)      │
                   └──────────────────────────────────────────────┘
                                      │
                                      ▼
                          CompressionResult
                          (payload + legend + stats)
                                      │
                                      ▼
                   ┌──────────────────────────────────────────────┐
                   │  Decoder.unpack_prompt()                      │
                   │  → ready-to-send prompt for downstream LLM    │
                   └──────────────────────────────────────────────┘

Pick which layers run for each call. Free layers stack with paid layers.

Quick start

pip install narratoflow

Set credentials:

export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...

Library — using a named profile (recommended)

from narrato import Compressor, Decoder

c = Compressor.from_profile("rag-en", provider="anthropic")
result = c.compress(long_source_text)

Run narratoflow profiles to see all built-in profiles, or define your own:

from narrato import Profile, register_profile, Compressor

register_profile(Profile(
    name="legal-en",
    description="English legal documents",
    source_lang="en",
    schema="qa",
    chunked=True,
    chunk_chars=6000,
))

c = Compressor.from_profile("legal-en", provider="openai")

Library — explicit construction

from narrato import Compressor, Decoder

c = Compressor(
    source_lang="no",
    provider="anthropic",
    extractor_model="claude-haiku-4-5-20251001",
    target_model="claude-opus-4-7",
    layers=["preprocess", "codebook", "extract"],
    schema="narrative",
)

result = c.compress(long_norwegian_text)

print(result.stats)
# {'input_tokens': 8421, 'output_tokens': 1102, 'ratio': 0.131, ...}

prompt = Decoder.unpack_prompt(
    result,
    instruction="Skriv en kort fortelling basert på faktene over.",
)
# Send `prompt` to your target LLM.

CLI

narrato compress input.txt --schema narrative --out compressed.json
narrato eval input.txt --schema narrative --target-task "Skriv en kort fortelling."

The eval command reports tokens_in, tokens_out, ratio, estimated cost savings, and an LLM-judge quality score.

Layer reference

Layer	What it does	Cost	Loss
preprocess	Whitespace/punct normalize, stopword strip, near-duplicate sentence dedupe	free	tiny
codebook	Frequent phrase → short code, entity → ID rewrite, emit legend	free	none (with legend)
extract	Small LLM extracts schema-conformant facts	cheap LLM call	lossy by design
learned (future)	Fine-tuned encoder produces dense codes	one-time train	tunable

Schemas

Schemas tell the extractor what to keep. Built-in presets live in narrato.schemas:

• narrative — characters, setting, ordered events, themes, tone, verbatim quotes
• More to come.

Define your own:

from pydantic import BaseModel
from narrato import Compressor

class MyFacts(BaseModel):
    summary: str
    speakers: list[str]
    key_dates: list[str]

c = Compressor(schema=MyFacts, ...)

Roadmap

• v0.1 — layered preprocess + codebook + schema extract, Anthropic + OpenAI, CLI, eval harness
• v0.2 — prompt-cache integration on Anthropic path, more schema presets, HF Spaces demo
• v0.3 — local model support (Ollama), Norwegian spaCy pipeline integration
• v0.4 — learned encoder (fine-tuned small model, distributed via HuggingFace)

Contributing

PRs welcome. Bring your own benchmark.

License

Apache-2.0. See LICENSE.