litellm-paritok

Paritok context compression as a LiteLLM Proxy callback.

Drop one line in your LiteLLM Proxy config and every request through the proxy is compressed before it hits the upstream provider — tool-schema filtering, file/tool-output compression, history compression, non-destructive recall (read_original / gateway_search_tools), and Edit-recovery — with zero changes to your agent or app.

Scope: this works under LiteLLM Proxy. The hooks it uses (async_pre_call_hook / async_post_call_success_hook) are proxy-only, so the bare SDK path (import litellm; litellm.completion(...)) is not intercepted. See Limitations.

Install

pip install litellm-paritok        # pulls in `paritok`

You also need a Paritok 4B compression backend (either one):

ollama pull paritok/paritok-4b-v1        # local Ollama, on your own machine
# or point Paritok at a hosted GPU server via an API key

Use

Add one line to your LiteLLM Proxy config.yaml:

litellm_settings:
  callbacks: ["litellm_paritok.paritok_handler"]

Then run the proxy as usual; your app keeps pointing at LiteLLM:

litellm --config config.yaml

The callback string is a dotted path module.instance, resolved by LiteLLM's get_instance_fn. Use litellm_paritok.paritok_handler (a ready-made instance this package exports) — not a bare "paritok", which LiteLLM would try to import as a module.

How it works

The package is a thin CustomLogger over the published paritok engine — the same compression/recall/recovery algorithms that power the Paritok proxy, wired to LiteLLM's hooks:

• async_pre_call_hook → ParitokEngine.process_request(...): filters tool schemas, compresses file reads / tool outputs / old history, injects the virtual recall tools, and returns the modified request for the upstream call.
• async_post_call_success_hook → recall resolve-loop + Edit recovery: answers any read_original / gateway_search_tools calls itself (re-requesting the model via litellm.acompletion until a plain turn returns), then rewrites any Edit whose old_string was authored against a lossy summary so it matches the real file byte-for-byte.

Limitations

• Proxy-only. The bare LiteLLM SDK path does not trigger these hooks. SDK coverage would need a separate wrapper / custom provider.
• Requires a Paritok 4B backend (Ollama or hosted GPU) — the prerequisite for any Paritok deployment, not specific to this integration.

Development

pip install -e ".[dev]"
PYTHONPATH=. pytest tests/ -v

Licensed under Apache-2.0.