cachy

We often call APIs while prototyping and testing our code. A single API call (e.g. an Anthropic chat completion) can take 100’s of ms to run. This can really slow down development especially if our notebook contains many API calls 😞.

cachy caches API requests. It does this by saving the result of each call to a local cachy.jsonl file. Before calling an API (e.g. OpenAI) it will check if the request exists in cachy.jsonl. If it does it will return the cached result.

How does it work?

Under the hood popular SDK’s like OpenAI and Anthropic use httpx.Client and httpx.AsyncClient.

cachy patches the send method of both clients and injects a simple caching mechanism:

• create a cache key from the request
• if the key exists in cachy.jsonl return the cached response
• if not, call the API and save the response to cachy.jsonl

Usage

To use cachy

• install the package: pip install pycachy
• add the snippet below to the top of your notebook

from cachy import enable_cachy

enable_cachy()

By default cachy will cache requests made to OpenAI, Anthropic, Gemini and DeepSeek.

Docs

Docs can be found hosted on this GitHub repository’s pages.

How to use

First import and enable cachy

from cachy import enable_cachy

enable_cachy()

Now run your api calls as normal.

from openai import OpenAI

cli = OpenAI()

r = cli.responses.create(model="gpt-4.1", input="Hey!")
r

Response(id='resp_0834eb6d2ba90964006a746409ecac8199ab89307aeb3ec7fb', created_at=1786012681.0, error=None, incomplete_details=None, instructions=None, metadata={}, model='gpt-4.1-2025-04-14', object='response', output=[ResponseOutputMessage(id='msg_0834eb6d2ba90964006a74640aa61881998f3464aee7761b3f', content=[ResponseOutputText(annotations=[], text='Hey! How can I help you today? 😊', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase=None)], parallel_tool_calls=True, temperature=1.0, tool_choice='auto', tools=[], top_p=1.0, background=False, completed_at=1786012682.0, conversation=None, max_output_tokens=None, max_tool_calls=None, moderation=None, previous_response_id=None, prompt=None, prompt_cache_key=None, prompt_cache_options=None, prompt_cache_retention='in_memory', reasoning=Reasoning(context=None, effort=None, generate_summary=None, mode=None, summary=None), safety_identifier=None, service_tier='default', status='completed', text=ResponseTextConfig(format=ResponseFormatText(type='text'), verbosity='medium'), top_logprobs=0, truncation='disabled', usage=ResponseUsage(input_tokens=9, input_tokens_details=InputTokensDetails(cache_write_tokens=0, cached_tokens=0), output_tokens=11, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=20), user=None, billing={'payer': 'developer'}, frequency_penalty=0.0, presence_penalty=0.0, store=True, tool_usage={'image_gen': {'input_tokens': 0, 'input_tokens_details': {'image_tokens': 0, 'text_tokens': 0}, 'output_tokens': 0, 'output_tokens_details': {'image_tokens': 0, 'text_tokens': 0}, 'total_tokens': 0}, 'web_search': {'num_requests': 0}})

If you run the same request again it will read it from the cache.

r = cli.responses.create(model="gpt-4.1", input="Hey!")
r

Response(id='resp_0834eb6d2ba90964006a746409ecac8199ab89307aeb3ec7fb', created_at=1786012681.0, error=None, incomplete_details=None, instructions=None, metadata={}, model='gpt-4.1-2025-04-14', object='response', output=[ResponseOutputMessage(id='msg_0834eb6d2ba90964006a74640aa61881998f3464aee7761b3f', content=[ResponseOutputText(annotations=[], text='Hey! How can I help you today? 😊', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase=None)], parallel_tool_calls=True, temperature=1.0, tool_choice='auto', tools=[], top_p=1.0, background=False, completed_at=1786012682.0, conversation=None, max_output_tokens=None, max_tool_calls=None, moderation=None, previous_response_id=None, prompt=None, prompt_cache_key=None, prompt_cache_options=None, prompt_cache_retention='in_memory', reasoning=Reasoning(context=None, effort=None, generate_summary=None, mode=None, summary=None), safety_identifier=None, service_tier='default', status='completed', text=ResponseTextConfig(format=ResponseFormatText(type='text'), verbosity='medium'), top_logprobs=0, truncation='disabled', usage=ResponseUsage(input_tokens=9, input_tokens_details=InputTokensDetails(cache_write_tokens=0, cached_tokens=0), output_tokens=11, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=20), user=None, billing={'payer': 'developer'}, frequency_penalty=0.0, presence_penalty=0.0, store=True, tool_usage={'image_gen': {'input_tokens': 0, 'input_tokens_details': {'image_tokens': 0, 'text_tokens': 0}, 'output_tokens': 0, 'output_tokens_details': {'image_tokens': 0, 'text_tokens': 0}, 'total_tokens': 0}, 'web_search': {'num_requests': 0}})

Scoped control

no_cachy sends the requests in its block live. It reads nothing from the cache and writes nothing to it. cachy enables caching for one block and restores the previous state on exit. It takes the same arguments as enable_cachy.

from cachy import cachy, no_cachy

with no_cachy(): r = cli.responses.create(model="gpt-4.1", input="Hey!")

with cachy(doms=('api.github.com',)): ...