Record and replay LLM interactions for langchain
Project description
VCR LangChain
Adapts VCR.py for use with LangChain. Refactor with confidence as you record and replay all your LLM logic in a contained environment, free from any and all side effects.
Quickstart
pip install vcr-langchain
Use it with pytest:
import vcr_langchain as vcr
from langchain.llms import OpenAI
@vcr.use_cassette()
def test_use_as_test_decorator():
llm = OpenAI(model_name="text-ada-001")
assert llm("Tell me a surreal joke") == "<put the output here>"
The next time you run it:
- the output is now deterministic
- it executes a lot faster by replaying from cache
- no command executions or other side effects actually happen
- you no longer need to have real API keys defined for test execution in CI environments
For more examples, see the usages test file.
Why not just use VCR.py directly?
- This offers higher-level, more human-readable recordings for inspection
- This supports recordings of langchain tool interactions as well, which includes non-network requests such as command executions inside Bash (coming soon) or the Python REPL. What if you wanted to make sure that your agent supports file deletion, without actually letting it delete anything?
Documentation
For more information on how VCR works and what other options there are, please see the VCR docs.
For more information on how to use langchain, please see the langchain docs.
Please note that there is a lot of langchain functionality that I haven't gotten around to hijacking for recording. If there's anything you need to record in a cassette, please open a PR or issue.
Projects that use this
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for vcr_langchain-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6edbc5156859ee7ca448c7f6876adfe16a316d10b7b2a873c9df2e5dab00358 |
|
MD5 | 9ce6911dcd711b64423eb2d1d9832065 |
|
BLAKE2b-256 | 4b68b69164ea902399c4f850fb7dfb4c02bb29645d99b01a40e36442e595adb4 |