A tool for caching prompts and compleation based on embedding
Reason this release was yanked:
critical bug
Project description
PromptCache
A simple-to-use prompt caching optmized for simplicity and speed.
Installation
pip install promptcache
Quickstart
from promptcache import RedisCache
>>> cache = RedisCache()
>>> cache.set("this is a prompt", "this is the completion")
>>> cache.search("this is a prompt")
{'completion': 'this is the completion', 'prompt': 'this is a prompt', 'distance': 0}
>>> cache.search("this is a prompt prompt prompt")
{'completion': 'this is the completion', 'prompt': 'this is a prompt', 'distance': 0.1254}
>>> cache.get("this is a prompt")
'this is the completion'
>>> cache.delete("this is a prompt")
>>> cache.get("this is a prompt")
None
Features
- Simplicity, speed and scalability using redis
- Fast embedding base on fastembed
- Insturction embedding based on InstructorEmbedding
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
promptcache-0.0.1a1.tar.gz
(9.6 kB
view hashes)
Built Distribution
Close
Hashes for promptcache-0.0.1a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3d95bed856d7bd4b89e2eaf36a6fc3dce5c53ee7ca9dfbfb2c0c2aa4dfe1ef1 |
|
MD5 | f67d0ed9cc01dc057d05bf58d3eaa81b |
|
BLAKE2b-256 | a3a1af9fde47028778b71816fcef161a48b40656ec5abf091243f350dc567f44 |