Wrapper around language model APIs
Project description
LmWrapper
Provides a wrapper around OpenAI API and Hugging Face Language models, focusing on being a clean and user-friendly interface. Because every input and output is object-oriented (rather than just JSON dictionaries with string keys and values), your IDE can help you with things like argument and property names and catch certain bugs statically. Additionally, it allows you to switch inbetween OpenAI endpoints and local models with minimal changes.
Installation
For usage with just OpenAI models:
pip install lmwrapper
For usage with HuggingFace models as well:
pip install 'lmwrapper[hf]'
For development dependencies:
pip install 'lmwrapper[dev]'
Please note that this method is for development and not supported.
Example usage
Completion models
from lmwrapper.openai_wrapper import get_open_ai_lm, OpenAiModelNames
from lmwrapper.structs import LmPrompt
lm = get_open_ai_lm(
model_name=OpenAiModelNames.gpt_3_5_turbo_instruct,
api_key_secret=None, # By default, this will read from the OPENAI_API_KEY environment variable.
# If that isn't set, it will try the file ~/oai_key.txt
# You need to place the key in one of these places,
# or pass in a different location. You can get an API
# key at (https://platform.openai.com/account/api-keys)
)
prediction = lm.predict(
LmPrompt( # A LmPrompt object lets your IDE hint on args
"Once upon a",
max_tokens=10,
)
)
print(prediction.completion_text)
# " time, there were three of us." - Example. This will change with each sample.
Chat models
from lmwrapper.openai_wrapper import get_open_ai_lm, OpenAiModelNames
from lmwrapper.structs import LmPrompt, LmChatTurn
lm = get_open_ai_lm(OpenAiModelNames.gpt_3_5_turbo)
# Single user utterance
pred = lm.predict("What is 2+2?")
print(pred.completion_text) # "2+2 is equal to 4."
# Conversation alternating between `user` and `assistant`.
pred = lm.predict(LmPrompt(
[
"What is 2+2?", # user turn
"4", # assistant turn
"What is 5+3?" # user turn
"8", # assistant turn
"What is 4+4?" # user turn
# We use few-shot turns to encourage the answer to be our desired format.
# If you don't give example turns you might get something like
# "The answer is 8." instead of just "8".
],
max_tokens=10,
))
print(pred.completion_text) # "8"
# If you want things like the system message, you can use LmChatTurn objects
pred = lm.predict(LmPrompt(
text=[
LmChatTurn(role="system", content="You always answer like a pirate"),
LmChatTurn(role="user", content="How does bitcoin work?"),
],
max_tokens=25,
temperature=0,
))
print(pred.completion_text)
# "Arr, me matey! Bitcoin be a digital currency that be workin' on a technology called blockchain..."
Hugging Face models
Causal LM models on Hugging Face models can be used interchangeably with the OpenAI models.
from lmwrapper.huggingface_wrapper import get_huggingface_lm
from lmwrapper.structs import LmPrompt
lm = get_huggingface_lm("gpt2") # The smallest 124M parameter model
prediction = lm.predict(LmPrompt(
"The capital of Germany is Berlin. The capital of France is",
max_tokens=1,
temperature=0,
))
print(prediction.completion_text)
assert prediction.completion_text == " Paris"
Features
lmwrapper
provides several features missing from the OpenAI API.
Caching
Add caching = True
in the prompt to cache the output to disk. Any
subsequent calls with this prompt will return the same value. Note that
this might be unexpected behavior if your temperature is non-zero. (You
will always sample the same output on reruns.)
Retries on rate limit
An OpenAIPredictor can be configured to read rate limit errors and wait the appropriate amount of seconds in the error before retrying.
from lmwrapper.openai_wrapper import *
lm = get_open_ai_lm(
OpenAiModelNames.gpt_3_5_turbo_instruct,
retry_on_rate_limit=True
)
Other features
Built-in token counting
from lmwrapper.openai_wrapper import *
from lmwrapper.structs import LmPrompt
lm = get_open_ai_lm(OpenAiModelNames.gpt_3_5_turbo_instruct)
assert lm.estimate_tokens_in_prompt(
LmPrompt("My name is Spingldorph", max_tokens=10)) == 7
assert not lm.could_completion_go_over_token_limit(LmPrompt(
"My name is Spingldorph", max_tokens=1000))
TODOs
If you are interested in one of these particular features or something else please make a Github Issue.
- Openai completion
- Openai chat
- Huggingface interface
- Huggingface device checking on PyTorch
- Move cache to be per project
- Anthropic interface
- Redesign cache to make it easier to manage
- sort through usage of quantized models
- async / streaming
- Additional Huggingface runtimes (TensorRT, BetterTransformers, etc)
- Cost estimating (so can estimate cost of a prompt before running / track total cost)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for lmwrapper-0.9.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c5c9e6b21f8caa14a7ab6ce0928e8a1f596c5ebc67886a76a540672d4f669152 |
|
MD5 | 70271e5d67a70a645954128878a82093 |
|
BLAKE2b-256 | bccc3867fbe5acb121d64ddd769eb65d4bfa766990c199734e6ace8293ce692e |