Skip to main content

No project description provided

Project description

PromptOptimizer

kevin inspired logo

Minimize LLM token complexity to save API costs and model computations.

Features

  • Plug and Play Optimizers: Minimize token complexity using optimization methods without any access to weights, logits or decoding algorithm. Directly applicable to virtually all NLU systems.
  • Protected Tags: Special protected tags to mark important sections of prompt that should not be removed/modified.
  • Sequential Optimization: Chain different optimizers together sequentially.
  • Optimization Metrics: Number of tokens reduced and semantic similarity before and after optimization.
  • Langhcain and JSON Support: Supports langchain style prompt chains and OpenAI request JSON Object.

Why?

  • Minimize Token Complexity: Token Complexity is the amount of prompt tokens required to achieve a given task. Reducing token complexity corresponds to linearly reducing API costs and quadratically reducing computational complexity of usual transformer models.
  • Save Money: For large businesses, saving 10% on token count can lead to saving 100k USD per 1M USD.
  • Extend Limitations: Some models have small context lengths, prompt optimizers can help them process larger than context documents.
Prompt # Tokens Correct Response?
Who is the president of the United States of America? 11
Who president US 3 (-72%)

Installation

Quick Installation

pip install prompt-optimizer

Install from source

git clone https://github.com/vaibkumr/prompt-optimizer.git;
cd prompt-optimizer;
pip install -e .

Disclaimer

There is a compression vs performance tradeoff -- the increase in compression comes at the cost of loss in model performance. The tradeoff can be greatly mitigated by chosing the right optimize for a given task. There is no single optimizer for all cases. There is no Adam here.

Getting started

from prompt_optimizer.poptim import EntropyOptim

prompt = """The Belle Tout Lighthouse is a decommissioned lighthouse and British landmark located at Beachy Head, East Sussex, close to the town of Eastbourne."""
p_optimizer = EntropyOptim(verbose=True, p=0.1)
optimized_prompt = p_optimizer(prompt)
print(optimized_prompt)

Evaluations

Following are the results for logiqa OpenAI evals task. It is only performed for a subset of first 100 samples. Please note the optimizer performance over this task should not be generalized to other tasks, more thorough testing and domain knowledge is needed to choose the optimal optimizer.

Name % Tokens Reduced LogiQA Accuracy USD Saved Per $100
Default 0.0 0.32 0.0
Entropy_Optim_p_0.05 0.06 0.3 6.35
Entropy_Optim_p_0.1 0.11 0.28 11.19
Entropy_Optim_p_0.25 0.26 0.22 26.47
Entropy_Optim_p_0.5 0.5 0.08 49.65
SynonymReplace_Optim_p_1.0 0.01 0.33 1.06
Lemmatizer_Optim 0.01 0.33 1.01
NameReplace_Optim 0.01 0.34 1.13
Punctuation_Optim 0.13 0.35 12.81
Autocorrect_Optim 0.01 0.3 1.14
Pulp_Optim_p_0.05 0.05 0.31 5.49
Pulp_Optim_p_0.1 0.1 0.25 9.52

Cost-Performance Tradeoff

The reduction in cost often comes with a loss in LLM performance. Almost every optimizer have hyperparameters that control this tradeoff.

For example, in EntropyOptim the hyperparamter p, a floating point number between 0 and 1 controls the ratio of tokens to remove. p=1.0 corresponds to removing all tokens while p=0.0 corresponds to removing none.

The following chart shows the trade-off for different values of p as evaluated on the OpenAI evals logiqa task for a subset of first 100 samples.

tradeoff

Contributing

There are several directions to contribute to. Please see CONTRIBUTING.md for contribution guidelines and possible future directions.

Social

Contact us on twitter Vaibhav Kumar and Vaibhav Kumar.

Inspiration

Image

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

prompt_optimizer-0.2.1.tar.gz (19.6 kB view details)

Uploaded Source

Built Distribution

prompt_optimizer-0.2.1-py3-none-any.whl (26.6 kB view details)

Uploaded Python 3

File details

Details for the file prompt_optimizer-0.2.1.tar.gz.

File metadata

  • Download URL: prompt_optimizer-0.2.1.tar.gz
  • Upload date:
  • Size: 19.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.11.2 Darwin/21.3.0

File hashes

Hashes for prompt_optimizer-0.2.1.tar.gz
Algorithm Hash digest
SHA256 26a86a9ba90420dc4d404495722de0ce49a9b59c13925aaaa934041b459426c1
MD5 85015a4ac9478405b2f541942b16e1d6
BLAKE2b-256 821c8a1feae81004c0e9db4476405040e139eb675fb44b6ba9764b0569f39179

See more details on using hashes here.

File details

Details for the file prompt_optimizer-0.2.1-py3-none-any.whl.

File metadata

File hashes

Hashes for prompt_optimizer-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 061dd5e22b29238bb1d07c4880a3656cc5293f4ba7794f1993a641ba7e357635
MD5 7072ef2cc4cb6f8d07f6c5c783596599
BLAKE2b-256 939cb3fc3eaa9fd440e2e2975f57329b64d182b4242099f5c0ab8c982355ad23

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page