To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Navigation

Verified details  (What is this?)

These details have been verified by PyPI
Maintainers
Avatar for siyunzhao from gravatar.com siyunzhao

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License
  • Author: The LLMLingua team
  • Tags Prompt Compression, LLMs, Inference Acceleration, Black-box LLMs, Efficient LLMs
  • Requires: Python >=3.8.0
Classifiers

Project description

The author of this package has not provided a project description

Project details

Verified details  (What is this?)

These details have been verified by PyPI
Maintainers
Avatar for siyunzhao from gravatar.com siyunzhao

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License
  • Author: The LLMLingua team
  • Tags Prompt Compression, LLMs, Inference Acceleration, Black-box LLMs, Efficient LLMs
  • Requires: Python >=3.8.0
Classifiers

Release history Release notifications | RSS feed

This version

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llmlingua_promptflow-0.0.1.tar.gz (17.8 kB view hashes)

Uploaded Source

Built Distribution

llmlingua_promptflow-0.0.1-py3-none-any.whl (18.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page