Skip to main content

Exa - Pytorch

Project description

Multi-Modality

Exa

Ultra-optimized fast inference library for running exascale LLMs locally on modern consumer-class GPUs.

Principles

  • Radical Simplicity (Utilizing super-powerful LLMs with as minimal code as possible)
  • Ultra-Optimizated (High Performance classes that extract all the power from these LLMs)
  • Fludity & Shapelessness (Plug in and play and re-architecture as you please)

🤝 Schedule a 1-on-1 Session

Book a 1-on-1 Session with Kye, the Creator, to discuss any issues, provide feedback, or explore how we can improve Zeta for you.


📦 Installation 📦

You can install the package using pip

pip install exxa

Usage

Inference

from exa import Inference

model = Inference(
    model_id="georgesung/llama2_7b_chat_uncensored",
    quantized=True
)

model.run("What is your name")

GPTQ Inference

from exa import GPTQInference

model_id = "facebook/opt-125m"
model = GPTQInference(model_id=model_id, max_length=400)

prompt = "in a land far far away"
result = model.run(prompt)
print(result)

Quantize

from exa import Quantize

#usage
quantize = Quantize(
     model_id="bigscience/bloom-1b7",
     bits=8,
     enable_fp32_cpu_offload=True,
)

quantize.load_model()
quantize.push_to_hub("my model")
quantize.load_from_hub('my model')

🎉 Features 🎉

  • World-Class Quantization: Get the most out of your models with top-tier performance and preserved accuracy! 🏋️‍♂️

  • Automated PEFT: Simplify your workflow! Let our toolkit handle the optimizations. 🛠️

  • LoRA Configuration: Dive into the potential of flexible LoRA configurations, a game-changer for performance! 🌌

  • Seamless Integration: Designed to work seamlessly with popular models like LLAMA, Falcon, and more! 🤖


💌 Feedback & Contributions 💌

We're excited about the journey ahead and would love to have you with us! For feedback, suggestions, or contributions, feel free to open an issue or a pull request. Let's shape the future of fine-tuning together! 🌱


License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exxa-0.1.0.tar.gz (8.0 kB view details)

Uploaded Source

Built Distribution

exxa-0.1.0-py3-none-any.whl (8.3 kB view details)

Uploaded Python 3

File details

Details for the file exxa-0.1.0.tar.gz.

File metadata

  • Download URL: exxa-0.1.0.tar.gz
  • Upload date:
  • Size: 8.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0

File hashes

Hashes for exxa-0.1.0.tar.gz
Algorithm Hash digest
SHA256 82e2b56efe7617f14816dd14862b447e6ba6dcc8329f72712e69b4a7c3a59a17
MD5 f4351a1f85e6f8ab45ae4e28b1328288
BLAKE2b-256 f6de468b914602c61b51c87caa5318fd54b65dbce34f0a3bec9228224f60af1d

See more details on using hashes here.

File details

Details for the file exxa-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: exxa-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 8.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0

File hashes

Hashes for exxa-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 40b9c1b716392640605182a12613c9a0098c26469097d9c8960f4c019531344e
MD5 ec24282c90b4eaad98209839565b8e43
BLAKE2b-256 0ed63062fe9b1914e86b9bd2c91a73e562903e074d8a572956ea6009fbecf046

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page