exxa 0.1.7

Exa - Pytorch

Exa

Ultra-optimized fast inference library for running exascale LLMs locally on modern consumer-class GPUs.

Principles

• Radical Simplicity (Utilizing super-powerful LLMs with as minimal code as possible)
• Ultra-Optimizated (High Performance classes that extract all the power from these LLMs)
• Fludity & Shapelessness (Plug in and play and re-architecture as you please)

---

🤝 Schedule a 1-on-1 Session

Book a 1-on-1 Session with Kye, the Creator, to discuss any issues, provide feedback, or explore how we can improve Exa for you.

---

📦 Installation 📦

You can install the package using pip

pip install exxa

---

Usage

Inference

from exa import Inference

model = Inference(
    model_id="georgesung/llama2_7b_chat_uncensored",
    quantized=True
)

model.run("What is your name")

GPTQ Inference

from exa import GPTQInference

model_id = "facebook/opt-125m"
model = GPTQInference(model_id=model_id, max_length=400)

prompt = "in a land far far away"
result = model.run(prompt)
print(result)

Quantize

from exa import Quantize

#usage
quantize = Quantize(
     model_id="bigscience/bloom-1b7",
     bits=8,
     enable_fp32_cpu_offload=True,
)

quantize.load_model()
quantize.push_to_hub("my model")
quantize.load_from_hub('my model')

---

🎉 Features 🎉

• World-Class Quantization: Get the most out of your models with top-tier performance and preserved accuracy! 🏋️‍♂️

• Automated PEFT: Simplify your workflow! Let our toolkit handle the optimizations. 🛠️

• LoRA Configuration: Dive into the potential of flexible LoRA configurations, a game-changer for performance! 🌌

• Seamless Integration: Designed to work seamlessly with popular models like LLAMA, Falcon, and more! 🤖

---

💌 Feedback & Contributions 💌

We're excited about the journey ahead and would love to have you with us! For feedback, suggestions, or contributions, feel free to open an issue or a pull request. Let's shape the future of fine-tuning together! 🌱

---

License

MIT