Llama CLI
Project description
llama-toolchain
This repo contains the API specifications for various components of the Llama Stack as well implementations for some of those APIs like model inference.
The Llama Stack consists of toolchain-apis and agentic-apis. This repo contains the toolchain-apis.
Installation
You can install this repository as a package with pip install llama-toolchain
If you want to install from source:
mkdir -p ~/local
cd ~/local
git clone git@github.com:meta-llama/llama-toolchain.git
conda create -n toolchain python=3.10
conda activate toolchain
cd llama-toolchain
pip install -e .
The Llama CLI
The llama CLI makes it easy to configure and run the Llama toolchain. Read the CLI reference for details.
Appendix: Running FP8
If you want to run FP8, you need the fbgemm-gpu package which requires torch >= 2.4.0 (currently only in nightly, but releasing shortly...)
ENV=fp8_env
conda create -n $ENV python=3.10
conda activate $ENV
pip3 install -r fp8_requirements.txt
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file llama_cmdline-0.0.1rc1.tar.gz.
File metadata
- Download URL: llama_cmdline-0.0.1rc1.tar.gz
- Upload date:
- Size: 31.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b075d69903c7d364b0f3bd25e1db817bce3aa2899bfb524dd53a5591f441fffa |
|
| MD5 | 30fdddde499fee4e215717c15bccac32 |
|
| BLAKE2b-256 | 5c970fa4ae34f5487784cbc999a195ea29ccd2106a169806033a5fde689cd3fb |
File details
Details for the file llama_cmdline-0.0.1rc1-py3-none-any.whl.
File metadata
- Download URL: llama_cmdline-0.0.1rc1-py3-none-any.whl
- Upload date:
- Size: 51.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f3c17c2bc42f9ebe4c4ccd6cfbf32864a914696e3ca24eadeacb534781a75307 |
|
| MD5 | 39bc75d18f8f1ca4e7bad1a7164eb89e |
|
| BLAKE2b-256 | dfac2594c4d13cfe9b5f89a0ee6bb336d2c997d58adfed8798b3d2b371303452 |
