pyllama 0.0.1

LLaMA: Open and Efficient Foundation Language Models

LLaMA

This repository is intended as a minimal, hackable and readable example to load LLaMA models and run inference. In order to download the checkpoints and tokenizer, fill this google form

Installation

pip install pyllama

Download

Once your request is approved, you will receive links to download the tokenizer and model files. Edit the download.sh script with the signed url provided in the email to download the model weights and tokenizer. The download process can also be automated by setting the respective environmental variables (PRESIGNED_URL and TARGET_FOLDER).

Inference

The provided example.py can be run on a single or multi-gpu node with torchrun and will output completions for two pre-defined prompts. Using TARGET_FOLDER as defined in download.sh:

torchrun --nproc_per_node MP example.py --ckpt_dir $TARGET_FOLDER/model_size --tokenizer_path $TARGET_FOLDER/tokenizer.model

Different models require different MP values:

Model	MP
7B	1
13B	2
30B	4
65B	8

Model Card

See MODEL_CARD.md.

License

See the LICENSE file.