AI for the Church
Modern LLMs are rooted in secular value systems that are often misaligned with religious organisations. This PyPI package allows anyone to train and deploying doctrinally correct LLMs based on Llama2. Effectively, we are aligning models to a set of values.
Model fine-tuning
from aiforthechurch import align_llama2
doctrinal_dataset = "/path/to/csv"
align_llama2(doctrinal_dataset)
aiforthechurch is integrated with HuggingFace shuch that the aligned model will be automatically pushed to your HuggingFace repo of choice at the end of the training.
At aiforthechurch.org we provide tools for generating doctrinal datasets, a few examples are available at huggingface.co/AiForTheChurch.
Model inference and deployment
We implemented an inference API in the same format as OpenAI's.
import aiforthechurch
aiforthechurch.Completion.create(denomination="catholic", message="Does Jesus love me?")
There is also an asynchronous streaming API, just set stream=True.
And you can use our code to create your own inference server with the models
you train by editing the MODELS dictionary in
gloohack/deployment/prod_models.py with a denomination and a path to your
model in huggingface. Start the server by running the following command on your
machine:
python gloohack/deployment/inference.py
Model training requirements
If you wish to train your models using this repo you will need access to a machine with over 16GB of GPU memory and 30GB RAM. The full model weights for Llama2-7B amount to almost 30GB, but we use parameter-efficient fine-tuning (PEFT) LoRA to save memory and avoid any catastrophic forgetting during the fine-tuning procedure.
References
We leaned heavily on open-source libraries like transformers, peft, and
bitsandbytes for this project.
- Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. "LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale." arXiv preprint arXiv:2208.07339.
- Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. "LoRA: Low-Rank Adaptation of Large Language Models." arXiv preprint arXiv:2106.09685.
Metadata
Release files for aiforthechurch 0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aiforthechurch-0.5.tar.gz | 2.4 kB | Without provenance | Details for aiforthechurch-0.5.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aiforthechurch-0.5-py3-none-any.whl | Python 3 | none | any | Details for aiforthechurch-0.5-py3-none-any.whl |
Total release size: 4.8 kB
Release files / aiforthechurch-0.5.tar.gz
| Download URL | aiforthechurch-0.5.tar.gz |
|---|---|
| Size | 2.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
197731f022ab1543e6283734fd4b821dceddf8086638da517bea285e725f7fe6
|
|
BLAKE2b-256 checksum How to use checksums |
c0c94d373b941ac385ac730bc09d0f947ce9334fa8b6c2e1eb956f3915e56cc1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / aiforthechurch-0.5-py3-none-any.whl
| Download URL | aiforthechurch-0.5-py3-none-any.whl |
|---|---|
| Size | 2.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
78877f555f8a1d82a78a97e71661feb8da64b0a0849c6b7c79bd0acb9e7b75b3
|
|
BLAKE2b-256 checksum How to use checksums |
93d125c2b73d0425111fb9519e679166a4ccdf8985a46ae01ac3d65e9069aecf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Security for aiforthechurch 0.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.