Run AI locally with as little friction as possible.
Project description
Cria, use Python to run LLMs with as little friction as possible.
Cria is a library for programatically running Large Language Models through Python. Cria is built so you need as little configuration as possible — even with more advanced features.
- Easy: No configuration is required out of the box. Defaults are built in, so getting started takes just five lines of code.
- Concise: Write less code to save time and avoid duplication.
- Efficient: Use advanced features with your own
ollama
instance, or a subprocess.
Guide
Quickstart
Running Cria is easy. After installation, you need just five lines of code — no configurations, no manual downloads, and no servers to worry about.
import cria
ai = cria.Cria()
prompt = "Who is the CEO of OpenAI?"
for chunk in ai.chat(prompt):
print(chunk, end="") # The CEO of OpenAI is Sam Altman!
# Not required, but best practice.
ai.close()
By default, Cria runs llama3:8b
. If llama3:8b
is not installed on your machine, Cria will install it automatically.
Important: If the default model is not installed on your machine, downloading will take a while (llama:8b
is about 4.7GB).
Installation
-
Cria uses
ollama
, to install it, run the following.Windows
Mac
Linux
curl -fsSL https://ollama.com/install.sh | sh
-
Install Cria with
pip
.pip install cria
Advanced Usage
Custom Models
To run other LLM models, pass them into your ai
variable.
import cria
ai = cria.Cria("llama2")
prompt = "Who is the CEO of OpenAI?"
for chunk in ai.chat(prompt):
print(chunk, end="") # The CEO of OpenAI is Sam Altman. He co-founded OpenAI in 2015 with...
You can find available models here.
Streams
Streams are used by default in Cria, but you can turn them off by passing in a boolean for the stream
parameter.
prompt = "Who is the CEO of OpenAI?"
response = ai.chat(prompt, stream=False)
print(response) # The CEO of OpenAI is Sam Altman!
Closing
By default, models are closed when you exit the Python program, but closing them manually is a best practice.
ai.close()
Message History
Message history is automatically saved in Cria, so asking followup questions is easy.
prompt = "Who is the CEO of OpenAI?"
response = ai.chat(prompt, stream=False)
print(response) # The CEO of OpenAI is Sam Altman.
prompt = "Tell me more about him."
response = ai.chat(prompt, stream=False)
print(response) # Sam Altman is an American entrepreneur and technologist who serves as the CEO of OpenAI...
Clearing history is available as well.
prompt = "Who is the CEO of OpenAI?"
response = ai.chat(prompt, stream=False)
print(response) # Sam Altman is an American entrepreneur and technologist who serves as the CEO of OpenAI...
ai.clear()
prompt = "Tell me more about him."
response = ai.chat(prompt, stream=False)
print(response) # I apologize, but I don't have any information about "him" because the conversation just started...
You can also create a custom message history, and pass in your own context.
context = "Our AI system employed a hybrid approach combining reinforcement learning and generative adversarial networks (GANs) to optimize the decision-making..."
messages = [
{"role": "system", "content": "You are a technical documentation writer"},
{"role": "user", "content": context},
]
prompt = "Write some documentation using the text I gave you."
for chunk in ai.chat(messages=messages, prompt=prompt):
print(chunk, end="") # AI System Optimization: Hybrid Approach Combining Reinforcement Learning and...
In the example, instructions are given to the LLM and context is given as the user, before the user prompts. You can use any mixture of roles to specify the LLM to your liking.
The available roles for messages are:
user
- Pass prompts as the user.system
- Give instructions as the system.assistant
- Act as the AI assistant yourself, and give the LLM lines.
The prompt parameter will always be appended to messages under the user
role, to override this, you can choose to pass in nothing for prompt
.
Multiple Models and Parallel Conversations
If you are running multiple models or parallel conversations, the Model
class is also available. This is recommended for most usecases.
import cria
ai = cria.Model()
prompt = "Who is the CEO of OpenAI?"
response = ai.chat(prompt, stream=False)
print(response) # The CEO of OpenAI is Sam Altman.
All methods that apply to the Cria
class also apply to Model
.
Multiple models can be run through a with
statement. This automatically closes them after use.
import cria
prompt = "Who is the CEO of OpenAI?"
with cria.Model("llama3") as ai:
response = ai.chat(prompt, stream=False)
print(response) # OpenAI's CEO is Sam Altman, who also...
with cria.Model("llama2") as ai:
response = ai.chat(prompt, stream=False)
print(response) # The CEO of OpenAI is Sam Altman.
Or they can be run traditonally.
import cria
prompt = "Who is the CEO of OpenAI?"
llama3 = cria.Model("llama3")
response = llama3.chat(prompt, stream=False)
print(response) # OpenAI's CEO is Sam Altman, who also...
llama2 = cria.Model("llama2")
response = llama2.chat(prompt, stream=False)
print(response) # The CEO of OpenAI is Sam Altman.
# Not required, but best practice.
llama3.close()
llama2.close()
Cria can also has a generate
method.
prompt = "Who is the CEO of OpenAI?"
for chunk in ai.generate(prompt):
print(chunk, end="") # The CEO of OpenAI (Open-source Artificial Intelligence) is Sam Altman.
promt = "Tell me more about him."
response = ai.generate(prompt, stream=False)
print(response) # I apologize, but I think there may have been some confusion earlier. As this...
Running Standalone
When you cria.Cria()
, an ollama
instance will start up if one is not already running. When the program exits, this instance will terminate.
To prevent this behaviour, either run your own ollama
instance in another terminal
ollama serve
or use the following code.
ai = cria.Cria(standalone=True, close_on_exit=False)
prompt = "Who is the CEO of OpenAI?"
# Ollama will already be running.
with cria.Model("llama2") as llama2:
response = llama2.generate("Who is the CEO of OpenAI?", stream=False)
print(response)
with cria.Model("llama3") as llama3:
response = llama3.generate("Who is the CEO of OpenAI?", stream=False)
print(response)
quit()
# Olama will keep running, and will be used the next time this program starts.
Contributing
If you have a feature request, feel free to make an issue!
Contributions are highly appreciated.
License
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.