A project to show good CLI practices with a fully fledged RAG system.
Project description
A project to show good CLI practices with a fully fledged RAG system.
RAG CLI
Installation
pip install rag-cli
Commands
sudo apt-get update && sudo apt-get install parallel jq curl
mkdir -p data/articles data/embeddings
Get Wikipedia articles
parallel -n0 -j 10 '
curl -L -s "https://en.wikipedia.org/api/rest_v1/page/random/summary" | \
jq -r ".title, .description, .extract" | \
tee data/articles/$(cat /proc/sys/kernel/random/uuid).txt 1> /dev/null
' ::: {0..10}
Run embeder
parallel '
rag-cli embed --ollama-url http://localhost:11434 --file {1} 2>> output.log | \
jq ".embedding" | \
tee data/embeddings/$(basename {1} .txt) 1> /dev/null
' ::: $(find data/articles/*.txt)
Vector store
parallel rag-cli vector-store --qdrant-url http://localhost:6333 --collection-name nomic-embed-text-v1.5 2>> output.log ::: $(find data/embeddings/*)
Complete pipeline for single article
curl -L -s "https://en.wikipedia.org/api/rest_v1/page/random/summary" | \
jq -r ".title, .description, .extract" | \
rag-cli embed --ollama-url http://localhost:11434 2>> output.log | \
jq ".embedding" | \
rag-cli vector-store --qdrant-url http://localhost:6333 --collection-name nomic-embed-text-v1.5 2>> output.log
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
rag_cli-0.2.2.tar.gz
(16.5 kB
view hashes)
Built Distribution
rag_cli-0.2.2-py3-none-any.whl
(16.9 kB
view hashes)