BilbyStats

A collection of statistical and machine learning functions for use in the Bilby pipeline

Install the package

Pip installation

The package installs a lightweight core by default, with heavier stacks available as optional extras.

pip install bilbystats

Install only the extras you need:

pip install "bilbystats[llm]"           # LLM API clients: OpenAI, Anthropic, Ollama, Groq, Gemini, tiktoken
pip install "bilbystats[transformers]"  # Transformer training/eval: torch, transformers, datasets, wandb, ... (heavy)
pip install "bilbystats[gcp]"           # Google Cloud / BigQuery
pip install "bilbystats[finance]"       # Finance / backtesting
pip install "bilbystats[scraping]"      # Scraping / HTML processing

Extras can be combined:

pip install "bilbystats[llm,gcp]"

Install every functional extra (llm + transformers + gcp + finance + scraping) with all:

pip install "bilbystats[all]"

Development-only tooling (spacy, geopandas, dspy, litellm, scikit-learn, selenium, ...) lives in the dev extra, which is not part of all:

pip install "bilbystats[dev]"

Quote the brackets ("bilbystats[llm]") so your shell doesn't try to glob them.

Alternative: local installation

To pull the repo run

git clone --depth=1 https://github.com/bilbyai/bilbystats/

Navigate to the root directory of the package and run

uv pip install .

or just

pip install .

if you don't have uv install. Extras work the same way for a local install, e.g. uv pip install ".[all]" or uv pip install -e ".[all]" for an editable install.

Set up API keys

API keys must be added in order to call LLMs. To do so they must be added to the environment. The simplest way to do so is to add lines of the following form to your .bashrc or .zshrc file.

export OPENROUTER_API_KEY=exampleapikey
export OPENAI_API_KEY=exampleapikey
export CLAUDE_API_KEY=exampleapikey

Where you should replace exampleapikey with the corresponding API key in each case.

Importing the package

Then the package can be imported from within python via e.g.

import bilbystats as bs

Setting local defaults

In order to set local defaults. E.g. for checkpoint saving etc. You'll need to navigate to

cp bilbystats/defaults/local_defaults_example.env bilbystats/defaults/local_defaults.env

and then change the default parameters and reinstall the package.

Run local LLMs using Ollama

If you'd like to use the Ollama functions which allow you to call LLMs on your local machine you'll need to install Ollama. Once you've installed ollama you can download LLMs such as llama3.2

ollama run llama3.2

or deepseek-r1:7b

ollama run deepseek-r1:7b

See https://ollama.com/search for a full list of the available models.

To call the LLM programmatically using bilbystats you can run

bs.llm_api('test call', 'you are an llm', 'llama3.2')

or

bs.llm_api('test call', 'you are an llm', 'deepseek-r1:7b')

Or in general use the model name in any llm related function such as bs.translate.