galtea 4.21.1

Galtea software development kit

Galtea SDK

Comprehensive AI/LLM Testing & Evaluation Framework

Overview

Galtea SDK empowers AI engineers, ML engineers and data scientists to rigorously test and evaluate their AI products. With a focus on reliability and transparency, Galtea offers:

1. Automated Test Dataset Generation - Create comprehensive test datasets tailored to your AI product
2. Sophisticated Product Evaluation - Evaluate your AI products across multiple dimensions

Documentation

All SDK usage and API documentation has moved to our official docs: Galtea SDK Documentation

---

Development

This project uses Poetry for dependency management and packaging.

Development Setup

# Activate the virtual environment (spawns a subshell)
poetry shell

# Install dependencies
poetry install

Exit the virtual environment with the exit command.

Using with Jupyter Notebooks

To use the Poetry virtual environment as a Jupyter kernel (e.g. for the example notebooks in examples/):

# Ensure dev dependencies are installed (includes ipykernel)
poetry install

# Register the virtualenv as a Jupyter kernel
poetry run python -m ipykernel install --user --name galtea-sdk --display-name "Galtea SDK"

Then select the "Galtea SDK" kernel in your notebook editor (VS Code, JupyterLab, etc.).

Running Tests

Tests are located in the tests/ directory and cover core functionality including file validation, string utilities, and model utilities.

Run All Tests

# Using Poetry (recommended)
poetry run pytest

# Or using Make
make test

# Using virtual environment directly
python -m pytest tests/

Run Tests with Verbose Output

poetry run pytest -v

Run Specific Test File

poetry run pytest tests/test_file_validation.py

Run Specific Test Class or Method

# Run a specific test class
poetry run pytest tests/test_file_validation.py::TestValidateKnowledgeBaseFile

# Run a specific test method
poetry run pytest tests/test_file_validation.py::TestValidateKnowledgeBaseFile::test_valid_txt_file

Run Tests with Coverage

poetry run pytest --cov=galtea --cov-report=html

This will generate a coverage report in the htmlcov/ directory.

Building the Project

To build the project:

poetry build

This will create distribution packages (wheel and source distribution) in the dist/ directory.

License

Apache License 2.0