ape-framework 1.0.0

Package for evaluating algebra problems using AI systems.

Algebra Problems Evaluator (APE)

What is APE?

APE is a framework that simplifies building and running mathematical benchmarks for AI systems.

It was initially meant as a way to evaluate mathematical problems from the field of algebra on whether they are suited to be a target goal for LLM reasoning research (hence the name). However it might be more useful to think about it the other way around - the subjects of the evaluation are LLMs or more broadly - AI systems and they are being evaluated on their ability to solve specific algebra problems. Problems, which solutions are hard to generate, but relatively easy to automatically check for correctness.

APE was created as a part of a Bachelor's project at the University of Warsaw.

User's Guide

[TODO]

Development setup

Install development dependencies:

pip install -r requirements-dev.txt

Install git hooks:

pre-commit install

Run all hooks manually:

pre-commit run --all-files