llm-mcts-inference 0.1.0

An experimental project using Monte Carlo Tree Search (MCTS) to refine LLM responses for better accuracy and decision-making.

LLM MCTS Inference

An experimental project using Monte Carlo Tree Search (MCTS) to refine Language Model (LLM) responses for better accuracy and decision-making.

Overview

This project leverages MCTS to explore multiple answer candidates generated by a language model. By iteratively generating an initial answer, evaluating it, and refining it based on targeted feedback, the system strives to improve response quality and decision-making. This approach increases test time compute to produce more precise and robust model outputs.

Features

• Initial Answer Generation: Uses greedy decoding to generate an initial response.
• Feedback Generation: Provides constructive, concise feedback on generated answers.
• Iterative Improvement: Refines responses based on feedback through additional model queries.
• Monte Carlo Tree Search: Employs MCTS to explore and evaluate multiple answer paths.
• Structured Response Handling: Validates responses against JSON schemas using tools like pydantic.
• Modular Codebase: Organized into modules for inference, prompts, MCTS logic, configuration, and utilities.

Installation

Dependencies

• Python: Version 3.11 or higher

The project depends mainly on the following packages:

• instructor for guided generation
• litellm provides a unified API to interact with multiple LLM providers

Setup Instructions

1. Clone the Repository:

   git clone https://github.com/brotSchimmelt/llm-mcts-inference.git
   cd llm-mcts-inference
   

2. Install the Project Dependencies:

   If you use uv, run the following commands to create a virtualenv and install all requirements:

   uv venv --python 3.11
   uv sync
   

   Otherwise, install the required packages with pip:

   pip install -r pyproject.toml
   

3. Configure Environment Variables: Rename the provided example.env file to .env and update it with your API keys or other configuration details as needed.

Usage

Use the MonteCarloLLM class to generate and improve responses via MCTS:

from llm_mcts_inference.MonteCarloLLM import MonteCarloLLM

# Initialize with a specific model; defaults are defined in settings
llm = MonteCarloLLM(model_name="gpt-3.5-turbo")

# Define your prompt
prompt = "What is the capital of France?"

# Generate a response using Monte Carlo Tree Search
result = llm.generate(prompt)

# Output the final improved answer
print("Final Answer:", result.answer)

# Optionally, display the sequence of nodes (answers) along the best path
print("Best Path:", [node.answer for node in result.valid_path])

License

This project is licensed under the MIT license.