llm-rsa 0.0.2

Recursive self aggregation

RSA - Recursive Self-Aggregation

RSA implements Recursive Self-Aggregation, a technique for improving LLM responses by generating multiple candidate answers and iteratively aggregating them. The algorithm samples k candidates from a pool of M responses, asks the LLM to synthesize an improved answer, and repeats this process across multiple loops to converge on higher-quality outputs.

Developer Guide

If you are new to using nbdev here are some useful pointers to get you started.

Install in Development mode

# make sure  package is installed in development mode
$ pip install -e .

# make changes under nbs/ directory
# ...

# compile to have changes apply to 
$ nbdev_prepare

Usage

Installation

Install latest from the GitHub repository:

$ pip install git+https://github.com//.git

or from conda

$ conda install -c  

or from pypi

$ pip install 

Documentation

Documentation can be found hosted on this GitHub repository’s pages. Additionally you can find package manager specific guidelines on conda and pypi respectively.

How to use

Basic Usage

Create an RSA instance with your task prompt and call it to run the aggregation:

task_prompt = '''Three people check into a hotel room that costs $30. They each contribute $10. 
Later, the manager realizes the room only costs $25 and gives $5 to the bellboy to return. 
The bellboy keeps $2 and gives $1 back to each person. 
So each person paid $9 (total $27), plus the bellboy has $2, which equals $29. 
Where did the extra dollar go?'''

agg_prompt = """Below is a reasoning problem followed by several candidate solutions. 
Your job is to:
1. Carefully analyze each candidate's reasoning step-by-step
2. Identify which candidates make logical errors or arithmetic mistakes  
3. Note which approaches lead to correct reasoning
4. Synthesize the best reasoning into a single, clear, correct solution

Show your work step-by-step, then state your final answer clearly."""

from llm_rsa.core import RSA

# Create RSA instance with a reasoning task
rsa = RSA(
    task_prompt=task_prompt,
    agg_prompt=agg_prompt, 
    M=4,
    k=2,
    loops=3
)

# Run the aggregation
results = rsa.run()
print(f"Generated {len(rsa.history)} total candidates across {rsa.loops} loops")
print('llm response: \n', results[-1].response)

Generated 12 total candidates across 3 loops
llm response: 
 ### Analysis of Candidate Reasoning

Both **Candidate 1** and **Candidate 2** provide identical logical conclusions and correctly identify the fallacy.
*   They both recognize that $27 is the total amount the guests spent ($30 - $3 refund).
*   They both correctly point out that the $2 held by the bellboy is a *subset* of that $27, not an additional amount to be added to it.
*   They both demonstrate that the correct way to reach the original $30 is to add the $3 refund to the $27 spent, rather than adding the $2 tip to the $27 spent.

The candidates effectively "debunked" the riddle's misdirection, which relies on the psychological trick of adding two numbers that do not belong together in a balance sheet.

---

### Step-by-Step Reasoning and Solution

To solve the mystery of the "missing dollar," we must track the $30 carefully and distinguish between **Assets** (money held) and **Expenses** (money spent).

**1. Track the $30 Total**
At the end of the transaction, the $30 is distributed as follows:
*   **$25** is in the hotel's register (the actual price of the room).
*   **$2** is in the bellboy's pocket (the stolen tip).
*   **$3** is in the guests' pockets ($1 each).
*   **Total: $25 + $2 + $3 = $30.**
Nothing is missing.

**2. Analyze the Guest Perspective (The $27)**
The riddle says: "Each person paid $9, total $27." This is correct. Let's look at what happened to that $27:
*   **$25** went to the hotel for the room.
*   **$2** went to the bellboy as a tip.
*   **Total: $25 + $2 = $27.**
The $2 is already *inside* the $27.

**3. Identify the Logical Fallacy**
The riddle's error is the statement: *"Each person paid $9 (total $27), plus the bellboy has $2, which equals $29."*
This is an accounting error. You cannot add the bellboy's $2 to the $27 because the bellboy's $2 is **part of** the $27. 

To reconcile the total to $30, you must add the money the guests **kept** (the $3 refund) to the money they **spent** ($27):
*   **$27 (Spent) + $3 (Refunded) = $30.**

**Conclusion:**
The "extra dollar" does not exist. The riddle creates an illusion by adding a component of an expense ($2) to the total expense ($27), rather than adding the remaining cash on hand ($3) to the total expense.

### Final Answer
The dollar is not missing. The mistake is in the calculation: it adds the bellboy's $2 to the $27 spent, even though the $2 is already included in the $27. The correct calculation is $27 (spent) + $3 (returned to guests) = $30.

from litellm import completion

# Single direct call (baseline)
response = completion(
    model='openrouter/google/gemini-3-flash-preview',
    messages=[{"role": "user", "content": task_prompt}],
    temperature=1.0
)
baseline_answer = response.choices[0].message.content
print("=== BASELINE (single call) ===")
print(baseline_answer)

=== BASELINE (single call) ===
The extra dollar didn't go anywhere; the confusion comes from **adding** the bellboy's tip to the guests' expenses instead of **subtracting** it.

Here is the correct breakdown of the math:

### 1. The Total Spent
Each person paid $9, for a total of **$27**.

### 2. Where that $27 is currently located
Of that $27:
*   **$25** is in the cash register (the actual price of the room).
*   **$2** is in the bellboy’s pocket.
*   **Total: $27.**

### 3. The Logical Fallacy
The riddle tricks you by saying: *"$27 (paid) + $2 (bellboy) = $29."* 

This is an error in logic because the **$2 is already included in the $27**. You are essentially adding the bellboy's tip twice. 

**The correct math should be:**
*   **Total Spent ($27) + Total Refunded ($3) = $30**
*   OR
*   **Total Spent ($27) - Bellboy's Tip ($2) = Room Price ($25)**

Configuration Options

Parameter	Default	Description
task_prompt	(required)	The main task/question to solve
model	'openrouter/google/gemini-3-flash-preview'	LLM model to use (any litellm-compatible model)
M	8	Number of candidates generated per loop
k	4	Number of candidates sampled for each aggregation
loops	3	Number of aggregation iterations
temperature	1.0	LLM sampling temperature
n_workers	4	Parallel workers for LLM calls
agg_prompt	(auto)	Custom aggregation prompt (optional)

How RSA Works

1. Loop 0: Generate M independent responses to the task prompt
2. Loop 1+: For each of M new candidates, randomly sample k previous candidates and ask the LLM to aggregate them into an improved answer
3. Repeat for the specified number of loops
4. Return the final pool of aggregated candidates

The history attribute stores all candidates across all loops, allowing you to trace the aggregation process.