twentyq 0.0.7

20 Questions Game using LLMs

TwentyQ: The 20 Questions Game Engine (Python Package)

This pip package enables users to simulate and evaluate a 20 Questions-style guessing game using Large Language Models (LLMs) such as OpenAI GPT, Anthropic Claude, Google Gemini, and Hugging Face-hosted models. This package forms the backbone of the experiments in the paper: The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities

---

File Structure

.
├── LICENSE                   # MIT License for the package
├── pyproject.toml           # Configuration file for pip installation
├── README.md                # Documentation for usage
└── twentyq/                     # Core package directory
    ├── __init__.py          # Exports play_game for public use and loads environment variables
    ├── play.py              # Main logic for running and saving the game
    ├── game.py              # Game logic, turn simulation, message storage
    ├── utils.py             # Helper functions for model loading and utilities
    └── eval_result.py       # Evaluation logic for computing metrics from saved game

---

Installation

pip install twentyq

---

Example Usage

from twentyq import play_game

result = play_game(
    name="Lionel Messi",
    guesser_model_name="gpt-4o",          # Supports OpenAI, Claude, Gemini, Hugging Face
    num_turns=20,
    temperature=0.8,
    language="english",                  
    base_output_dir="./outputs",
    game_type="celebrities"               # "celebrities" or "things"
)

---

Input Parameters

• name: (str) – The target entity (celebrity or thing) to be guessed (e.g., "Lionel Messi").
• guesser_model_name: (str) – Model to be used. Supported:
  • OpenAI models (e.g., gpt-4o)
  • Claude (e.g., claude-3-5-sonnet-latest)
  • Gemini (e.g., gemini-2.0-flash)
  • Hugging Face models (e.g., meta-llama/Llama-3.3-70B-Instruct)
• num_turns: (int) – Number of allowed turns (e.g., 20).
• temperature: (float) – Sampling temperature for the guesser model (e.g., 0.8).
• language: (str) – The language to conduct the game in. Supported:
  english, hindi, mandarin, spanish, japanese, turkish, french
• base_output_dir: (str) – Base path for saving results.
• game_type: (str) – "celebrities" or "things" depending on the entity type.

---

API Keys

Regardless of which model is used, the environment must have the following keys set:

• OPENAI_API_KEY – (mandatory)
• If using Hugging Face models: HUGGINGFACE_TOKEN
• If using Gemini: GENAI_API_KEY
• If using Claude: ANTHROPIC_API_KEY

You can either export these directly in your shell:

export OPENAI_API_KEY=your_key_here
export HUGGINGFACE_TOKEN=your_token_here

Or save them in a .env file in your project directory:

OPENAI_API_KEY=your_key_here
HUGGINGFACE_TOKEN=your_token_here

---

Output

• A folder named like {guesser_model_name}_{num_turns}_turns_{language} will be created inside base_output_dir (e.g., gpt-4o_20_turns_english)
• Inside this folder, a .txt file named after the entity (e.g., Lionel Messi.txt) will contain the full game conversation.

./outputs/gpt-4o_20_turns_english/
├── Lionel Messi.txt   # Full dialogue from the 20Q game  

---

Return Value

The play_game function returns a dictionary like:

{
  "Lionel Messi": {
    "guesser": [
      "Is the person a footballer?",
      "Has he won a Ballon d'Or?",
      "Is it Lionel Messi?"
    ],
    "answerer": [
      "Yes.",
      "Yes.",
      "Bingo!"
    ],
    "Success": True,
    "Has_Given_Up": False,
    "Turn_At_Which_Given_Up": "N/A",
    "Num_Turns_To_Answer": 3,
    "Num_Yes": 3
  }
}

If Has_Given_Up is True, then Num_Turns_To_Answer will be "N/A".
If Success is True, then Turn_At_Which_Given_Up will be "N/A".

---

Design Notes

To ensure consistency and mitigate knowledge disparities, the same model is used for both the Guesser and Judge roles.

We use a temperate of 0.2 for the Judge to ensure it provides consistent and deterministic responses.

We use max_tokens of 100 for the Guesser to allow for concise responses while still being able to ask follow-up questions

We use max_tokens of 30 for the Judge to ensure it just provides a simple "Yes", "No", or "Maybe/Dunno" response.

---

Citation

If you use this python package, please cite:

@inproceedings{lalai20q,
  title={The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities},
  author={Lalai, Harsh Nishant and Shah, Raj Sanjay and Pei, Jiaxin and Varma, Sashank and Wang, Yi-Chia and Emami, Ali},
  booktitle={Second Conference on Language Modeling}
}