Skip to main content

A Python package to reason by communicating with agents

Project description

ReComA: A Library for Reasoning via Communicating Agents

ReComA is a library designed to enable easy development of solutions for reasoning problems via communicating agents. It is a generalization of the codebase for Decomposed Prompting. The key features of the library:

  • A general-purpose framework that implements many existing approaches for reasoning via mutliple agents -- DecomP, ReACT, Least-to-Most, Faithful CoT
  • Can be easily extended to use other control flows (e.g., Self-Ask, IRCoT)
  • Provides an interactive GUI which includes the entire reasoning trace (with underlying prompts) for easy debugging
  • Built-in Best-First Search to explore multiple reasoning traces
  • Can be used as a pip-installable library in your own codebase
  • Configurable via JSONNET files -- no code change needed for many use cases

Table of Contents

Setup

If you want to directly make changes in this library, set it up using conda

  conda create -n recoma python=3.9
  conda activate recoma
  pip install -r requirements.txt

To install it as a dependency in your own conda environment

  pip install -e .

OpenAI Setup This library relies on the OPENAI_API_KEY environment variable to call GPT3+ models. Make sure to set this env. variable

  export OPENAI_API_KEY=<key>

Running ReComA

The library can be used to solve complex reasoning tasks in two modes:

Demo/Interactive Mode

 python -m recoma.run_inference \
  --config configs/inference/letter_cat/decomp.jsonnet \
  --output_dir output/letter_cat_decomp/ \
  --gradio_demo

This will start an interactive server on http://localhost:7860 for the kth letter concatenation task. Try the following question (no QID/Context needed):

Take the letters at position 3 of the words in "Reasoning via Communicating Agents" and concatenate them using a space.

The library will use text-davinci-002 model with Decomposed Prompting (specified via the input config file) to answer this question. You can open the collapsed nodes (indicated with ▶) to see the full execution trace (along with the prompts).

Batch Inference Mode

To use the library to produce predictions for an input file (e.g. the 3rd letter concatenation dataset with 4 words):

 python -m recoma.run_inference \
  --config configs/inference/letter_cat/decomp.jsonnet \
  --output_dir output/letter_cat_decomp/ \
  --input datasets/letter_cat/n4_eg100_pos2_space.json

Running this script will populate the output directory with :

  • predictions.json: qid-to-prediction map
  • all_data.jsonl: Input examples with model predictions and correctness label (using exact match)
  • html_dump/: Dump of the execution traces for all the examples in HTML format
  • source_config.json: JSON config used to run this experiment (for future reproducibility)

Using ReComA in your work

Using existing agents

If the provided agents are sufficient for your work, you can use this library by just defining the configuration files and prompts. See examples in the configs/ folder.

Defining a new agent

If you define a new agent (see the models README), you need to load them when running inference. Assuming your agents are defined under the package my_new_agents_pkg

python -m recoma.run_inference \
  --config configs/inference/letter_cat/decomp.jsonnet \
  --output_dir output/letter_cat_decomp/ \
  --input datasets/letter_cat/n4_eg100_pos2_space.json \
  --include_package my_new_agents_pkg

Please reach out if there are any questions or issues.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

recoma-0.0.4.tar.gz (33.0 kB view details)

Uploaded Source

Built Distribution

recoma-0.0.4-py3-none-any.whl (40.5 kB view details)

Uploaded Python 3

File details

Details for the file recoma-0.0.4.tar.gz.

File metadata

  • Download URL: recoma-0.0.4.tar.gz
  • Upload date:
  • Size: 33.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.14

File hashes

Hashes for recoma-0.0.4.tar.gz
Algorithm Hash digest
SHA256 1f7b8335e271c09868322222ae236258c55423aa3ac54629a167a63d48276aac
MD5 d5ddf13e28ec8f6f4e02c004a5adb553
BLAKE2b-256 900480d2f08f765ff2744e4f1075d2689db7d38ac09fa181733b4767a24050e0

See more details on using hashes here.

File details

Details for the file recoma-0.0.4-py3-none-any.whl.

File metadata

  • Download URL: recoma-0.0.4-py3-none-any.whl
  • Upload date:
  • Size: 40.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.14

File hashes

Hashes for recoma-0.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 c7bf72e8520c8432bd0ec608cf56742545e57c8d92e25377c13443e326db5de8
MD5 c09a2706485ae03b90068370ada74387
BLAKE2b-256 76b74fba3d495d665f8a5648e8c77352a5a19db9fb6605e41a90a92947167d22

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page