ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients" (ACL 26 Finding)

TODOs:

This is an official repo for <ToolGrad: Efficient Tool-use Dataset Generation with Textual “Gradients”>.

Get Started: A Quick Demo

Step 0: Install packages

Install nvm for MCP service:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
nvm install 20
nvm alias default 20

# verify npx
npx --version

Clone this repo and setup the uv env:

cd toolgrad
uv venv
uv sync
source .venv/bin/activate

Step 1: launch your first ToolGrad framework on a MCP service

# If you have no Gemini API key in your environment,
export GEMINI_API_KEY=YOUR_GEMINI_KEY

export PYTHONPATH=./
python examples/mcp_filesystem.py

During the execution, you should be able to see some useful logs to better understand the ToolGrad framework.

For example, the following is an example log of the API proposer.

INFO:root:[Iteration 2] Proposed 3 API proposals
INFO:root:  Proposal 1 (proposal_1): Read the content of the 'favorite_books.txt' file.
INFO:root:    └─ read_text_file
INFO:root:  Proposal 2 (proposal_2): Read the contents of 'favorite_books.txt', 'favorite_cities.txt', and 'favorite_songs.txt' simultaneously.
INFO:root:    └─ read_multiple_files
INFO:root:  Proposal 3 (proposal_3): List the contents of the current directory.
INFO:root:    └─ list_directory

The following is an example log of API executor.

INFO:root:[Iteration 2] Executed 3 proposals: 3 successful, 0 failed
INFO:root:  ✓ proposal_1: 1 tool call(s)
INFO:root:      Tool: read_text_file
INFO:root:      Input: {'path': 'favorite_books.txt'}
INFO:root:  ✓ proposal_2: 1 tool call(s)
INFO:root:      Tool: read_multiple_files
INFO:root:      Input: {'paths': ['favorite_books.txt', 'favorite_cities.txt', 'favorite_songs.txt']}
INFO:root:  ✓ proposal_3: 1 tool call(s)
INFO:root:      Tool: list_directory
INFO:root:      Input: {'path': '.'}

After the execution, you should be able to see the output data in examples/outputs/. It should look similar to examples/outputs/trace_example/00123.json and examples/outputs/example_seed=123__iter=3__num_apis=5.json.

ToolGrad-500

This is a TODO.

BibTex

@misc{zhou2025toolgradefficienttoolusedataset,
      title={ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"}, 
      author={Zhongyi Zhou and Kohei Uehara and Haoyu Zhang and Jingtao Zhou and Lin Gu and Ruofei Du and Zheng Xu and Tatsuya Harada},
      year={2025},
      eprint={2508.04086},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.04086}, 
}