NESTFUL

This is the official repository for NESTFUL.

Paper Title: NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls
Link: https://arxiv.org/abs/2409.03797v2
HuggingFace Data Link: https://huggingface.co/datasets/ibm-research/nestful

We have shared the latest NESTFUL evaluation set under data_v2 dir.

nestful_data.jsonl: It has 1861 evaluation data for nested sequencing.
executable_functions: Contains the implementation of all the functions in the benchmark.

The data_v1 directory includes the data for the previous version of the paper - link.

executable: contains data and spec with necessary information to execute them through RapidAPI.
non-executable: contains the nested sequencing data from SGD and GLAIVE that are hand-picked by human annotators from data synthetically generated using an LLM.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

nestful_wrapper-0.1.7.tar.gz (12.9 MB view details)

Uploaded Oct 9, 2025 Source

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

Uploaded Oct 9, 2025 Python 3

Details for the file nestful_wrapper-0.1.7.tar.gz.

Hashes for nestful_wrapper-0.1.7.tar.gz
Algorithm	Hash digest
SHA256	`65437ecabedf2e0e36239d9be67d293d3e6f64a8cd36124dcd571cc28db66bf5`
MD5	`d3eb18788d1882ea7ee1e27ee98f669c`
BLAKE2b-256	`b54a7865a872ff5fdf4a81c65dff1e939d155c4e446464cb96af2ecb64148ce1`

Details for the file nestful_wrapper-0.1.7-py3-none-any.whl.