NDK-SDK

Natural-language tool calling for LLM agents — a Python implementation of the NLT (Natural Language Tools) approach.

Benchmark results

Tested on 5 free-tier models, 29 prompts, 7 tools. Baseline = standard JSON-schema tool calling. NLT = natural-language descriptions + ACTION: name(args) free-text output.

Model                          BL Tool% NLT Tool% BL Full% NLT Full% BL Malf% NLT Malf%
------------------------------------------------------------------------------------------
deepseek-v4-flash-free              83%       97%      66%       76%      17%        0%  ↑14%
lfm-2.5-1.2b-instruct:free          79%       93%      69%       72%      21%        7%  ↑14%
llama-3.2-3b-instruct:free          97%      100%      79%       76%       3%        0%  ↑3%
llama-3.3-70b-instruct:free         93%      100%      72%       76%       3%        0%  ↑7%
gpt-oss-20b:free                    86%      100%      72%       76%      14%        0%  ↑14%

Aggregate: Tool selection 88% → 98% (+10.3%) | Full success 72% → 75% (+3.4%) | Malformed 12% → 1% (-10.3%)

Smallest/weakest models benefit most — exactly as the paper predicts.

How it works

Quick start

pip install NDK-SDK

from ndk_sdk import NLTConverter, FreeTextParser

# Define your tools (standard OpenAI format)
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"}
            },
            "required": ["city"]
        }
    }
}]

# Convert to NL descriptions
converter = NLTConverter(tools)
system_msg = converter.to_openai_system_message()

# Use with any OpenAI-compatible API
response = client.chat.completions.create(
    model="your-model",
    messages=[system_msg, {"role": "user", "content": "Weather in Paris?"}]
)

# Parse the free-text response
parser = FreeTextParser()
tool_call = parser.parse(response.choices[0].message.content)
# -> ToolCall(name="get_weather", arguments={"city": "Paris"})

Two-phase selection (decoupled approach)

from ndk_sdk import ToolSelector, NLTConverter

converter = NLTConverter(tools)
selector = ToolSelector(converter)

# Phase 1: YES/NO — should we use a tool?
msgs = selector.build_selection_prompt("What's the weather in Paris?")
selection = selector.parse_selection(model_response)
# -> SelectionResult(use_tool=True, reasoning="...")

# Phase 2: Only if YES, pick the tool and args
if selection.use_tool:
    msgs = selector.build_tool_call_prompt("What's the weather in Paris?", selection.reasoning)
    tool_call = parser.parse(model_response)

Limitations

• Full-success gains are modest (+3.4%). NLT dramatically improves tool selection (88→98%), but argument correctness is the harder problem and only improves slightly. The model picks the right tool more often, but still sometimes gets the args wrong.
• The malformed-output improvement (12%→1%) is partly from the retry fallback. When the parser fails to extract an ACTION: line, NLT retries once with a clarifying prompt before counting it as a failure. Without the fallback, the malformed rate would be higher.
• Tested on 5 models, 29 prompts. Larger-scale benchmarks needed to confirm generalization.

Roadmap

• Improve argument extraction accuracy (the weakest dimension)
• Larger test set (50+ prompts, 10+ models)
• Structured argument parsing (e.g. guided generation for complex types)
• LangChain / MCP integration adapters

License

MIT