needle-rs
Local tool calling with no server, no API key and no GPU. A compiled Rust extension — importing it costs milliseconds, not the seconds a JAX or PyTorch import takes, and it pulls in no Python ML dependencies at all.
This is the Python binding for needle-rs, a pure-Rust runtime for Cactus Compute's Needle tool-calling models. All three model generations are supported, and output is verified token-exact against the upstream JAX reference.
pip install needle-rs
Needle v2
One .cact file carries the weights, the geometry and the tokenizer.
from needle_rs import V2Engine
engine = V2Engine.load("weights/needle2.cact")
tools = """[{"name":"get_weather","description":"Get current weather for a city",
"parameters":{"type":"object","properties":{"city":{"type":"string"}},
"required":["city"]}}]"""
query = "What's the weather in Paris?"
out = engine.run(query, tools)
# <tool_call>[{"name":"get_weather","arguments":{"city":"Paris"}}]</tool_call>
# Gate execution on the model's confidence in the answer it just gave.
p = engine.confidence_for(query, tools, out)
if p is not None and p >= 0.5:
... # act on the call
else:
... # escalate
The confidence head scores a judgement already made, so confidence_for needs
the completion. Passing a bare query to the lower-level confidence() reads
near zero however answerable the query is.
Grammar-constrained decoding restricts the payload to the declared schema — valid tool names and argument keys only:
engine.generate(query, tools, max_new_tokens=96, temperature=0.0, constrain=True)
Temperature above zero samples, and seed makes that reproducible.
Needle v1
from needle_rs import NeedleEngine
engine = NeedleEngine.load("weights/needle.safetensors", "weights/vocab.txt")
result = engine.run("Book a flight from London to JFK tomorrow", tools)
engine.run_stream(query, tools, lambda tid, piece: print(piece, end="", flush=True))
engine.run_batch([(query1, tools1), (query2, tools2)])
v1 post-processes its output: the <tool_call> marker is stripped and your
original tool-name casing restored, so with run_stream the streamed pieces are
a progress view and the returned string is the answer.
Tool retrieval
Needle 2 and Needle 1 carry a contrastive head for narrowing a large catalogue
before routing. Embeddings are L2-normalised, so similarity is a plain dot
product. The published Needle 3 weights do not — they export only a
confidence head, so V3Engine has no retrieve_tools. Needle 3's architecture
defines an embedding head; it is not in this checkpoint.
engine.retrieve_tools(
"What's the weather in Paris?",
["Get current weather for a city", "Book a flight", "Send an email"],
top_k=2,
)
# [(0, 0.897), (2, 0.547)] — (index, score), descending
Weights
Weights are not bundled — download them once:
| Version | Files | Size | Source |
|---|---|---|---|
| v2 | needle2.cact |
13.7 MB | Cactus-Compute/needle2 |
| v1 | needle.safetensors + vocab.txt |
22 MB + 122 KB | Abdalrahman/needle-rs-safetensors |
from huggingface_hub import hf_hub_download
cact = hf_hub_download("Cactus-Compute/needle2", "needle2.cact")
Notes
- Needle is a tool-calling router, not a chat model: one query plus tool definitions in, one JSON call out. It will not produce useful free-form text.
- Single-shot. No multi-turn dialogue and no reasoning over tool results — your application executes the call and decides what to do with the response.
- English-trained; multilingual behaviour is not evaluated upstream.
- Constrained decoding guarantees syntactic validity, not semantic correctness.
Credit and license
This package is MIT. The models — architecture, training and weights — are the work of Cactus Compute and carry their own terms: Needle 3's weights are Apache-2.0, Needle 2's and Needle 1's are MIT, and the upstream repository is Apache-2.0. Check the licence on the generation you ship. If you publish work using them, please cite Needle (arXiv:2607.18363); the entry is in the repository README.
This package is the runtime only.
Metadata
Release files for needle-rs 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Actions Reset | |||
|---|---|---|---|---|
| needle_rs-0.3.1-cp38-abi3-win_amd64.whl | CPython 3.8 | abi3 | Windows x86-64 | Details for needle_rs-0.3.1-cp38-abi3-win_amd64.whl |
| needle_rs-0.3.1-cp38-abi3-musllinux_1_2_x86_64.whl | CPython 3.8 | abi3 | Linux musl 1.2+ x86-64 | Details for needle_rs-0.3.1-cp38-abi3-musllinux_1_2_x86_64.whl |
| needle_rs-0.3.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.8 | abi3 | Linux glibc 2.17+ x86-64 | Details for needle_rs-0.3.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
| needle_rs-0.3.1-cp38-abi3-macosx_11_0_arm64.whl | CPython 3.8 | abi3 | macOS 11.0+ ARM64 | Details for needle_rs-0.3.1-cp38-abi3-macosx_11_0_arm64.whl |
| needle_rs-0.3.1-cp38-abi3-macosx_10_12_x86_64.whl | CPython 3.8 | abi3 | macOS 10.12+ x86-64 | Details for needle_rs-0.3.1-cp38-abi3-macosx_10_12_x86_64.whl |
Total release size: 2.7 MB
Release files / needle_rs-0.3.1-cp38-abi3-win_amd64.whl
| Download URL | needle_rs-0.3.1-cp38-abi3-win_amd64.whl |
|---|---|
| Size | 466.5 kB |
| Tags | CPython 3.8 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
162924f093d25ec204a4f5fb14f55962500d0ab7dcc1a975069194dd9ec188b8
|
|
BLAKE2b-256 checksum How to use checksums |
4e96fcc1692ae220876c735bee6208fc886d3eb2c105165cac90e49cbff3d93e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / needle_rs-0.3.1-cp38-abi3-musllinux_1_2_x86_64.whl
| Download URL | needle_rs-0.3.1-cp38-abi3-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 619.4 kB |
| Tags | CPython 3.8 Linux musl 1.2+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
68cd9e2a747d3303a8c94b261ab304f62d93966fcdf1b5edbe8c885bf58b6537
|
|
BLAKE2b-256 checksum How to use checksums |
911ca4b930262fafe348beb0a84cba4217f2785625026294fdb214f7d98308f1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / needle_rs-0.3.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | needle_rs-0.3.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 557.7 kB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
7afd20bda58cbe612e59eb9fd9f43c81bf16607e741516ddb71208d1d72a7022
|
|
BLAKE2b-256 checksum How to use checksums |
d25fce5b4a52afcc920600ee7403da01e891b2b1fa3999fd570db796cae1a43a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / needle_rs-0.3.1-cp38-abi3-macosx_11_0_arm64.whl
| Download URL | needle_rs-0.3.1-cp38-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 487.0 kB |
| Tags | CPython 3.8 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
afd14efa4ccf289a590fccc18b8db1fbbb3a8c2e8891f3665d8a474787e3c230
|
|
BLAKE2b-256 checksum How to use checksums |
fbd022af2287ebea44c0f5fb8c67052ef0e33da509d16b4b88ccdfe88643383c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Release files / needle_rs-0.3.1-cp38-abi3-macosx_10_12_x86_64.whl
| Download URL | needle_rs-0.3.1-cp38-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 522.5 kB |
| Tags | CPython 3.8 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
ab2674e12b596f498ab4bd1e67dd1473e213a5ef7302071adb7c2b340067ce7e
|
|
BLAKE2b-256 checksum How to use checksums |
694c6a9f73c81cef156ff4114afd8579d1fece465cc5f8482b1f0c886d44426b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.16
|
Security for needle-rs 0.3.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.