NAT action tokenizer
Turns robot action chunks into discrete tokens for a vision-language-action model, and tokens back into actions. Every 32-step chunk of up to 7 action columns becomes 14 tokens, in time order, so a model can stream actions before it has generated the whole chunk.
Quick start
pip install nat-tokenizer
or from source: pip install "git+https://github.com/ahadjawaid/nat.git".
The weights and config download from the Hugging Face repository ahad-j/nat-action-tokenizer on first use.
import numpy as np
from nat_tokenizer import NATActionTokenizer
tokenizer = NATActionTokenizer.from_pretrained() # downloads ahad-j/nat-action-tokenizer
# 1. Fit the normalization once, on your dataset's actions (any shape ending in action_dim).
tokenizer.fit(dataset_actions)
# 2. Tokenize 32-step chunks: (batch, 32, action_dim) or one chunk (32, action_dim).
tokens = tokenizer(action_chunks) # list (one per chunk) of lists of ints in [0, 25600)
# 3. Decode back to actions in your original units.
actions = tokenizer.decode(tokens, action_dim=action_chunks.shape[-1]) # (batch, 32, action_dim)
# 4. Save the fitted tokenizer (keeps your min and max) and load it later the same way.
tokenizer.save_pretrained("my-nat-tokenizer")
tokenizer = NATActionTokenizer.from_pretrained("my-nat-tokenizer")
Normalization (on by default)
The tokenizer was trained on actions in [-1, 1]. With normalize=True (the default) every column is mapped from
its min and max to [-1, 1] before encoding (values outside are clipped) and decoded actions are mapped back, so you
work in your own units throughout. Set the min and max with fit(dataset_actions), or pass them directly:
tokenizer = NATActionTokenizer.from_pretrained(action_min=[...], action_max=[...])
If your actions are already in [-1, 1], pass normalize=False to both calls:
tokenizer(chunks, normalize=False) and tokenizer.decode(tokens, action_dim=d, normalize=False).
A column whose min equals its max is encoded as 0 and decoded back to its constant value.
Streaming (causal decoding)
Within a group the 14 tokens are time ordered: the first 4 decode the chunk's first 8 steps, the first 7 its first 16, the first 11 its first 24, and all 14 the whole chunk. A prefix decodes exactly the same first steps as the full chunk, so a model can execute actions while it is still generating:
for steps in (8, 16, 24, 32):
prefix = [chunk[:tokenizer.prefix_token_count(steps)] for chunk in tokens]
first_steps = tokenizer.decode(prefix, action_dim=7) # (batch, steps, 7)
Streaming works for actions of up to 7 columns (one group).
Action dimensions
Columns are split into consecutive groups of up to 7 (8 columns = 7 + 1, 14 = 7 + 7, 21 = 7 + 7 + 7). Each group gets
its own 14 tokens and a chunk's tokens are the groups' tokens one after another: tokenizer.tokens_per_chunk(action_dim)
gives the count (14 per group). Chunks must be exactly 32 steps long.
GPU and CUDA graphs
tokenizer = tokenizer.to("cuda") # eager on the GPU
tokenizer = tokenizer.to("cuda", cuda_graphs=True) # captures the encoder and decoder once per batch shape
Inputs and outputs stay NumPy. GPU results match the CPU up to floating point (rarely a token on a quantization boundary differs).
API
| call | does |
|---|---|
NATActionTokenizer.from_pretrained(repo_or_folder, action_min=None, action_max=None) |
load (optionally with your min and max) |
tokenizer.fit(actions) |
per-column min and max of your actions (returns the tokenizer) |
tokenizer(chunks, normalize=True) |
(batch, 32, d) or (32, d) actions to a list of token lists |
tokenizer.decode(tokens, action_dim, normalize=True) |
token lists (whole chunks, or one group's prefix) to (batch, steps, d) actions |
tokenizer.prefix_token_count(steps) |
tokens that decode the first steps (8, 16, 24 or 32) |
tokenizer.tokens_per_chunk(action_dim) |
14 per group of 7 columns |
tokenizer.to(device, cuda_graphs=False) |
run on a device, optionally with CUDA graphs |
tokenizer.save_pretrained(directory) |
save, with the fitted min and max |
tokenizer.vocab_size |
25,600 (4 scales x 6,400 codes) |
Model
4 latents per chunk, 4 FSQ scales of 6,400 codes (one coarse token per two latents, three finer tokens per latent),
trained on a large corpus of robot and human action data (every window once, batch 4,096). The repository holds only
the weights (model.safetensors) and a JSON config; the network is plain PyTorch in the nat-tokenizer package.
Reconstruction on robot benchmarks, and speed
Random 32-step chunks of each benchmark's demonstrations, each normalized by its own min and max (MetaWorld's actions clipped to the [-1, 1] its simulator executes). Errors are mean absolute differences between the input and its reconstruction. Speed is in milliseconds per call on an RTX 4090 (shared with another job while measured) and 4 CPU threads, mean and SD over 20 calls.
| benchmark | chunks x shape | MAE in [-1, 1] units | MAE | max error |
|---|---|---|---|---|
| LIBERO-10 | 1000 x (32, 7) | 0.0304 | 0.0210 | 1.9762 |
| MetaWorld mt4 (clipped) | 1000 x (32, 4) | 0.0651 | 0.0651 | 1.7578 |
| RoboMimic mt4 | 1000 x (32, 7) | 0.0466 | 0.0404 | 1.5712 |
| RoboCasa mt4 | 1000 x (32, 12) | 0.0130 | 0.0107 | 0.9970 |
| device | batch of 32 x 7 chunks | encode ms | decode ms |
|---|---|---|---|
| cpu, eager | 1 | 11.57 ± 0.39 | 29.86 ± 4.06 |
| cpu, eager | 64 | 322.06 ± 1.76 | 263.23 ± 6.67 |
| cuda, eager | 1 | 6.54 ± 1.67 | 11.07 ± 1.18 |
| cuda, eager | 64 | 7.86 ± 2.41 | 12.67 ± 1.80 |
| cuda, CUDA graphs | 1 | 0.71 ± 0.57 | 2.44 ± 0.39 |
| cuda, CUDA graphs | 64 | 1.05 ± 0.35 | 5.18 ± 2.09 |
Development
pip install -e ".[test]"
pytest
License
MIT, see LICENSE.
Metadata
Release files for nat-tokenizer 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| nat_tokenizer-0.1.0.tar.gz | 10.5 kB | Without provenance | Details for nat_tokenizer-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| nat_tokenizer-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for nat_tokenizer-0.1.0-py3-none-any.whl |
Total release size: 21.6 kB
Release files / nat_tokenizer-0.1.0.tar.gz
| Download URL | nat_tokenizer-0.1.0.tar.gz |
|---|---|
| Size | 10.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
050f31cae33d4772b6c1507dd27e43e38937f578a573ebed2fb763370da6e438
|
|
BLAKE2b-256 checksum How to use checksums |
50942a361cbf00ca905dfc101766ec4f18b2905c196edaa6f87c0a94f8eb328a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.2
|
Release files / nat_tokenizer-0.1.0-py3-none-any.whl
| Download URL | nat_tokenizer-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f4535ce9c0cdd3f235fd4e28fc98b26d10e653ef3ee94323ceb79b2b87cd6c1b
|
|
BLAKE2b-256 checksum How to use checksums |
2e26787a44f97b15f87b5b7bd40ad732007256edb6582fea9f0fe24f8a8e4feb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.2
|
Security for nat-tokenizer 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.