Komorebi
Move the data, move the thread, or move the code.
A programming model for a network of independently programmable processors, in which a thread reads what it needs or goes to where it is. Today it runs on the Tensix grid and the X280 cores of a Tenstorrent Blackhole card.
Komorebi explores how to program a network of independently programmable processors, where computation, communication, code placement and execution control are all the program's own concern. It sees a program as threads, and a thread meets its data in one of two ways:
- The data comes to the thread. A processor reads what it needs from shared memory, or receives it from other processors.
- The thread goes to the data. A message carries a thread's state to the processor that holds what it needs next, and that processor is chosen while the program runs, from the data.
A thread needs its code where it arrives: placed there in advance, or loaded when it is needed. Nothing in this asks for one particular chip. It asks for processors with their own memory and a network on which any of them can reach any other, so other grids of this kind, and several cards joined together, fit the same description. Two cards joined together run today; no other grid has been tried.
On a Blackhole card, today
This repository is the model built for one machine, a Tenstorrent Blackhole card (developed on a P150A): 120 Tensix cores in a grid, 16 X280 cores in four clusters, and the card's network between them. Two such cards joined by their Ethernet links run as one program.
| Concern | On the card |
|---|---|
| Computation | A program is a set of persistent kernels on every Tensix core. Each command they receive names a handler of code and a core group, and every core of the group works on its partition. |
| Communication | Cores read and write DRAM, and send each other data through ports, reduces and broadcasts. A routed message goes to a core chosen at run time. Both data-movement CPUs of a core can carry one stream (Movement). |
| Code placement | A program's kernels are loaded on every core when the run starts. Code that is not in them, an overlay, is built on the host and kept in DRAM; a core loads it into a slot of its own L1 when it is first needed, and calls it. A handler can name it, a message (its handler is loaded where the message arrives), a leader's cast, or the word of a pull's answer from an X280 core. A program can itself be an overlay under kernels the runtime owns: several programs share one run, each on its own group of cores or all on every core, and run.load (or an X280 thread's kmx_load) gives a group another program while the run is live (overlays). |
| Execution control | A dispatcher on one X280 cluster submits the commands and runs streams and graphs of them on the card. C threads on the other three X280 clusters run the irregular part of a program. |
| More than one card | Each card is an island with its own grid and dispatcher. Ranks stay flat across islands, the host places each tensor by a hint, and commands, pages, parts of a page, messages and the parts of a reduce or a broadcast cross the links between the cards. The kernels of a program written for one card run on two unchanged (islands). |
What is not there in code placement: the dynamic handlers of two programs in one run; a
message with a body in a run that has X280 cores as ranks (a message carries a
body between Tensix cores, and X280 cores are
ranks of a run whose messages are one line); a switch of program in a run where every
core holds every program; a load issued by a cast or a pull (the host issues
run.load, an X280 thread kmx_load); a switch of a dynamic program; and what the old
program kept in its buffers, which a switch loses.
Status: early development; 0.2.0 is the second release, for one card model. Everything below runs on one P150A, from the two wheels on PyPI or from a clone against a tt-metal source tree; two P150As as two islands run from a clone. It has run on no other hardware.
New in 0.2.0:
- Code goes where the work is. Overlays loaded into slots of a core's L1 and called;
a handler, a message, a leader's cast or an X280 core's answer names the code; a group
changes program inside a run, from the host (
run.load) or from an X280 thread (kmx_load). - Two cards as one program. Islands with flat ranks, tensors placed by a hint, and commands, pages, parts of a page, messages, reduces and broadcasts across the links. One page call reads or writes a tensor wherever its pages are.
- Messages carry more. A body of up to 4 KiB beside a message; X280 cores as ranks of a dynamic run.
- A thread reaches any DRAM.
kmx_fetchandkmx_storeof any bank, or of a tensor another card holds, at the cores' rate. - The host at the link's speed. Tensor reads and writes go through the host channel: a 20 MiB frame read in 7 ms inside a run, where the device API took 10 s.
- A run survives a stuck core.
run.inspectof a running core; a command given up at its deadline; one core reset and loaded again by the dispatcher, by itself. - The X280 clusters at 1,350 MHz by default (1,000 before), and a mover's NoC as a field of the program.
At a glance
One P150A, 2026-11-13 and 2026-11-14; every number is a row of Measured, from the program's page and its committed results.
| Program | Shape | Komorebi | Reference | Reference / Komorebi |
|---|---|---|---|---|
| Matrix product | 128 x 896 x 128 | 6.0 µs | Kohaku-TT, same card: 28.7 µs | 4.75 |
| Matrix product | 8192 cubed | 8608.3 µs | Kohaku-TT, same card: 11797.6 µs | 1.37 |
| Matrix product | 16,384 cubed | 87944.5 µs | Kohaku-TT, same card: 75522.5 µs | 0.86 |
| SwiGLU | 256 x 256 x 512 | 42.6 µs | Kohaku-TT, same card: 181.6 µs | 4.26 |
| SwiGLU | 16,384 x 384 x 384 | 161.7 µs | Kohaku-TT, same card: 458.1 µs | 2.83 |
| Attention | 2 x 8 x 1024 x 128, 2 K / V heads | 149.9 µs | Kohaku-TT, same card: 261.8 µs | 1.75 |
| Attention | 1 x 8 x 1024 x 64, causal | 58.1 µs | Kohaku-TT, same card: 53.7 µs | 0.92 |
| Rasterizer, whole frame | 10^6 triangles, 1024 x 1024 | 6.34 ms | CUDA on a V100: 4.42 ms | 0.70 |
| Ray tracer, whole frame | Sponza close, 1024 x 1024 | 359.86 ms | CUDA on a V100: 102.00 ms | 0.28 |
Two P150A joined by four links, 2026-11-15: a linear layer Y = X W in bfloat16, single
calls, from the dense page.
"Speed-up" is one card's time over two cards'.
| Linear layer | Cube | One card | Two cards | Speed-up | Kohaku-TT, two cards, same form |
|---|---|---|---|---|---|
| W gathered on both cards | 8192 | 4,488.1 µs | 2,357.6 µs | 1.90 | 2,482.8 µs (1.83) |
| W gathered on both cards | 16,384 | 34,544.8 µs | 17,731.5 µs | 1.95 | 17,067.8 µs (1.90) |
Above 1, Komorebi takes less time. The neural workloads are on parallel with or ahead of Kohaku-TT on most shapes; the rasterizer and the ray tracer run whole frames on the card, bit-exact against numpy, today at 0.4 to 0.9 of the V100's speed from 10^5 triangles up (the rasterizer) and a third or less (the ray tracer).
A kernel
A kernel is loaded once on its core and stays there: run calls the function below for
every command the core receives. A program is a data-movement kernel and a compute
kernel.
The example computes the gate of a SwiGLU layer, out = silu(a) * b, over two tensors
in DRAM. A command goes to a core group; each core takes its share of the tiles. Its
two data-movement CPUs, the movers, bring that share of a and b from DRAM into L1
and take out back, sharing every stream by a pattern (AAAB: mover A three of every
four blocks, mover B the fourth). Its compute kernel does the math in the vector unit,
a block of tiles at a time (kBlock, 8: the tiles its registers hold). Both files take
the three buffer pairs and kBlock from a header that the host reads too. The two
kernels below are reduced from 22_gate, which also checks
every wait and is run against numpy:
// ---- kernels/movement.cpp: one file, built for both data-movement CPUs of a core ----
void kernel_main() {
constexpr auto tensors = TensorAccessorArgs<0>(); // a, b and out share one layout
km::Movement movement; // stays loaded; runs once per command
movement.run([&](const km_command_t& command) {
const uint32_t tiles = command.a3; // tiles a core
const uint32_t first = (movement.me() - km::first(command)) * tiles;
const km::Pattern pattern = km::Pattern::of(command.a4);
const auto stream = [&](uint32_t direction, uint32_t address, km::BufferPair pair) {
const TensorAccessor tensor(tensors, address);
return km::Stream(direction, tensor, first, tiles, kBlock, 1, pair, pattern);
};
return movement.move(
stream(km::kMoveRead, command.a0, kA),
stream(km::kMoveRead, command.a1, kB),
stream(km::kMoveWrite, command.a2, kOut)
);
});
}
// ---- kernels/compute.cpp: out = silu(a) * b in the vector unit, a block at a time ----
void kernel_main() {
km::Compute compute;
compute_kernel_hw_startup(kAMoverA, kOutMoverA);
compute.run([&](const km_command_t& command) {
const km::Pattern pattern = km::Pattern::of(command.a4);
for (uint32_t block = 0; block < command.a3 / kBlock; block++) {
// each block from the half of the buffer pair its mover filled
const uint32_t a = km::block_half(kA, pattern, block);
const uint32_t b = km::block_half(kB, pattern, block);
const uint32_t out = km::block_half(kOut, pattern, block);
compute.wait_front(a, kBlock);
compute.wait_front(b, kBlock);
tile_regs_acquire();
copy_init(a);
for (uint32_t tile = 0; tile < kBlock; tile++) copy_tile(a, tile, tile);
silu_tile_init();
for (uint32_t tile = 0; tile < kBlock; tile++) silu_tile(tile);
mul_reuse_dest_init<kDestToSourceA>(b);
for (uint32_t tile = 0; tile < kBlock; tile++) {
mul_reuse_dest_tiles<kDestToSourceA>(b, tile, tile);
}
tile_regs_commit();
tile_regs_wait();
compute.reserve_back(out, kBlock);
for (uint32_t tile = 0; tile < kBlock; tile++) pack_tile(tile, out);
tile_regs_release();
cb_push_back(out, kBlock);
cb_pop_front(a, kBlock);
cb_pop_front(b, kBlock);
}
});
}
The host declares the program, loads it and submits the command to a core group:
program = km.Program(
"gate",
kernels="kernels/", # movement.cpp for both movers, compute.cpp
buffers=[
km.BufferPair(0, 1, "bfloat16", 2048, pages=32), # a: mover A's, mover B's
km.BufferPair(2, 3, "bfloat16", 2048, pages=32), # b
km.BufferPair(16, 17, "bfloat16", 2048, pages=32), # out
],
compile_args={"movement": a.accessor_args()}, # how the kernel finds a, b and out
handlers="kernels/gate.h", # kHandlerGate = 1
)
run = session.load(program)
pattern = km.Pattern("AAAB")
words = [a.address, b.address, out.address, tiles, pattern.word]
run.wait(run.submit("gate", (0, 120), words))
The part of a program that decides is C on an X280 work cluster. A thread that splits the grid between two programs every round, after one command from the host (reduced from 19_thread_programs):
const uint32_t spin_cores = split(cores, spin_units, spin_rate, tiles, scale_rate);
km_command_t spin, scale;
kmx_command(&spin, kProgramSpin, kProgramSpinHandlerSpin, 0, spin_cores);
kmx_command(&scale, kProgramScale, kProgramScaleHandlerScale, spin_cores, cores - spin_cores);
kmx_submit(&spin, &spin_handle);
kmx_submit(&scale, &scale_handle);
kmx_wait(spin_handle, deadline, &spin_event);
kmx_wait(scale_handle, deadline, &scale_event);
Code that is not in the kernels is an overlay, loaded into a slot of a core's L1 while the run lives. Every example is a page under examples, with what it measured.
The programming model
The card as Komorebi runs it
The grid is the card's real one in NoC0 coordinates; the numbers in the Tensix cores are Komorebi's core index. A command is one 64-byte line: the handler, the group (a contiguous range of cores, whose first core is its leader) and ten 32-bit argument words. Every core sees every command and skips those whose group does not hold it; cores report to the leader, which acknowledges to the dispatcher, with a result word when the command asks for one.
Commands are one way cores talk. The figure draws each of the others once, in its own colour; the next two figures take them one at a time, with the code that does each.
A thread has a program counter and a place
Each panel is one way cores talk, on the piece of the grid it uses, with lines of the example that does it (the example's name is in the panel's corner). A breakpoint is a point where a thread must deal with another core; between two breakpoints it runs on its core. Today a program is written as the handler bodies of its kernels, as above.
Code goes where the work is
An overlay is kernel code that is not in the kernels when the run starts: the host
builds it, and a core loads it into a slot of its L1 and calls it; a held
overlay is called again without a load (overlays,
12_overlays). The code can be decided where the work is: a
message names its handler and the core it reaches loads that handler's part if it does
not hold it (20_message_overlays), or another party
decides, a leader casting the overlay's number
(17_leader_overlay), an X280 core's answer to a pull
naming it (18_thread_split), run.load making a group hold
another program (16_two_programs). The cycles in the figure
are those of the overlay costs and of
a group switching program; the
numbers measured on one card are below.
Groups and commands in time
Up to 64 commands are in flight at once; a run's window is 4 unless it asks for more. A
command that must run after an earlier one on overlapping cores is submitted with
run.submit(..., exclusive=True).
A command need not come from the host, and the code it runs need not be fixed when the run starts. In the second panel the host sends one command and an X280 thread splits the grid between two programs every round (19_thread_programs); in the third a group's leader casts which overlay every core runs next (17_leader_overlay).
One program on the grid
The first panel is the simplest program, one command to a core group (01_mapcollect). The others are one program that uses most of the model (18_thread_split): every core pulls ranges of work from its X280 core, the answer names the overlay for that range, and the core loads it when it does not hold it and calls it. Choosing the code where the work is takes 3,158 µs where the best single overlay takes 4,222 µs.
A thread reaches any DRAM
An X280 cluster's cache reaches one DRAM bank, the one beside it, and its own loads
through a NoC window move 0.024 GB/s. So a thread does not load far data: it calls
kmx_fetch, and Tensix cores copy the range, from any bank or from a tensor's pages,
into the thread's region at the bank's rate; kmx_store is the same the other way.
The call names the tensor and the range, never where they are: for a tensor another
island holds, the same call goes straight between the two cards' DRAMs
(threads, bank_fetch).
More than one card
An island is a set of cores that reach the others only through links: one card's
chip. Ranks stay flat and contiguous across islands, so a command names a group as
on one card and a kernel finds its place in it the same way; a core holds a table
with one entry for each island, not one for each rank. The host places a tensor by a
hint in the kernel's own words, and a kernel reads and writes it with the page calls,
which ask the other island for what it holds: the kernels of a program written for
one card run on two unchanged. Crossing a link is slower than the NoC, and the run
counts what crossed (run.crossings()), so a program can be placed to cross less
(islands, reference).
What exists
| Part | On main | Page | Example |
|---|---|---|---|
| Persistent kernel: commands, 16 mailbox slots per core, the reader, compute and writer loops, every wait with a timeout, the trace | implemented | Tensix runtime | 01_mapcollect |
| Groups per command, several in flight, completion per group, a result word per command | implemented | Tensix runtime | 01_mapcollect |
Channels: emit, arrive, take, groups, reduce by owner and by tree |
implemented | ports | 02_merge_tree |
| Channels: broadcast along a row or a column, two broadcasts at once | implemented | broadcasts | 03_broadcast |
| Routed messages: messages through per-core queues in DRAM, a send quota, the quiet end found by the dispatcher; a body of up to 4 KiB beside a message; a handler as an overlay; X280 cores as ranks | implemented; one dynamic run at a time | dynamic handlers | 05_forward, 23_message_body, 24_thread_rank |
| Movement: both data-movement CPUs of a core on one stream, the pattern chosen by the command; each mover's NoC a field of the program | implemented | movement | 11_movement |
| X280 ports: a core tells, asks and pulls from its X280 core; a pull's answer carries a range and a word | implemented | ports | 10_x280_ports, 18_thread_split |
Code placement: overlays built on the host, loaded into slots of a core's L1 and called; named by a handler, a message, a leader's cast or a pull's answer; a group changes program inside a run by run.load or a thread's kmx_load |
implemented; not: the dynamic handlers of two programs in one run | overlays | 12_overlays, 16_two_programs, 17_leader_overlay, 20_message_overlays, 21_thread_load |
Page calls: km::Tensor, km::read_pages, km::write_pages, km::put_bytes, km::read_bytes, one call wherever the page is held |
implemented | islands | 06_graph, 09_bucket_sort |
| The dispatcher on X280 cluster 0: host ring, slots, deadlines, events | implemented | dispatcher | all |
| Micro-op streams and the job queue in the dispatcher | implemented; builder, check and simulator on the CPU | streams | 04_uop_stream |
| Graphs in the dispatcher: command nodes, CPU-task nodes, joins, a kept graph run again | implemented; also host-fed | graphs | 06_graph |
X280 threads: kmx_parallel, kmx_spawn / kmx_join, regions, the command ring, a host fallback in Python; a thread on each of work clusters 1 to 3, regions shared on one bank; kmx_parallel_all over twelve cores and kmx_push across banks; kmx_fetch / kmx_fetch_clean / kmx_store of any bank through Tensix cores |
implemented | X280 threads | 07_work_cluster, 08_parallel_thread |
The host package: Program, Session, Run, uploads and engine events, reads by page range, errors that name the core |
implemented | host API | all |
| Host transfers through the host channel: the cores move a tensor's pages between DRAM or L1 and the host's memory | implemented; a read under 80 KiB without a run still goes through the device API | the host channel | all |
Hardening: the kernel instruction check, an optimisation level per kernel, Session(kernel_bytes=...), an exit with a timeout |
implemented | host API | |
Recovery without a card reset: run.inspect of a running core, the give-up of one command at its deadline, the reset and reload of one core by the host and by the dispatcher (by itself at a give-up), the watch on the dispatcher's own loop |
implemented; the session hands the dispatcher the reset lists at load (recover=True) |
recovery | 01_mapcollect (give_up.py) |
Islands: two cards as one session, each card's chip an island with its own grid and dispatcher; flat ranks across the cards, tensors placed by a hint; across the links: a command's part from the host and from a thread, pages and parts of a page from a kernel's calls, messages, the parts of a reduce or a broadcast, a thread's kmx_fetch / kmx_store; programs loaded by island, inspect, time-outs and recovery by island; one card split into two to four islands, to develop with one card |
implemented over two P150A cards joined by four links; not: more than two cards, X280 cores as ranks over a link, a dynamic run on part of the ranks | islands, reference | 01_mapcollect (islands.py), 25_islands_threads, 26_islands_sum |
Measured
Whole programs on one P150A, every row from the program's page and its committed results, all dated 2026-11-13: AICLK 1,350 MHz nominal (the card lowers it under its power limit in the long matrix loops; each page gives the clock it sampled) and the X280 at the session's 1,000 MHz. Power is the whole card's mean over the loop.
Rasterizer and ray tracer: whole frames
Both programs run the whole frame on the card, 1024 x 1024, and both are far below half of a V100's speed today. The V100 and C++ columns are the reference's whole frame (the same algorithm in CUDA on one idle Tesla V100 32 GB, and in C++ on a shared 96-thread AMD EPYC 7K62); none of the three counts the scene's upload or the image's read-back.
The rasterizer: the passes and the counting run on the 120 cores, the serial rest of
the middle pass between them on a thread on the X280 work cluster; the image is bit-exact
against numpy. Median of 5 loads, each about 1 s of frames; card 1
(raster, results in
projects/raster/results/,
reference):
| Scene | Triangles | Frame | Card frame, ms | of which the X280 step | Card W | C++ 96 threads, ms | V100, ms | V100 / card |
|---|---|---|---|---|---|---|---|---|
| tiny | 398 | first | 0.987 | 0.436 | 61.9 | 2.66 | 0.29 | 0.29 |
| tiny | 398 | settled | 0.987 | 0.436 | 63.4 | 2.66 | 0.29 | 0.29 |
| 10^4 | 11,994 | first | 0.982 | 0.440 | 67.0 | 3.32 | 0.25 | 0.25 |
| 10^4 | 11,994 | settled | 0.952 | 0.453 | 65.8 | 3.32 | 0.25 | 0.26 |
| 10^5 | 102,308 | first | 2.050 | 0.753 | 68.0 | 4.08 | 0.79 | 0.39 |
| 10^5 | 102,308 | settled | 1.967 | 0.772 | 68.2 | 4.08 | 0.79 | 0.40 |
| 10^6 | 997,806 | first | 6.894 | 0.807 | 68.9 | 17.87 | 4.42 | 0.64 |
| 10^6 | 997,806 | settled | 6.339 | 0.823 | 70.2 | 17.87 | 4.42 | 0.70 |
| Sponza, overview | 262,266 | first | 5.770 | 0.457 | 68.8 | 7.88 | 2.31 | 0.40 |
| Sponza, overview | 262,266 | settled | 3.795 | 0.466 | 71.6 | 7.88 | 2.31 | 0.61 |
| Bistro interior, overview | 1,020,903 | first | 8.762 | 0.736 | 66.9 | 14.10 | 5.99 | 0.68 |
| Bistro interior, overview | 1,020,903 | settled | 6.503 | 0.753 | 69.4 | 14.10 | 5.99 | 0.92 |
"First" is a scene's first frame, with COUNT's and SETUP's ranges cut into equal parts: what a single image costs. "Settled" is a later frame of the same camera, whose ranges are cut by the frame before's predicted costs. Where the time goes (settled frame): at 10^6 FILL 2.1 ms, SETUP 1.9 ms, COUNT 1.1 ms and the X280 step 0.8 ms; on Bistro SETUP and FILL 2.1 ms each and COUNT 1.3 ms. The fill alone is ahead of the V100's fill stage (10^6: 2.1 against 3.3 ms; Bistro: 2.1 against 5.6 ms).
The ray tracer: ray generation, four traces and shading are eight commands on the 120 cores; trees larger than L1 are read from DRAM as treelets. Card 1. A cold frame has no history and is submitted by the host. A warm frame is the second or third of the same camera, submitted by a thread on the X280. Before each trace the thread cuts the rays into the cores' ranges by the frame before's costs; its plan time counts in the frame (raytrace):
| Scene | Frame | Card frame, ms | of which the X280 plan | Card W | C++ all threads, ms | V100, ms | V100 / card | Pixels differing from numpy |
|---|---|---|---|---|---|---|---|---|
| tiny (tree in L1) | cold and warm | 22.24 | - | 68.0 | 20.59 | 2.59 | 0.12 | 0 |
| 10^4 | cold | 57.91 | - | 63.0 | 26.44 | 4.16 | 0.07 | 0 |
| 10^4 | warm | 70.63 | 5.27 | 61.0 | 26.44 | 4.16 | 0.06 | 0 |
| 10^5 | cold | 81.51 | - | 63.0 | 35.47 | 6.57 | 0.08 | 0 |
| 10^5 | warm | 93.01 | 5.34 | 66.9 | 35.47 | 6.57 | 0.07 | 0 |
| 10^6 | cold | 154.21 | - | 63.0 | 53.66 | 11.61 | 0.08 | 1 |
| 10^6 | warm | 192.50 | 5.32 | 70.4 | 53.66 | 11.61 | 0.06 | 1 |
| Sponza, close | cold | 359.86 | - | 63.0 | 264.78 | 102.00 | 0.28 | 244 |
| Sponza, close | warm | 323.38 | 3.64 | 71.0 | 264.78 | 102.00 | 0.32 | 244 |
| Sponza, overview | cold | 348.53 | - | 69.2 | 230.36 | 95.52 | 0.27 | 234 |
| Sponza, overview | warm | 313.63 | 4.14 | 68.2 | 230.36 | 95.52 | 0.30 | 234 |
| Bistro interior, close | cold | 596.97 | - | 61.0 | 401.87 | 211.42 | 0.35 | 2 |
| Bistro interior, close | warm | 596.86 | 4.35 | 70.0 | 401.87 | 211.42 | 0.35 | 3 |
| Bistro interior, overview | cold | 523.72 | - | 73.1 | 336.80 | 153.94 | 0.29 | 2 |
| Bistro interior, overview | warm | 477.48 | 4.57 | 70.5 | 336.80 | 153.94 | 0.32 | 2 |
Where the time goes: the four traces take 67 % (10^4) to 97 % (Bistro) of the frame, the bounce trace alone 36 to 59 %; each core sorts its bounce and second-shadow rays (about 15 ms a frame) and reads the treelets its rays need from DRAM. In the traces, box tests on packets with few live broadcasts come first (61 % on Sponza's overview), then the wait for each command's slowest core. The differing pixels are ties between two triangles at the same float32 distance, but for one or two on each Bistro view. The plan by the frame before takes 35 to 46 ms off Sponza and Bistro's overview, nothing off Bistro's close view, and costs 12 to 38 ms on 10^4 to 10^6: it cuts the traces into up to 8 commands, and the cores read more treelets.
Read back from the card, 512 x 512 downscales of 1024 x 1024 frames. The first two are rasterized, both bit-exact against numpy: Sponza overview (3.80 ms a settled frame above) and Bistro interior overview (6.50 ms). The last two are ray traced: Sponza close (359.86 ms a frame above) and Bistro interior close (596.97 ms). Crytek Sponza, (c) 2010 Frank Meinl, Crytek, CC BY 3.0; Amazon Lumberyard Bistro, (c) 2017 Amazon Lumberyard, CC BY 4.0 (Open Research Content Archive); both from the McGuire Computer Graphics Archive.
Matrix product, attention and SwiGLU against Kohaku-TT
The same card, ours and Kohaku-TT (at ac81ff2) one after the other in one session; bf16 operands, fp32 accumulation; time per call in a loop of calls, uploads and read-back outside it. "Kohaku-TT / ours" above 1 means ours is faster. Ours was measured again on 2026-11-14 with the programs' own bench scripts (best of three runs); a row that moved by less than 3 % keeps its value of 2026-11-13, and Kohaku-TT's columns were not run again. Replaced: four matrix product rows, three attention rows and the one SwiGLU cell in its page that moved.
Matrix product, HiFi2, Kohaku-TT's fastest of its
three routes (results/<preset>-ours.json, -ktt-<route>.json):
| M x K x N | Ours, µs | W | Kohaku-TT, µs | W | Kohaku-TT / ours |
|---|---|---|---|---|---|
| 128 x 896 x 128 | 6.0 | 71 | 28.7 | 63 | 4.75 |
| 16,384 x 384 x 384 | 61.2 | 130 | 94.1 | 116 | 1.54 |
| 16,384 x 1024 x 1024 | 393.5 | 128 | 389.9 | 136 | 0.99 |
| 4096 cubed | 958.3 | 138 | 957.0 | 151 | 1.00 |
| 8192 cubed | 8608.3 | 141 | 11797.6 | 139 | 1.37 |
| 16,384 cubed | 87944.5 | 137 | 75522.5 | 142 | 0.86 |
| 32 x 4096 x 4096, K split | 92.4 | 83 | 81.0 | 91 | 0.88 |
From 4096 cubed up both sides are bound by the card's power limit in a loop (ours ran in a slower mode in most loops of 4096 and 8192 cubed on 2026-11-14, see the page; the table shows the fastest run); the last row is the K split of the page (14,1,16,5 tree, groups of 16 cores to an output block), bound by reading W from DRAM: 363 GB/s in the call against Kohaku-TT's 414.
Attention, K and V multicast down broadcasts of
cores, the same compute body as Kohaku-TT's flash kernel
(results/ours-<preset>.json, ktt-<preset>.json):
| B x H x S x D | Ours, µs | W | Kohaku-TT, µs | W | Kohaku-TT / ours |
|---|---|---|---|---|---|
| 1 x 32 x 4096 x 64 | 2675.3 | 138 | 2918.5 | 135 | 1.09 |
| 1 x 24 x 4096 x 128 | 2328.3 | 144 | 3001.8 | 132 | 1.29 |
| 8 x 16 x 256 x 64 | 76.7 | 123 | 80.7 | 110 | 1.05 |
| 1 x 8 x 1024 x 64, causal | 58.1 | 103 | 53.7 | 99 | 0.92 |
| 2 x 8 x 1024 x 128, 2 K / V heads | 149.9 | 136 | 261.8 | 105 | 1.75 |
The causal row is behind per call (58.1 against 53.7) and ahead per pass (48.3): the cost of a command.
SwiGLU, silu(x W1) * (x W3) then W2, HiFi2,
ours in its faster form (results/swiglu-<preset>-<form>-hifi2.json, -ktt-<route>.json):
| M x D x H | Ours, µs | W | Kohaku-TT, µs | W | Kohaku-TT / ours |
|---|---|---|---|---|---|
| 256 x 256 x 512 | 42.6 | 76 | 181.6 | 73 | 4.26 |
| 4096 x 256 x 512 | 73.7 | 114 | 183.5 | 95 | 2.49 |
| 16,384 x 256 x 512 | 180.2 | 137 | 542.1 | 109 | 3.01 |
| 16,384 x 384 x 384 | 161.7 | 142 | 458.1 | 108 | 2.83 |
Two cards as one program
Two P150A joined by four Ethernet links, one session, 2026-11-14 and 2026-11-15; each number from the page it links.
A linear layer Y = X W, bfloat16 with fp32 accumulation, cubes from 128 to 16,384,
the same script and kernels on one card and on two
(dense); median single
call in microseconds, "x" one card's time over two cards':
| Cube | One card | Two cards, W gathered | x | Kohaku-TT: one card / two cards (W gathered) |
|---|---|---|---|---|
| 128 | 7.2 | 7.6 | 0.95 | |
| 256 | 11.6 | 10.0 | 1.16 | |
| 512 | 15.9 | 19.8 | 0.80 | |
| 1024 | 43.8 | 40.6 | 1.08 | |
| 2048 | 167.1 | 125.4 | 1.33 | |
| 4096 | 655.0 | 420.7 | 1.56 | 819.8 / 487.3 |
| 8192 | 4,488.1 | 2,357.6 | 1.90 | 4,549.5 / 2,482.8 |
| 16,384 | 34,544.8 | 17,731.5 | 1.95 | 32,489.3 / 17,067.8 |
With the weight gathered on both cards nothing crosses in a call, and two cards take half of one card's time from 8192 up, as Kohaku-TT's default two-card form does. The two forms that keep one copy of the weight (the input dimension sharded, with one addition across the cards; the output dimension sharded, with the activation on both) need the gather and the reduce as calls (TODO item 2) and are not measured here.
What crosses a link, and what it costs (islands):
| What | Measured |
|---|---|
| A command to a group over both cards | 7.46 µs split by the host, 10.89 µs sent across by a thread's dispatcher |
| A message forwarded to a rank of the other card | 4.7 µs a crossing in a run of round trips |
| A reduce over both cards | a sum over 238 ranks in 172.6 µs against 162.8 µs on one card of 119 |
| Pages put by kernels, both cards at once | 512 MiB each way in 7.41 ms, 72.5 GB/s each way |
| Pages read by kernels | 512 MiB in 11.1 to 13.2 ms, 40.6 to 48.3 GB/s |
Part of a page, written (km::put_bytes) |
65,536 keys placed in the other card's DRAM in 5.99 ms |
Part of a page, read (km::read_bytes) |
a range every 1.70 to 1.95 µs |
A thread's kmx_fetch of a tensor the other card holds |
64 MiB in 1.13 ms, 59.5 GB/s |
The session's own copy across (session.cross) |
512 MiB in 5.35 ms, 100.4 GB/s |
Code placement
Overlays, programs in one run and handlers loaded where they are needed, each number from the page it links (the runs are on those pages; none was measured again for this table). One P150A, AICLK 1,350 MHz.
| What | Number | Page |
|---|---|---|
| A held overlay called, against the same code compiled in | from a leader's cast to the last call done, 1.19 to 1.31 µs held against 1.00 to 1.13 µs compiled in, on a row, a column and the grid | 17_leader_overlay |
| A load | load() of a reader part 0.88 µs; of a reader and compute parts, meeting, 1.82 to 1.88 µs; held, 0.07 µs |
17_leader_overlay |
A group changes program (run.load) |
2.3 to 2.6 µs (3,124 to 3,360 cycles), on 1 core and on 120 | 16_two_programs |
A group changes program from an X280 thread (kmx_load) |
6.2 µs from issue to done in the thread, against 134.9 µs for run.load and its wait from the host; 40 rounds of a load and two commands on 120 cores in 1,825 µs against 16,950 µs host-fed |
21_thread_load |
| A command to an X280 thread | 133.0 µs from the host one at a time against 132.6 µs for a grid command; in a graph on the X280, a round with the thread's command 19.3 µs against 11.2 µs with a second grid command | 13_thread_commands |
| A message through handlers as parts | 10^6 messages on 120 cores in 10.76 ms with a slot for each handler against 10.83 ms with the handlers compiled in (94.2 and 93.6 million messages a second); 11.40 ms with one slot | 20_message_overlays |
| A thread splits the grid between two programs | 24 rounds in 974 µs on 12 cores and 1,023 µs on 120, against 1,001 and 1,070 µs for the best fixed split and 7,378 and 7,521 µs with the host feeding it | 19_thread_programs |
The examples' own timings (commands per microsecond, broadcasts, forwards, streams, graphs and threads) are in each example's README.
Install
Komorebi 0.2.0 is on PyPI as two distributions, komorebi-tt (the framework) and
komorebi-runtime (the pinned tt-metal build), with the GitHub release v0.2.0. It is
for one card model, the Blackhole P150A. Komorebi is used from those
wheels (the two wheels) or from a clone
(packaging, from a wheel), on
Linux x86-64 with Python 3.13:
git clone <this repository> komorebi && cd komorebi
uv venv -p 3.13 && source .venv/bin/activate && uv pip install -e ".[dev]"
kmr doctor # the Python, the tt-metal tree and its X280 runtime
The CPU tests and the stream and graph tools need nothing else. To run on a card: a
Blackhole P150A whose X280 memory is prepared for the X280 runtime, a tt-metal source
tree at the version in packaging/tt-metal.pin (v0.81.1) built
with its X280 runtime, and TT_METAL_HOME naming that tree (neither ttnn nor
torch is needed). The dispatcher image and the native module are
built from the tree on first use (python -m komorebi.native builds them ahead).
Details: install.
The distribution is named komorebi-tt; the package is imported as komorebi and the
command is kmr. pip install "komorebi-tt[runtime]" installs it with
komorebi-runtime, the pinned tt-metal build, and a program runs on a card with no
tt-metal tree (the two wheels); kmr setup
prepares the card once.
Usage
Every example runs on its own, checks its output against numpy and prints one JSON
line with ok (examples); each program's page lists its
commands (projects):
cd examples/02_merge_tree
python merge.py --jobs 20 --count 8 --form both
python merge.py --group-list 0:8,0:119 --form both --stall-rank 3
cd ../..
python -m projects.raytrace.frame --scene tiny --reps 16
The second command adds a load in which rank 3 of job 1 never sends: the host raises
km.CommandTimeout naming the handle and the cores that did not report, and the next
load in the same session runs exact.
Repository map
src/komorebi/ the Python package: Program, Session, Run, layout, power, kmr
engines/ micro-op streams and graphs: builders, encodings, checks,
simulators; the host-fed executor
thread.py X280 threads: declaration, regions, results, host fallback
manifest/ reserved: a program's commands and handlers declared
src/komorebi_native/
include/komorebi/abi/ the command, the event and the stream format, shared by all
include/komorebi/device/ the persistent kernel, groups, ports, groups, broadcasts, dynamic
handlers, the trace
kernels/ the generic reader and writer of dynamic handlers
x280/kernel/ the dispatcher for X280 cluster 0
x280/engines/ x280/kmx/ the plain-C graph engine; the X280 thread library
csrc/host/ the native module: X280 runtime, commands, events
examples/ 01_mapcollect to 08_parallel_thread
projects/ raster, raytrace, dense (with SwiGLU), attention
experiments/ hardware and design probes (see its README)
reference/ CPU, C++ and CUDA references the programs are checked against
tests/core/ tests/projects/ tests with no card: the framework; the programs
tests/hardware/ card tests (KMR_CARD=1)
docs/ concepts, references, install, development
Sizes a program works within
These are fixed sizes of the runtime and limits of the card. They are not on the TODO list: a program is written to fit them, and what has been built so far (matrix product, attention, a rasterizer, a ray tracer) fits.
- A command is one 64-byte line. It carries ten 32-bit argument words. Anything larger travels by reference: two of the words can name a block of up to 1,024 bytes, and tensors are named by their address.
- 64 commands in flight at most. A command costs more the more are in flight (about 7 % at 16, 21 % at 48), so a program keeps a small window; the default is 4.
- Handlers 1 to 239 per program. More code is another program, or an overlay a core loads when it is first needed.
- A core's L1 holds its kernels, its buffers and its overlay slots. A program that
does not fit is refused at load with the budget table; the answer is smaller buffers,
or code moved into overlays. Code runs from below 1 MB of L1 (a taken branch to code
above it costs about 4 cycles more): the kernels and slots sit at the bottom, the
buffers above them, and a load whose code would reach past 1 MB raises
SlowCodeErrorunless the session allows it (slow_code=True). - A link is slower than the NoC. What crosses between islands is bound by the links, so a program over islands is placed to cross little: a tensor split along the dimension its work is split along, or gathered where every island reads all of it, and what must cross sent as one gather or one reduce.
- A message is one line. 64 bytes, with a body of at most one 4 KiB page beside it. Messages have no order between them, and a message cannot ask for a DRAM fetch.
- What costs a card reset. A core stuck inside a compute unit, a dispatcher whose first core stopped executing, an X280 thread that never returns and polls nothing, and a process killed with SIGKILL (recovery).
- One clock for the four X280 clusters. 1,350 MHz by default; the session takes
1,000, 1,250, 1,350 or 1,500 (
pll_mhz). 1,750 MHz is an opt-in that is not stable under full matrix load: the card lowers its core voltage there, and the dispatcher trapped in 9 of 18 runs. - One card's grid. Up to 128 cores in at most two multicast panels; one harvesting pattern has been run. Two cards run as two such grids: 236 ranks on two P150A joined by four links, since two cores of each card serve the links.
TODO
What Komorebi does not do yet. None of these is a property of the model; what a program has to be written around is under sizes.
- More than two cards. Two P150A run as two islands; three or four islands have run only as one card split into islands. Not yet over a link: X280 cores as ranks, a dynamic run on part of the ranks, a put to an island two links away.
- Collectives across islands as calls. An all-gather and an all-reduce are made by a program from puts today, and a small one costs 144 to 331 µs where a message crosses in 4.7 µs. They should be calls of the runtime, at a message's latency and the links' rate.
- Tensors an island cannot hold yet. A tensor is placed by pages of one size set at build. A range of one DRAM bank (an X280 thread's region) and a stream whose page size is set by the command are named by bank address, so only their own island reads them.
- Messages over every link by default. A message service on each of the four links carried a run's messages across in about a quarter of one link's time (80 ms against 302 ms), but one run of three stalled; until that is explained one link carries the messages and the rest is an option.
- Dynamic handlers. One dynamic run at a time, and not the dynamic handlers of two programs in one run (dynamic handlers).
- A link core reset by its dispatcher. A dispatcher resets and loads again the cores of its own island; a link core that hangs is named by it and reset by the host (recovery).
- Debugging on the card.
run.inspectshows each core's state, the wait it is in and its trace. Nothing reads a core's registers, its buffers' contents or a program's own variables, and tt-metal's watcher cannot run beside the X280 runtime. - A language above the kernels. A program is C++ kernels and a Python script today. A Python layer that writes the kernels from classes, and compiles tile arithmetic, is designed and not built.
- Loading TT-NN's kernels. A TT-NN operation is a tt-metal program, which is what a program loaded as overlays is; nothing loads an operation's kernels as they are.
- Starting without tt-metal's launch. The session opens the card in slow dispatch and tt-metal launches the kernels once, core by core; everything after that is Komorebi's own.
- Wormhole. Only Blackhole. A Wormhole card has the Tensix grid and no X280 cores, so the dispatcher and the threads need another home there.
- The build from a clone.
kmr build hostandkmr build pymoduletake their compile flags from two targets that the tt-x280-dispatch project adds to a tt-metal build (development). The wheels need neither.
Komorebi has run on Blackhole P150A cards, one and two together, and on no other hardware.
Documentation
| Page | Read it for |
|---|---|
| docs index | every concept and reference page |
| Tensix runtime | the device headers, the command, timed waits, the control block |
| ports, broadcasts, dynamic handlers | messages between cores |
| streams, graphs | deciding on the card in the dispatcher |
| islands, recovery | two cards as one session; a command that does not return |
| X280 threads | C on a work cluster with kmx |
| host API | Program, Session, Run, errors, builds |
| examples, projects | runnable examples; the five programs and their results |
| install, development | building from a clone; layout, checks, conventions |
Contributing
Open an issue first for a feature. Before a pull request, from the repository in the environment of install:
python -m pytest tests/core
python -m pytest tests/projects
python -m komorebi.tools.cxx_format --check
python -m komorebi.tools.comment_lint --check src tests examples experiments packaging tutorial projects
ruff check . && black --check . && blacken-docs -t py313 -l 88 $(git ls-files '*.md')
Conventions: development.
Licence
Apache-2.0; see LICENSE.
Metadata
Release files for komorebi-tt 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| komorebi_tt-0.2.0-cp312-abi3-manylinux_2_32_x86_64.whl | CPython 3.12 | abi3 | Linux glibc 2.32+ x86-64 | Details for komorebi_tt-0.2.0-cp312-abi3-manylinux_2_32_x86_64.whl |
Release files / komorebi_tt-0.2.0-cp312-abi3-manylinux_2_32_x86_64.whl
| Download URL | komorebi_tt-0.2.0-cp312-abi3-manylinux_2_32_x86_64.whl |
|---|---|
| Size | 1.0 MB |
| Tags | CPython 3.12 Linux glibc 2.32+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
b5516250a20481627204649dff34001d1cc74413fa220c7b6efb92669a525298
|
|
BLAKE2b-256 checksum How to use checksums |
4fa42ddb6c7149a1caf6ba3e658323481fe59785d808e3ec513371825a136e14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logSecurity for komorebi-tt 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
1 file in this release has been uploaded with provenance metadata.
Attestation for komorebi_tt-0.2.0-cp312-abi3-manylinux_2_32_x86_64.whl
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.