datacenter-layout-viewer
A very light datacenter viewer. One plain-text file describes the whole
datacenter — rooms, rows, racks, servers, and the logical networks between them
— compactly enough that a hyperscale campus fits on a single page. A second
append-only file carries test results, which the viewer paints over the layout
as any number of simultaneous color overlays. A wide TSV table —
Timestamp host var1 var2 …, which is what monitoring already writes — is
read as it stands, and a folder of them can be reloaded on a timer as a live
dashboard. Where the layout is not written yet, one can be read out of the
hostnames on request.
No build step, no dependencies, no server-side anything. Static files only.
Current version 1.54.2 — see CHANGELOG.md. One number covers the viewer, the file formats and the tools; the formats are the compatibility promise, so a file that loads today loads on every later 1.x (how a release is cut).
Run it
Any static file server works, and a clone is already the whole app:
python3 -m http.server 8000 # in the clone
# open http://localhost:8000/
Or install it, and get the server and the two overlay tools with it:
pip install datacenter-layout-viewer
dcviz serve # opens the viewer in a browser
dcviz serve --dir ~/layouts # ... with that directory at /files/
The package has no runtime dependencies — it is the same static files
plus a hundred lines of http.server — and neither route is more supported
than the other. A server is not optional, though: browsers refuse to load ES
modules from a file:// URL, so opening index.html directly gives a page
with dead buttons and a banner saying so.
The viewer starts empty. Load files with the Load files… button, from the Files browser in the left panel, by dragging them onto the window, or with URL parameters. Where the browser supports it (Chromium), the button reopens in the directory it was last used in; elsewhere it is the ordinary file input:
http://localhost:8000/?layout=examples/small.dc&results=examples/small-results.tsv
Try the scale test: ?layout=examples/mega.dc is a ~256,000-element
campus with ~560,000 cables, described in 45 lines.
Or write a layout from nothing: Edit layout opens an editor under the
canvas that re-parses on every keystroke, so the floor plan, a per-kind tally
(3 rows · 24 racks · 216 nodes) and any warnings follow the line being
written. Insert template drops in a commented skeleton to start from, and
Download .dc saves the result as a file. The same editor works on a loaded
file, which makes it the quickest way to learn what a line expands to. Drag
the bar along its top edge to make it taller or shorter; double-click the bar
to put it back.
The editor completes as you type (or on Ctrl+Space): kinds at the start of
a line, then the keys that line takes, then values — enumerated ones like
mode= and style=, and ones harvested from the layout itself, so role=
offers the roles this file already uses and + offers its tags. Syntax
opens a reference panel beside the text with every construct as a
click-to-insert snippet.
The layout file (.dc)
Indentation-based. Every line is <kind> <id> [key=value ...] [+tag ...]:
dc IAD1 name="Acme IAD1" +prod region=us-east
room DH1 name="Data Hall 1"
row A..D +compute
rack R[01..06] u=42
node tor at=42 role=tor +switch
node u[01..20] role=server +x86 +gpu model=hgx-h100
Those eight lines are 504 elements: ranges expand (A..D, R[01..06],
u[01..20]), and the children of an expanded line are created once per
expansion. That multiplication is what makes the biggest datacenter fit on a
page — the examples/mega.dc campus is 8 halls × 24 rows × 40 racks × 41
devices from one nesting.
Kinds
dc, room, row, rack, node get purpose-built layout (rows line racks
up on a shared floor; racks stack their children into real U-slots). Any
other word works too — pod, hall, cage, suite — and is drawn as a
generic container. There is no schema to declare.
Indentation is the structure, so a line indented one level too shallow
re-parents everything below it. The tell is a node that ends up containing
other elements — its rack sits empty with the "contents" drawn beside it —
and the parser reports it as a warning. Children sit deeper than their
parent; siblings align. (A deliberate container should be a non-node kind
like chassis, which nests silently.)
Ranges
| Spec | Expands to |
|---|---|
R[01..20] |
R01 R02 … R20 (zero-padding kept) |
u[1..40x2] |
u1 u3 … u39 (step) |
A..H |
A B … H (letters; bare or bracketed — both ends must be the same alphabet, so A..z is refused rather than expanded through the punctuation between Z and a) |
[web|db|cache] |
web db cache (alternatives) |
r[1..2]-[a|b] |
r1-a r1-b r2-a r2-b (cartesian) |
R[1..4,7..10] |
R1 … R4 R7 … R10 (comma-separated segments) |
A spec that expands to nothing (r[,]) is reported: it would otherwise take
the line and everything indented under it away without a trace.
A mistake in a line's attributes is reported once for the line, naming what
was written ("R[01..40]") rather than once for each of the forty racks it
became.
Segments are how numbering with holes stays on one line: racks 1–4 and 7–10
around a gap are rack R[1..4,7..10], with the children written once instead
of once per block — see examples/three-rows.dc, which also pins sparse
U-slots the same way (node [7..15x2,25..31x2] id=u{id} at={id}).
Placeholders
{placeholders} in attributes refer to enclosing elements:
name="Hall {id}", power=grid-{i}, {room}, {row}, {parent}.
A kind names the nearest element of that kind, and on that kind's own line
the nearest is the element itself — so {dcm} on a dcm line is its own
id, the same as {id}: dcm [1,2] name=dcm-{dcm} names dcm-1 and
dcm-2. Nothing is inserted between the text and the id, so the hyphen is
yours to write (name=dcm{dcm} is dcm1). The built-in names — {id},
{i}, {i0}, {n}, {seq}, {kind}, {parent}, {path} — always mean
what they say here, even under a kind of the same name.
This is how flat hostname-style names work: node u[01..40] name={room}{rack}{id} names every server like wr12r06u15, results files
can then target that name directly, and the row stays expressed by nesting
without appearing in the name — see examples/hostnames.dc.
Any whole-number placeholder takes a width: {i:3} is 001 and {seq:2}
is 07, padded as a range written R[01..20] is. A value that is not a
number is left as it is.
A placeholder naming something not in scope is reported rather than left in
the text — {rak} for {rack}, or {id} in an id spec, where the id does
not exist yet — and the warning lists the names that line could have used.
Numbering through the floor: {seq}
{i} starts again under every parent; {seq} carries on, counting every
element the line has made so far across all the copies of it. So racks
numbered uniquely across a floor are one block, not one per row:
row [1..4]
rack [1..5] id=R{seq:2} u=42 # row 1: R01-R05, row 2: R06-R10, ...
node u[01..40] role=server name={rack}{id} # R07u15
[1..5] says how many racks each row has, id= gives each its number, and
{seq:2} pads it to two digits. The count keeps going however deep the
repetition goes: under room [A|B] the same rack line runs on to R40.
examples/dual-plane.dc numbers its pods' racks this way. {seq} belongs
on an attribute (id=, name=): an id spec is expanded before anything is
counted, so rack R{seq} is reported.
Lines side by side share the count. Lines of one kind in the same block number as one, so a row can hold different racks in different places and still be numbered straight through — here the middle four racks of each row are network racks:
row [1..4]
rack [1..8] id=R{seq:2} u=42 # R01-R08
node u[01..40] role=server
rack [1..4] id=R{seq:2} u=42 +network # R09-R12, the middle four
node agg[1..4] u=2 role=agg
rack [1..8] id=R{seq:2} u=42 # R13-R20; row 2 is R21-R40
node u[01..40] role=server
Only lines that use {seq} take a number — a node tor line beside a
numbered server line does not use one up.
Starting again in every room. seq=room on a line gives it one count
per room: racks numbered R01–R80 in each room, however many dcms or systems
repeat the rooms, the way scope=room groups a link rule. Any enclosing kind
works — seq=row, seq=dcm — and what is counted is kept per kind, so racks
and servers numbered per room keep two counts. Lines sharing a seq=room
share the count within the room, so a row split into three rack lines still
runs in order:
dcm [1..2] name=dcm-{dcm}
room [1..2] name="Room {room}"
row [1..4]
rack [1..8] id=R{seq:2} seq=room u=42 # R01-R08 ... in every room
rack [1..4] id=R{seq:2} seq=room u=42 +special # R09-R12
rack [1..8] id=R{seq:2} seq=room u=42 # R13-R20, then R21- in row 2
Any other word names a count instead: seq=net keeps network racks
numbered N1, N2, … apart from the racks beside them, and lines anywhere with
the same name share one count.
Lines that apply only in some places: if=
if= on a line makes its elements only where the condition holds, so one
block for four rows can still give the first row something the others do
not have — here rack 12 is filled in row 1 and stands empty in rows 2–4:
row [1..4]
rack [1..11] id=R{seq:2} seq=room u=42
node u[01..40] role=server
rack 12 id=R{seq:2} seq=room u=42 if={row}=1 # row 1: R12, filled
node g[01..10] u=4 role=server +gpu
rack 12 id=R{seq:2} seq=room u=42 if={row}!=1 # rows 2-4: R32, R52, R72, empty
rack [1..8] id=R{seq:2} seq=room u=42
node u[01..40] role=server
A condition is {placeholder}=value or !=, and several joined by commas
must all hold (if={room}=1,{row}=1 is only the first row of room 1). A
value may list alternatives ({row}=1|3) and use * and ? as a selector
does; matching ignores case. An element not made gives its {seq} number
back, so the numbers stay in order, and anything indented under it is not
made either. A condition that cannot be read is reported and ignored, and
if= is not kept among the attributes of what it made.
Attributes and tags
-
key=valueattributes are free-form and inherit downward (children see the parent'sregion=us-eastunless they override it). Layout-only keys —id,name,u,at,cols,dir,gap,align,seq,if— do not inherit. -
Attributes the viewer does not draw are still worth writing: they show in the inspector, and other tools read the same file.
nic_gbps=on servers anduplinks=/uplink_gbps=on racks are the hardware binnacle'sreckongrades a test run against — onenic_gbps=25on a row covers every server in it. What the viewer itself draws as capacity isgbps=on a net or a link rule — see Capacity. -
+tagtags are free-form keywords; every element can carry many, and every element also carries its ancestors' tags.+a,b,cadds three at once. One tag means something to the viewer: cable load forwards traffic only through elements tagged+switch. -
Rack children:
u=4gives a node 4 U of height,at=42pins it to a slot; unplaced children auto-fill the lowest free run of slots. A node that lands above the rack's declaredu=height is reported as a warning —at=42only fits a rack at least 42 U tall. A rack with nothing in it is drawn as an empty rack, as tall as its slots, rather than as a collapsed stub. -
u=is an element's height, in U, everywhere — the scale racks are drawn to, where one U is the height of a 1U server. On a rack it is how many slots the rack has, and on a rack's child how many it fills, as above. Outside a rack, a device is exactly that tall (node pdu u=4in a cage) and a container is at least that tall (cage C u=60): what its children need always fits. A row made taller than its racks keeps them standing on the floor, with the room above. A collapsed container keeps its compact size whatever itsu=. -
w=is its width, in U, on the same scale — whatu=is to height: a device outside a rack is exactly that wide (node PANEL w=96 u=8, a patch panel as wide as the racks it serves), and a container is at least that wide. A rack and what is in it keep the rack's width, and a collapsed container its compact size. It is not inherited. -
cols=2/dir=x|y/gap=shape generic containers —gap=0packs children with no gutter, and omitting it keeps the per-kind default. -
align=center(orright;leftis the default) places each line of a container's children across the room it has — a narrow network layer centred over the racks below it. Analign=that is not one of those is reported and laid out from the left. -
A row lays its racks out in one line, until it says otherwise:
cols=ordir=ymakes it a grid like any other container. That is how a row carries a network layer above its racks —cols=1stacks the two, anddir=xon each keeps its own contents in one line:row [1..4] name="Row {row}" cols=1 align=center network dir=x spine [1..4] name=spine-{spine} role=spine servers dir=x rack [1..20] id=R{seq:2} u=42 node u[01..40] role=server -
The numbers are checked:
u,w,at,sizeandcolsmust be whole and between 1 and 1000 (gapfrom 0), and a net'swidthbetween 0.1 and 100. Anything else is reported and ignored rather than quietly coerced —u=1e9used to read as 1 andu=-5drew a rack of negative height.
Networks
Logical fabrics, drawn as differently colored/styled cable layers. Rules match elements by selector, so you never enumerate cables:
net data label="Data / east-west" color=#4fa3ff
net mgmt label="Out-of-band mgmt" color=#8b93a7 style=dashed
link data role=server role=tor scope=rack # star, per rack
link data role=tor role=spine scope=hall # every ToR to every spine
link storage +storage,role=server scope=row mode=mesh # full mesh within a row
- Selectors:
+tag,^tag(non-inherited),kind=rack,attr=value, bare glob against id/path/ancestors,!negation,|alternatives,,for AND. color=takes any CSS colour — hex (#4fa3ff,#fff), a name (teal), or a function (rgb(1,2,3), quoted if it contains a space). One the browser cannot read is reported and dropped: the canvas ignores a colour it cannot parse and silently keeps the last one, which would paint the net in the previous net's colour.- Globs work in every part of a selector, tags included:
*stands for any run of characters and?for exactly one.model=r76*takes r760 and r7625 alike,model=r762?only the five-character one,u1?the slots u10–u19, and+stor*any tag startingstor. scope=groups matches per rack/row/room/… before wiring.mode=isstar(A×B, default with two selectors),mesh(default with one),chain,ring, orpair(A[i] to B[i]; with one selector, consecutive matches pair off — 1st–2nd, 3rd–4th, …).pairbetween two selectors of different lengths stops at the shorter one and reports how many were left unwired — two spines paired with eight ToRs is two cables and six racks with no uplink, whichmode=staris almost always what was meant. With one selector an odd count leaves one over, which is inherent and says nothing.cap=stops a rule after that many cables, which is how a selector that turns out to match half the floor is kept from wiring all of it. A cap that is not a whole number between 1 and 100,000,000 is reported and ignored, and a rule that hits its cap says so.splice=Ngathers every N consecutive matches of the first selector into one cable to each element of the second — a breakout harness, four servers spliced into one cable to a ToR port. See Splices below.- A rule wires two selectors. A third token is reported rather than
dropped, which is what catches a mistyped option —
scoope=rackcannot be told apart from selecting on an attribute calledscoope, but it can be counted. - A rule that wires nothing says so: a selector that matched no elements (the
shape a typo makes) and a rule whose matches produced no cables are both
reported as warnings rather than left as a silently empty fabric. A
scope=that names no kind in the layout says which kinds there are (scope=rw— did you meanscope=row?), and one that keeps the two sides apart — ToRs in the rack rows, spines in a row of their own,scope=row— says so and names the scope that holds both, or none. - A
netline takeslabel=,color=,style=,width=,show=,gbps=andtraffic=; anything else is reported with the setting it was most likely meant to be (gpbs=— did you meangbps=?). A quote left open anywhere is reported too: it runs to the end of the line, so"label="x" gbps=400is a label readingx gbps=400and no capacity. - Declared nets start visible on a modest floor (up to 20,000 cables in
total), so wiring something draws something. Past that they start unticked —
a hyperscale floor's first render should be the floor.
show=on anetline (oron=) settles it either way, and takes any spelling of yes or no —yesyon1true,nonoff0false. Anything else warns and leaves the decision to the size rule.
Networks toggle on and off in the UI, and modest fabrics start on (see above). Collapsed or zoomed-out regions merge their cables into one thicker line; very dense views fade automatically.
How cables are drawn
A cable leaves its device from the left or right edge, never the middle, steps out into a lane running beside the devices, and follows it — the way cables run up a rack's cable manager. Everything in one rack (or any one container) puts a given net on the same side at the same distance, so a rack's twenty server-to-ToR cables share one line beside the servers, with a short stub from each server into it, rather than twenty lines fanning across the rack.
When a device carries several nets, they leave from opposite sides: data on the right and mgmt on the left, say, so the two never draw on top of each other. A third net takes a lane of its own further out, on whichever side has fewer. The choice is made per container — each net goes to the side its devices have fewer nets on — and racks that carry the same nets make the same choice, so an uplink leaves its ToR on the side the spine's own lane is on.
Two devices in the same column of the same container (a server and its ToR) are joined along their shared lane. Anything further apart steps out into its lane at each end and crosses between the two as a curve: it leaves each device sideways, the way that device's lane runs, and bends round to arrive at the other end the same way. The cables sharing a port are fanned out along its lane — ordered by where their other end is, a little apart, within the device's height — so forty ToR uplinks arriving at a spine no longer meet in one spot. curved cables in the Networks panel switches the crossings back to straight lines (they stay fanned out), and a view dense enough to fade to a haze — more than 10,000 cables — draws them straight anyway, since there a curve costs half as much again to draw and cannot be told from a line.
A bus is met along its length. cables=bus on an element — a long,
thin one, drawn across the devices it serves with w= — makes every cable
arrive at its top or bottom edge, whichever faces the device at the other
end, square to it and at the point nearest that device: the racks under an
overhead busway drop straight onto it where they stand, instead of all
meeting it at one end. The device end still leaves by
its side and lane. Two buses join edge to edge where they overlap.
cables=sides is the default, and anything else is reported.
node BUSWAY w=200 cables=bus +switch # an overhead busway along the row
node PDU[1..8] role=pdu # each rack's power, under it
link power name=BUSWAY role=pdu
Splices
In a rack of sixteen servers, every run of four is often cabled into a
harness that splices the four into a single cable, and that one cable runs to
the ToR. splice=N on a link rule says so:
link data role=server role=tor scope=rack splice=4
Each run of N servers gets a slim marker beside it, in the net's own colour, with the N stubs drawn coming together inside it; one heavier cable runs on from the marker to the ToR. The marker is part of the cable layer — it has no label, takes no rack space, and toggles with its net — so it never reads as a device.
- Runs are consecutive in the rack, from the bottom — U1–4, U5–8, … — in U
order whatever order the lines were written in, and never cross from one
rack into the next, even with a wider
scope=. Eighteen servers make four runs of four and one of two; a run of one is simply a cable. - Every server is still wired to its ToR: the links are the same ones the
rule makes without
splice=, and each carries the splice it runs through. Link counts, the inspector, and show only this element's cables work as before — isolating one server shows its own cable into the marker and on to the ToR, and none of the other three on the harness. - The inspector names a server's splice (
data: 4 into one cable to tor) and counts the splices arriving at a ToR or below a container (data ×4 (16 cables into 4)). - A splice gathers the first selector's matches onto cables to the second, so
it needs two selectors and
mode=star; anywhere else it is reported and ignored. N is a whole number from 2 to 1000. - Collapse the rack and its splices are inside it. A harness to a ToR in another rack folds into the collapsed rack's one thicker cable, as unspliced cables do.
examples/splice.dc is a small floor wired this way.
Capacity: what a cable carries
gbps= says how much a cable can carry, in Gb/s. On a net line it covers
every cable in the net; on a link rule it covers that rule's cables and wins
over the net's:
net nica gbps=25 # 25G NICs
net fabric gbps=100 # 100G above
net mgmt gbps=1 traffic=no # drawn, but carries no load
link nica role=server role=tor scope=rack splice=4
link fabric role=tor role=spine
link fabric role=tor role=spine,model=new* gbps=400 # these uplinks are 400G
- Fractions are fine: a 100 Mb/s console port is
gbps=0.1. A value that is not a number from 0.001 to 1,000,000 is reported and ignored. - A cable with no
gbps=from its rule or its net has no capacity, which is not zero: the views below say so rather than painting it full. - A splice is a breakout: four 25G cables into one marker are a 100G trunk.
- Parallel cables are rules written twice (as in
examples/dual-plane.dc, four cables from every spine to every superspine); each carries its owngbps=, and they add up.
Where the capacity shows:
- The Networks panel puts each net's speed beside its name (
25G, or25G–400Gwhere rules differ), and marks atraffic=nonet. - The inspector gives every element's cables with what they carry: a
server's
nica ×1 · 25G each; a rack'snica ×8 (all internal) · 200G insideagainstfabric ×4 (all leaving) · 400G leaving— its oversubscription, at a glance. - Between picks, each route says the most the two could move over it —
the capacity of its narrowest hop, all that hop's cables together (
data: 4 hops · up to 25Gb/sfor two servers; a rack's uplinks for two racks). - width by capacity in the Networks panel draws every cable as wide as what it carries, on a scale where each step of port speed — 1G, 10G, 25G, 100G, 400G — is one step wider, with a key of the speeds in the layout. A lane beside a rack is as wide as the fastest cable in it.
Cable load: where a flow metric lands
A flow metric (samples tagged peer= — what iperf_orchestrator, mx export --peers and any a -> b table write) says how much one host sent to another.
Tick load the cables on its card and the viewer works out which cables
that traffic crossed and draws each one by what it carries. (Greyed out, it
says why under it: a percentage or a latency does not add up along a path.)
- Width is the traffic through that piece of cable. A rack's lane thickens on its way to the ToR, as it gathers its servers' cables.
- Colour is how full the fullest cable at that point is: its busier
direction against its
gbps=, green through yellow to red, and magenta for a full cable — one holding back the flows that cross it (see Contention). Cables carrying traffic with nogbps=are grey; idle ones stay, faint, in their net's colour. With no capacity anywhere, colour is the load against the busiest cable. - The panel names the busiest cables (
R02/torb → R02/s5 25G/25G · 100% · 8 held; click one to go to it) — Show more lists more of them, 25 at a time, and Show all every one (up to 10,000; past that it says so) — and the busiest switches, how many flows were routed, and any that could not be and why. The inspector gives the selected element's traffic out and in against its capacity, per net — for a rack, over its uplinks — and its busiest cables. - Expected, ticked in the inspector's Cable load, shows in place of the
selected element's own figures the average of every element of its class
that sits where it does, in every rack, row, room … (the kinds of
container it is in; the box says how many each finds): the tor b of the
first rack of every row, the third server of every rack. One draw of
random peers can give a ToR 400 or 600 where one like it carries 587 on
average; this is that average. Only the report changes — the load and how
it is routed are as set. Of one class means the same kind and
role=, or the samemodel=where roles are named per room.
How a flow is routed, which is a model, not a measurement — the measurement is end to end, and only switch counters see the cables themselves:
- Each flow takes the shortest routes, in hops, from the sample's target
to its peer: the target is the sender. Over
examples/dual-plane.dcthat is NIC, ToR, spine, plane and back down. - Only switches forward. A server with two NICs is not a path between its
two ToRs. Switches are the elements tagged
+switch; a layout that tags nothing that way forwards through everything, and the panel says which. A flow with no way through switches — between servers cabled only to each other in a ring — is listed as not routed; tag the hosts that do forward+switchand it goes. traffic=noon a net keeps it out: out-of-band mgmt beside the data fabric would otherwise take an equal share of every flow it ties with.- Where several cables continue a shortest route the flow is shared, by
the panel's choice. The model tries to do what a real fabric does, so the
choices are what real switches are set to do, and the default is what
nearly every Ethernet fabric does out of the box:
- ECMP: hashed per connection (the default) — each connection takes one cable at every fork, picked by a hash of its 5-tuple with a seed of each switch's own, as Arista, Cisco, Juniper and SONiC switches do. A handful of big flows can land on one uplink and overload it while its neighbour idles. Re-roll hashes again with another seed: the spread between rolls is the luck of the hash.
- ECMP, hashed alike at every tier (polarized) — the same hash at every switch, as switches left on one factory seed (or one config copied to every tier) do: flows that collide on a ToR's uplink collide again on the spine's, so the hot spots line up tier after tier. A misconfiguration to test for, and one of the first things to suspect when superspines flood while the average looks fine.
- adaptive routing (per packet) — InfiniBand adaptive routing,
Spectrum-X, a switch's dynamic load balancing: each packet goes out of
the emptier port, so every switch leans on its emptier cables until its
choices are about as full as each other. A switch sees only its own
cables, though: congestion further on is avoided only where an earlier
choice leads round it, so with a ToR's uplink out the spine left
carrying its traffic is no better off than under an even split. With
one uplink degraded to 40G in
examples/dual-plane.dc, all-to-all at 40% fills the busiest uplink to 44% where an even split fills it to 77%. A host does not route adaptively: a bonded one still hashes. - weighted by capacity (WCMP) — shares in proportion to
gbps=, as weighted ECMP (UCMP, the BGP link-bandwidth community) spreads over unequal links; - packet spraying (even split) — every flow's packets sprayed evenly over every shortest route, round-robin or at random, whatever the load: what a leaf that sprays does, its spines and superspines keeping to the route each packet was given. It needs NICs that put packets back in order, as RDMA NICs do. It is also the long-run average of hashing with endless connections, and the best any balancing could do on an even fabric: if a cable is full here, nothing will save it. Where spines also move traffic off a busy route, adaptive routing is the closer model; where they move it off a dead one, take that out with the what-if.
- connections a flow, beside the routing, is how many connections each
flow between two hosts is made of —
iperf -P, a test's parallel streams, an RDMA job's queue pairs — 1 by default. Each is hashed on its own, so this is the biggest single thing a hashed fabric's outcome turns on: inexamples/dual-plane.dc, every server sending half its NICs to the one sixteen places on (the shift pattern) fills the busiest uplink to 125% at one connection a flow, 75% at four, 63% at thirty-two, and 50% split evenly. Under contention each connection is its own share of a full cable, as TCP shares it: a flow of eight beside a flow of one gets eight shares, and a flow is held back only on the connections whose own path is full. One connection a flow is also what a hash on the IP addresses alone does, however many connections there are. weight=on a link rule scales its cables' share in every mode —weight=3takes three times what a weight-1 cable beside it does.count=on a link rule makes that many cables between each pair — a bundle's members:link plane1 role=spine role=superspine count=6 gbps=400is six 400G members per pair, drawn as one heavier line. Hashed, a flow lands on one member, so one 400G member can be full while the bundle reads 60% — what one 2.4T cable cannot show.bond=on nets bonds a host's cables on them:net nica gbps=100 bond=nicandnet nicb gbps=100 bond=nicmake every server's NIC a and NIC b one bond, run as LACP with the layer3+4 hash: each connection keeps to one NIC, picked by its own hash in proportion to the NICs' capacity — under hashed and adaptive routing alike, since a host's bond hashes whatever its switches do. With one connection a flow two NICs can come out a flow or so apart; with many they even out, and under the ideal even split every flow is shared over the bond exactly. A connection arrives on the same side it left, NIC a to NIC a, so in one rack it never leaves the ToR.balance=evenon the bond's nets (net nica gbps=100 bond=nic balance=even) shares every flow over the bond by capacity instead, under every routing — for a bond whose NICs carry the same whatever the connections, as round-robin bonding and some NIC drivers do;balance=hashis the default. Take a ToR or a NIC's cable out and the rest of the bond carries everything. Only hosts bond: a switch's cables on the same nets are what the hosts hang off. A host with cables in two bonds uses the bigger.- A pair's samples are reduced by the card's own aggregation (the mean of a
run's passes, by default). The metric must be a rate (
unit=Gb/s,Mb/s,MB/s,pps, … — or a name likeiperf_mbps_out) or a volume; a bit or byte rate is set against capacity, anything else loads without one. A percentage or a latency does not add up along a path, and is not offered. - Loads are kept per direction: a cable is full duplex, so 20 Gb/s each way on a 25G cable is 80%, not 160%.
examples/dual-plane-flows.tsv loads examples/dual-plane.dc: one rack's
servers sending 40 Gb/s each across the planes — 80% of their NICs and their
rack's uplinks — and an incast of eight servers into one, 160% of the two
NICs it lands on. Hashed, each 40 Gb/s flow takes one NIC, not two; under
the even split it is shared over both.
Contention: a full cable shares itself out
A cable cannot carry more than its line rate. What the flows offer — what their senders want — can add up to more, and what happens then is what happens on a real fabric: the cable's queue fills, packets are dropped (or marked, with ECN), and every TCP sender crossing it slows down until they share it. contention: full cables share out, on by default, models that steady state:
- Every flow is raised from nothing at the same pace until it has all it asked for, or a cable on its way is full; there it stops, at its share of that cable. This is max-min fairness, the textbook idealization of TCP flows sharing a bottleneck: no flow can get more without taking it from one that has less.
- A flow held back at one full cable carries less on every other cable
along its path too. In
examples/dual-plane-flows.tsv, eight servers sending 10 Gb/s each into one server's two 25G NICs each get 6.25 Gb/s, and that is all that crosses their own cables and the uplinks on the way — not the 10 they offered. - Full cables are magenta, and say how many flows they hold back
(
25G/25G · 100% · 8 held). The panel gives what contention cost:contention: 2 cables full, holding back 8 of 17 flows · 385Gb/s carried of 415Gb/s offered (93%). Hovering a cable gives what it carries each way of what was offered, and each flow over it, held back or not. - The split matters more with contention. Even shares every flow over every route, so a flow is held to its slowest one; hashed puts each connection on one path, as ECMP does with TCP, so only the connections hashed onto a full cable slow down — and two big flows hashed onto one uplink can leave its neighbour idle.
Untick it to see what the flows offer: then a cable can read past 100%, in magenta, which says how oversubscribed it is rather than what it carries.
bursts of × … for … µs says how bursty the load is. The model is a steady average, and real traffic comes in clumps — a synchronised start, an incast all answering at once — that last microseconds and overflow a cable while the second's average looks easy. Above 1, the panel counts the cables a burst that many times the average would overflow, and what becomes of each:
- What a cable cannot take at once queues at the switch sending onto it,
in its shared packet buffer,
buffer=(buffer=132MB). Every port overflowing at that switch shares it as a switch's dynamic threshold shares it: with one port congested it may fill half the buffer, with n of them 1/(n+1) each. A burst whose excess over its length fits that share is absorbed; one that does not drops. - All-to-all at 50% over
examples/dual-plane.dcwithbuffer=2MBon the ToRs, in 2.5× bursts: each 25G NIC is 6.25 Gb/s over for the burst, 78 KB in 100 µs, against the 222 KB its ToR gives each of its eight overflowing ports —64 absorbed by their switch’s buffer. At 1000 µs it is 781 KB, and64 drop — more than their switch’s buffer holds. - A switch with no
buffer=is counted apart (at switches with no buffer= to say), and a host sending more than its NIC takes queues in the host, dropping nothing on the wire. - A cable's tooltip says what the burst would need of it, how much queues
where, of how much:
78.1 KB queued at R03/tora, of the 222.2 KB its buffer gives this port: absorbed. - What drops is lost: the excess past the port's share, out of what the
burst sends onto the cable while it lasts. The panel and the headline give
it in all — the same load in 1000 µs bursts reads
a 2.5× burst for 1000 µs drops 35.8 MB on 64 cables — 7% of what they carry while it lasts— and a dropping cable's tooltip gives its own. It is a loss per burst, not a rate: how often bursts come is the traffic's, and TCP backs off after each, which the steady average does not see.
The buffer is the one figure that tells an incast a switch rides through from one that loses packets; real switches add per-queue reserves and priority queues the model leaves out, so read it as which bursts are close.
Sharing out costs in proportion to the cables every flow crosses, added up over the flows. Flows that leave the same switches for the same server share that part of the work, and the page shares out up to twenty million crossings: a thousand servers each sending to 150 random peers over a four-plane fabric is about nine million (fourteen with bonded NICs), and takes a few seconds. Past that the panel says too many flows to share out here, and draws what the flows offer, as if contention were unticked.
The model is the steady state, and real fabrics are rougher. TCP favours flows with shorter round trips; an incast of many senders can lose packets all at once and collapse well below the fair share while senders wait out their timeouts; UDP does not back off at all, so its excess is simply dropped at the full queue, after it has crossed the cables before it; and a lossless RoCE fabric drops nothing but pauses the senders upstream, which can spread the congestion to flows that never crossed the full cable.
How full a switch is
With load on the cables, every element tagged +switch has a gauge
along the foot of its box: filled as far as it is busy, in the colour a cable
that full would be, magenta when it is full and holding flows back. Its
tooltip, the inspector and the panel's Busiest switches say how much
passes through it against what it can pass:
- With nothing more said, a switch can pass what its ports carry, one
way —
switching 132.8Gb/s of the 500G its ports carry (27%)— and it is full only where its ports are; the tooltip counts its full ports. gbps=on a switch is its own forwarding capacity, as switches are rated: 64 ports of 800G isgbps=51200. Where that is less than its ports add up to, the switch is oversubscribed — the tooltip says by how much — and contention shares it out like a cable:switching 150Gb/s of the 150G it can forward (100%) · full — holding back 326 flows itself.
node spine1 role=spine +switch gbps=12800
pps=on a switch is the packets a second it can forward. Small packets fill a switch long before its bandwidth — what a matrix test of small requests does to a fabric — and with a load that knows its packets (a packet rate with a size, mx's own or the generator's) contention holds flows back there too. Overexamples/dual-plane.dcwith its spines atpps=2000000, every server sending a million 64-byte requests a second out of its rack readspassing 2 Mpps of the 2 Mpps it can (100%), of 13.3 Mpps offered · out of packets — holding back 640 flows itselfwith the spine's bandwidth all but idle. The busiest switches say· 100% of its pps, and the gauge fills by whichever is fuller.
node superspine role=superspine +switch gbps=51200 pps=8000000000
pps=on a server (or any host) is the packets a second it can send, and separately take in — what its NICs and network stack manage, which is often the real limit on a test of small packets. Contention holds its own flows back at that rate each way, before any switch is involved. Overexamples/dual-plane.dcwithpps=500000on every server and the same million 64-byte requests a second, a server's tooltip readspackets: sending 500,000 pps, receiving 500,000 pps of the 500,000 pps it can each way (100%)andout of packets — holding back 48 flows itself, and the panel's contention line says32 hosts out of packets. Like a switch's, it needs a load that knows its packets; one in bits alone leaves it be.
node s[1..16] role=server pps=20000000
gbps=on a server is what its NICs can move each way, whatever its cables add up to — a card in a PCIe slot narrower than its ports, two cards sharing one. Contention holds its own flows back there, as at a full cable, and its tooltip saysits NICs: 100% of the 20G they can move each way — full, holding back 62 flows itself; the panel countsN hosts full.
A gbps= on a rack or a room is not passed down to what is in it.
Device models: a datasheet written once
A model line defines a device once, and every element with model= its
name takes its settings and tags — u=, +switch, gbps=, pps=,
buffer=, latency=, ports=, breakout= and anything else — wherever it
does not set its own. Names match in any case; a model= no model line
defines is kept as a note, and one a letter off a defined name is named as
a typo. The two Arista switches, from their datasheets:
model 7050DX4-32S u=1 +switch gbps=12800 pps=5300000000 buffer=132MB latency=900ns ports=32x400,2x10,1x1 breakout=400:32,200:64,100:128,50:144,40:64,25:128,10:130
model 7060DX5-64S u=2 +switch gbps=25600 pps=10600000000 buffer=114MB latency=825ns ports=64x400,2x10,1x1 breakout=400:64,200:128,100:256,50:256,25:256,10:258
node leaf[1..8] model=7050DX4-32S
node spine[1..4] model=7060DX5-64S
node leaf9 model=7050DX4-32S buffer=64MB # its own setting wins
gbps=is one direction: what the ports add up to, 32 × 400G = 12.8T. Datasheets also quote the two directions added (25.6 and 51.2 Tbps); that is not the figure. These two are non-blocking, sogbps=never binds; it is there for one that is oversubscribed.pps=is the forwarding rate — 5.3 and 10.6 billion packets a second. Both are rated with packets averaging over about 295 bytes: from there up they forward at line rate, and below it packets, not bits, are the limit. 5.3 billion 64-byte requests a second, with framing, are about 5.5 Tb/s: a matrix run of small packets fills a 7050DX4-32S's packet rate at under half its bandwidth (seepps=above).ports=is what the device has, so many of so many Gb/s (1x1is the management port, for agbps=1mgmt net). Its cables are checked against it: each takes a port at least as fast, and a fast port breaks out into several slower cables — a 400G port is four 100G cables, two 200G or eight 50G.breakout=is the datasheet's most ports of each speed, which is a limit of the chip as much as of the cages: a 7050DX4-32S breaks out to 128 × 100G but 144 × 50G, not 256. Too many cables, a cable faster than any port, or more of a speed thanbreakout=allows is a warning on the device's line, once however many devices the line made.buffer=is the shared packet buffer bursts queue in (see bursts, above), andlatency=the switch's own time per packet (see Between picks).
A server's NICs are specified the same way, on the server: ports=,
gbps= each way, pps= each way and latency=. A Broadcom BCM57508
card in its 2 × 100G form (P2100G; 1 × 200G as the P1200G) sits on a PCIe
4.0 x16 slot, about 250 Gb/s each way before overhead — enough for its 200G.
Broadcom's DPDK reports measure it at about 106 million 64-byte packets a
second with no loss across both ports, which is the card's ceiling; a
kernel network stack is well below it, so measure your own servers for
pps= where the matrix runs use sockets:
model P2100G ports=2x100 gbps=200 pps=106000000
node s[1..16] role=server model=P2100G
A server with two such cards has twice each figure: ports=4x100 gbps=400 pps=212000000, or a card a net if they are cabled apart.
A server is more than its NICs, and two of its own limits usually set what a test reaches well before they do:
stream=— the most one connection carries, in Gb/s. On a kernel stack a TCP stream is mostly one core's work, and tops out around 30–50 Gb/s however empty the cables. It holds at either end, and a flow carries at most its connections (the routing box's connections per flow) times it:stream=40and four connections is 160.cpugbps=— what its CPU can send and receive together, in Gb/s: one budget for both directions, since the same cores do both. A server withcpugbps=310sends its 200 alone, but sending and receiving at once it manages 155 each way.rxcost=says how many times more receiving costs than sending (1 unless set): atrxcost=1.3it is about 135 each way. (cpu=stays free to name the part,cpu=epyc-9654, as layouts already use it.)cores=— how many cores it gives the network,cores=32. Each carries one connection at itsstream=, so a server sends and receives at most cores ×stream=together (receiving at itsrxcost=, as below) — whichever of that and itscpugbps=is less.stream=8.5 cores=16is 136 Gb/s in all: 68 each way, or all of it one way. With nostream=there is nothing for a core to carry, and it sets no ceiling.busy=— the share of its CPU other work takes,busy=20%(or a barebusy=20), off itscpugbps=(or its cores):cpugbps=310 busy=20%leaves 248 for the network. Write the idle figure once on the model andbusy=on the servers or lines that are loaded. With nocpugbps=it has nothing to take from, and the tooltip says so. Measured is better than written: with servers that have acpugbps=, the routing box offers CPU busy elsewhere, and choosing a loaded per-server CPU metric there — in per cent (unit=%), or unitless and named for the CPU (cpu_util,busy) with readings as shares — reads each server's share from it in place of itsbusy=. The metric must be ticked to count.
The routing box shows connections a flow whenever a server has a
stream=, packet spraying included: it sets what a flow may carry. The
Load generator has the same setting, always shown — connections a
flow, counted for each pair of hosts, not each server: a server holds
that many for every peer it sends to and every one sending to it. When
stream= holds generated flows below what they were asked, it says how
many, what one flow can carry at the connections set, and how many would
carry every flow in full, with a Use N button: 50 Gb/s from each server
over five peers is 10 Gb/s a flow, which stream=8.5 holds to 8.5 with one
connection (42.5 from each server) and two carry whole. It is what the run
files write as iperf's -P, mx's --streams and netmesh's flows=.
Beside it, cores per server says how many cores the run may use on
each — blank for what each server's cores= says, or no limit. It caps
cores=, never raises it, and holds the way cores= does: cores ×
stream= for sending and receiving together. The generator says how many
cores the busiest server needs for what it sends and takes in — that
50 Gb/s each way at stream=8.5 is twelve — and, when servers have fewer,
how many, with a Use N button. The run files give it to mx as
--workers N, one process a core.
So the three numbers are:
| Setting | Counted per | Holds | Run files |
|---|---|---|---|
| connections a flow | pair of hosts | stream= each |
iperf -P, mx --streams, netmesh flows= |
cores per server (or cores=) |
server | cores × stream=, send and receive together |
mx --workers |
cpugbps= |
server | the CPU's own budget, send and receive together | — |
Both hold flows back in contention
just as a full cable does; hovering a server says which it is at, and the
headline counts hosts out of CPU and hosts at their stream=. Telling the
two apart is one test: an iperf run with -P 1, -P 4 and -P 8 gains with
each extra connection only where the limit is per connection.
model P2100Gx2 ports=4x100 gbps=400 pps=212000000 stream=40 cpugbps=310
What the datasheets also give and the viewer does not model: power draw, fans and power supplies, memory, table sizes (128-way ECMP, routes, MACs), queues per port and QoS (PFC, ECN), MACsec, and the load-balancing features the 7060X5 adds — dynamic load balancing is close to adaptive routing, and Cluster Load Balancing places RDMA flows with a view of both leaf and spine. Write any of them as attributes for the inspector.
Findings: what went wrong, and whether the load explains it
The Findings box under the cable load lists, by kind, every server something went wrong on, and marks each on the floor plan with a corner — red for the critical, amber for drops, magenta for contention. Its tooltip and the inspector say what was found. Two sides feed it:
- The files. Any per-server metric that reports trouble: drops
(
rx_drop_per_s,tcp_retrans_pct,loss_pct, …), soft lockups or hung tasks (soft_lockups,hung_tasks), kernel errors (edac_errors,mce_count,oom_kills, …) and failed checks — any text metric readingFAIL,CRITICAL,WARNorNO-DATA, asreckon_verdictdoes. The kind is read from the metric's name;finding=on its!testline says it outright, orfinding=nothat a metric is not one. A zero, or anOK, is not a finding. - The load. With load on the cables: servers held back by contention — sending less than they offer, or receiving less than is offered to them, an incast when many send to one — and flows with no route. With contention unticked, a server offered more than its NICs carry.
Side by side they answer the question binnacle's reckon asks with
LOSS_BELOW_CAPACITY: is a server dropping packets because the load says it
must, or with room to spare? A drop on a server whose traffic is held back is
explained — that is what a full cable does. A drop on a server whose cables
the load leaves short of full is loss the load does not explain, the one to
go and look at, and the box counts those first.
examples/dual-plane-health.tsv beside examples/dual-plane-flows.tsv shows
both: the incast target drops packets and the load explains it, one other
server drops with room, and two log soft lockups — one of them an EDAC error
and a failed check too. With the load generator, the same: run a load, and
the findings say which of the servers' troubles that load accounts for.
Between picks: only their flows, and a test flow
Picking two or more elements decides which cables are drawn — the routes between them — but not which traffic is counted on those cables: every flow in the metric that crosses them still adds to their width. With two or more picks the picks box offers two things more:
- count only the flows between the picks loads the cables with just the metric's flows that start in one pick and end in another (either way; a rack picked holds everything in it). The panel says how many of the metric's flows that is. For per-host totals it keeps only the host pairs between the picks, each at the share it has in the whole estimate.
- test flow [N] [unit] 1 → 2 · Send routes N from the first pick to the second and draws it like any metric — with nothing loaded, on its own and in Gb/s, Mb/s or pps; with a metric loaded, on top of it and in its unit, so you can see whether one more flow fits. A rack picked sends (or receives) it shared evenly over its devices. Stop, Esc or Clear ends it.
A single test flow, hashed as the page routes it by default, takes one uplink, so every cable from server to server is drawn the same width; the even split (packet spraying) shares it over every uplink — four spines, a quarter each.
Between two devices the picks box also gives the latency floor of one
packet's trip, at 64 and 1500 bytes, over every shortest route: each
switch's latency=, and each cable's time to send the packet — once for a
run of cut-through hops, and again wherever a switch must hold the whole
packet first because the cable out is faster than the cable in. Over
examples/dual-plane.dc with latency=900ns on its switches, two servers
in one rack are 942 ns at 64 B, 1.4 µs at 1500 B · 1 switch; in two racks,
2.78 µs at 64 B, 3.7 µs at 1500 B · 3 switches — the ToR stores each packet
where 25G steps up to 100G. A latency= on either host — its NIC and
stack — is added too. Not counted: the cables' length (about 5 ns a metre);
a switch with no latency= is counted as none, and said so.
With a load on the cables, a second line gives the trip under this
load: each cable's queue added. A cable a fraction ρ full holds a packet
about ρ/(1−ρ) of the load's packet times — the single queue's textbook
figure, 9 at 90% and 99 at 99% — and a full one as long as its port's share
of the sending switch's buffer= takes to drain, which is also the most
any queue can hold it. The load's packets are its own size where it says
one, else IMIX. Over the dual-plane fabric with buffer=2MB on its ToRs,
two racks apart, a load of 150% shared out fills the ToR's downlink, and a
1500-byte packet takes about 75 µs where it took 3.7 — 71 of them waiting
for 222 KB to drain at 25G. A full cable at a switch with no buffer= is
said to have no ceiling known, and a host's own full NIC queues in the host,
which is not counted.
The load generator: a load with no file
The Load generator box under the Networks panel puts a made-up load on the cables — nothing to measure first, nothing to load. Run routes it and draws it like a metric; Stop, or Off in the cable load panel, takes it off.
-
The pattern:
- all-to-all — every host sends to every other host, an equal share each;
- to random hosts — every host sends to
×Nhosts picked at random, an equal share each, and is sent to by exactly N: the peers are drawn as N random pairings, each one in which every host sends to one host and receives from one, so no host is picked more often than another and every server's own cables carry the same. It is the way to load a fabric too big for all-to-all — a few thousand servers at 16 or 64 peers each — and the panel says how many peers fit when all-to-all is too many. The same seed picks the same peers; New peers draws them again. One draw can land a server's peers inside its own rack or not, so a switch's figures move from draw to draw: the inspector's expected shows what one like it carries on average (see Cable load, above); - incast: all to one — every host sends to one target, the first pick or else the selection: many to one, the pattern that fills a single server's cable. A rack as the target shares it over the rack's devices;
- one to many — the target sends to every other host, as a storage server feeding its clients: its own cables fill outward;
- hotspot: a share to one — every host sends a share of its load to the target (20% unless set) and the rest to ×N random peers: a few servers everyone calls, over a background;
- shift: each to one, k on — every host sends everything to the host k places on, in the layout's order, and is sent to by one. With k empty, half the hosts on, so every flow crosses the middle of the fabric: the classic worst case for ECMP;
- ring: each to the next — shift by one, the last to the first: how an all-reduce passes its data round.
-
each sends how much every host sends in all: in Gb/s, Mb/s or pps, or % of its NICs — a share of what the host's own cables can carry, their
gbps=added up, which is what it can send one way — which is the default once the cables have agbps=. 50% of two 25G NICs is 25 Gb/s, so 50% and 25 Gb/s are the same load; the box says what the share comes to from each host. -
in all or to each peer — who that amount is for:
- in all, the default, makes it the host's total, whatever the pattern: more random peers spread the same load thinner — at ×8 each flow is an eighth of it, at ×150 a 150th — so the cables carry much the same at any count, and only the number of flows changes;
- to each peer (to each other host for all-to-all, to each target device for incast) gives every flow the whole amount, so a host sends it times its peers and every peer adds load: 1 Gb/s to each of 150 peers is 150 Gb/s from every server, past two 25G NICs, and contention shows where it gives. A share of the NICs works the same way: 1% to each of 150 peers is 150% of them.
Read off a metric, the amount is always the host's total: a reading is what the server sent in all.
-
requests of … B, replies of … B — in pps, the packets' size: a made-up matrix_orchestrator run. Each host sends its packet rate as requests of that payload (plus 66 bytes of framing on the wire), and every request is answered by a reply of its own size, the other way. The load is then a bit rate — set against
gbps=, shared out where full — and the headline gives the packets beside the bits. Overexamples/dual-plane.dc, all-to-all at 1,000,000 pps of 500 B requests and 4096 B replies is 32 Mpps each way and fills every server's NICs to 76%, almost all of it the replies coming back. Choosing pps starts at IMIX requests and no replies: 316 B of payload, the average of the standard 7:4:1 mix of 64, 594 and 1518-byte frames. Real datacentre traffic is two-humped — small acks and requests beside full-size data — and averages a few hundred bytes, and switches are rated for their packet rate at about this size; 1500 B would make a packet-rate limit look harmless. One average size is exact here: bits and packets both add up linearly, so a mix and its mean load the cables the same. 64 B · IMIX · 1500 B beside the box set each at a click, here and on a measured packet rate's own row. With the request size emptied, pps is plain packets, drawn by load alone. -
to hosts — anywhere; within its own rack (row, pod, room — the kinds in your layout), where every rack is a load of its own, its servers sending to each other in the pattern as if it were the whole floor — a test run rack by rack, or row by row — and nothing goes above that level; or outside its own rack, the other way round: every flow leaves the sender's rack, which loads the uplinks and nothing inside a rack. Explaining results uses the same choice, so set it to how the test ran.
-
hosts — which devices send and receive, as a filter:
role=server,+gpu,role=server in:A[01..04], or several sections at once,role=server (in:A01 or in:B[03..05]); the box counts the hosts it takes as you type. Empty means every device with cables that is not tagged+switch, and a filter only narrows those: naming a rack takes its servers, never its ToRs. A layout that tags nothing+switchcannot tell a server from a switch, so the box says so and every device with cables counts. -
split — the hosts as groups that share none, and what they are for: peers only within each, and host lists to save. Splitting them:
Split Groups each rack (row, pod, …) a container each each pick each Ctrl-click pick: a rack, a row, a server each rule a filter each, one a line or ;apart:in:A01 ; in:A02 ; rack=R[05..08]into blocks of N N runs of neighbours, in the layout's order groups of N runs of N neighbours, the last what is left interleaved N dealt a host at a time, so every group spans every rack at random N dealt at random; Reshuffle deals again by attribute a group for each value of one: model,plane, …No host is ever in two groups: picks or rules that overlap — a server inside a picked rack, two rules that both hold
r1h1— are refused, naming where, rather than one of them quietly winning, and nothing runs until they are fixed. Hosts no pick or rule holds are counted, and send nothing. The row says what the split comes to ("3 groups: g1 2 · g2 2 · g3 2").- peers only within each group — each group a load of its own, its hosts sending to each other in the pattern (all, random peers, shift, ring) as if it were the whole floor: the disjoint sets of a test that runs several at once. Within its own rack is the same thing with the racks for groups; this is any groups. to hosts, outside its own rack, still holds inside a group.
- Save host lists —
hosts-lists.tar: ahosts-<group>.txta group, a host a line as the load tools take one (name, orname=addr), and all of them inhosts-grouped.txtwith a[group]header each — the groupedhosts.txtmodel_network's autopilot, iperf2 mesh and topology discovery read. Taken-out hosts are left out. Each list goes as it is to--servers, or into binnacle'smuster addto be handed out a batch at a time. - An area each — every group an area of its own (below), with this
area's settings and the group's hosts as its filter: a container's
in:/D/A/r1, a pick's, a rule's, an attribute'splane=2, or the hosts themselves by path,path=[D/A/r1/h1,D/A/r2/h1].
-
area — several loads at once, each with settings of its own on hosts of its own: an all-to-all at 20 Gb/s in one pod while another runs an incast of 64-byte packets. + area adds one like the area shown, with no hosts yet; the tabs show each, a name box names it (its files are named for it), and Remove takes it away. Run runs every area; the cables carry them all, shared out together, and the panel says what each sends ("A: 3 of 3 hosts send · 6 flows · 60G").
No host is in two areas — unless you say so. Two areas whose filters take a host — including an area with no filter, which takes every host — are refused, naming the areas and a host they share, and nothing runs until each has hosts of its own. An incast's or outcast's target is the first pick (or the selection) inside its area and in no other, so no area reaches into another's hosts. Connections a flow and cores per server are the whole run's.
areas may share hosts lets them: two things running on some hosts at once — a storage load on every server while one rack also takes an incast. A host in two areas runs both, and the model shares its CPU and NICs between them as it does any two flows; the row says how many hosts are shared. What the tools need for it is ports of their own and stopping only their own, so with hosts shared each area's run listens a hundred ports on from the one before (mx 5300, 5400, …; netmesh 5310, 5410, …; iperf 5001, 5101, …), the notes list them to open, and the Host status pane's preflight checks them all. mx's workers share a port by
SO_REUSEPORT, so two mx runs on one port would quietly take each other's packets. Stopping by pid is the other half: mx 1.11.2 and netmesh from the binnacle after 0.8.0 stop only their own agents, where older ones also kill every other agent on the host when they stop. iperf-orchestrator already stops its servers by port.The run files make each area a run of its own, all started at once, so no run reads, writes or cleans up after another — the tools' own options for runs side by side, nothing new on the hosts:
Tool Each area's own mx matrix-A.csv,--remote-dir /var/tmp/mx-A,--reports reports-A,mx_results-A.tsviperf-orchestrator iperf_plan-A.conf(remote_dir=in it),--run-id dcviz-A,iperf_results-A.tsvnetmesh mesh-A.csv,--remote-dir /var/tmp/netmesh-A,--reports reports-A,netmesh_results-A.tsvEach area has its own
expected-A.tsv, and its run file,mx-run-A.tsv, carries the layout and says which area it was (#~run area A): load them together and they land side by side. The script checks every area, starts every area's run in the background, each to its own log (mx-run-A.log), waits for them all, and then exports and collects each. -
Take out at random — servers, racks and switches, each by a number or % of the ones there are, and each among all of its kind or those a filter names (
role=tor,role=compute,ht11r[01..04]*); the row says what it comes to ("→ 41"). Take out draws them into the what-if beside anything taken out by hand — their cables carry nothing, the load goes round them, and the panel says what it cost — and the what-if shows them as one chip, "41 at random". Draw again takes out a fresh set in their place, and Put them back returns them, leaving what was taken out by hand. A rack is any rack with a cabled device in it, the switches' racks included; a server inside a drawn rack still counts as a server drawn only if it was drawn itself. -
from — an amount of the generator's own, or what a metric says: pretend the servers make the load they measured. Any per-host metric that is loaded and ticked can be picked: a rate (
egress_gbps,tx_mbps) is read as what each server sends, a percentage (unit=%, a NIC utilization, say) as that share of the server's own NICs. each sends is then a share of the reading — 100% as measured, 200% for twice it — and the pattern still says where it goes. Servers with no reading send nothing, and are counted. (load the cables (estimated) on a rate metric's card is the other way to the same idea: every host's total sent to the others by the gravity model.)
With a metric loaded onto the cables, the generated load goes on top of it, in a unit that adds up with the metric's (a share of the NICs only where the metric is a bit rate). A test flow between picks goes on top of either.
Every pattern is routed like any flow metric, by the panel's split. Over
examples/dual-plane.dc, all-to-all at 50% fills every server's NICs to 50%
and the ToR uplinks to 39%; outside its own rack, every flow takes an uplink
and they fill to 50%. Generating is
quick, routing is the cost: every destination is a walk over the whole
fabric, so a load too big to route in the page is refused with why. That
walk is shared by every server behind the same switches, so it is done once
per ToR (or ToR pair), not once per server: random peers over four thousand
servers route in about a second. All-to-all grows with the square of the
servers — a thousand of them is a million flows, past what the page will
route — so above a few hundred, send to random hosts instead, with as many
peers as the panel says fit, or narrow the hosts.
Run it for real: the files and the commands
Under a running load, Run it for real hands over what each load tool needs to make that same load on the real servers — its input files, to save, and the commands that run it and export what it measured for this viewer. Nothing is run from the page. Set how long the run lasts (60 s unless changed) and the fabric's MTU (1500 or 9000), pick a tool, and save its files into one directory:
| Tool | Files | What it is given |
|---|---|---|
| iperf-orchestrator (iperf_orchestrator) | iperf_plan.conf (one per rate), hosts.map |
a pair grid of who sends to whom, -b each stream's share of the pair's payload, -P the routing box's connections a flow |
| mx (matrix_orchestrator) | matrix.csv |
each pair's requests a second; a sized load's request and reply sizes as they are, a bit rate as full frames at the MTU — 1472-byte requests at 1500, 8972 at 9000, a sixth of the packets — answered by mx's smallest reply, 32 B; --streams the connections |
| netmesh (binnacle) | mesh.csv |
the load's pairs, probed ten times a second for latency, jitter, loss and path MTU |
The commands come with them, ready to copy — for mx, say:
mx check --matrix matrix.csv # what each host will send and take in
mx run --matrix matrix.csv --for 60 --streams 1
mx export --matrix matrix.csv --peers -o mx_results.tsv
Save run-mx.tar puts all of it — the tool's files, expected.tsv,
the layout as model.dc and the script — in one archive, a folder of its
own, the script executable:
tar -xf run-mx.tar && cd run-mx && ./run-mx.sh
One download rather than four, so a browser cannot quietly save a second
run's matrix as matrix (1).csv beside the first one's matrix.csv.
Save run-mx.sh saves the script alone: every note
a comment, every command as it is, and a check that its files are beside
it before anything starts. Save it with the files, change what you like —
--for, --user root, a line commented out — and run bash run-mx.sh.
For iperf at more than one rate its runs go together, in the background,
each into its own log, and are waited for before the export; for netmesh
the run under the load is there commented, to swap in for the idle one.
Each tool's export lands back on the floor plan, because the hosts go
into the files by the layout's own names: load mx_results.tsv,
iperf_results.tsv or netmesh_results.tsv (made by dcimport) beside the
layout, and compare what ran with what the model said.
-
The NIC to load goes in the box beside the MTU, as mx's and iperf's
--bindtake it: an interface name (eth1,mlx5) or part of an address (10.20.), matched againstip -o -4 addr showon each host, for a fleet reached over a management NIC whose traffic should ride another. It goes into the files — iperf's plans asbind=, which every command run against the plan reads; mx's and netmesh's runs as--bind, the mx run under netmesh too — and is kept for next time, so the files need no editing. netmesh reads it as a NIC from the binnacle after 0.8.0, which also points each host's probes at its peers' matching addresses; an older netmesh took only a whole address and refused a name (cannot bind data:5310 -- Name or service not known). Empty, there is none. A pattern a shell would expand (eth[12]) goes quoted; one with a space, a quote or shell syntax in it is refused, and the box says why. -
A host the tools cannot reach by its name gets an
addr=— the address they reach it at,node s7 role=server addr=10.1.7.15, or one per server from the line that makes them,addr=10.1.{rack}.{id}. An address is one device's own, so a rack'saddr=is not its servers'. mx and netmesh take it asname=address; iperf tests the address, andhosts.mapturns it back into the name for the export. -
Names must tell the hosts apart, and be letters, digits,
.,_and-: a layout whose racks all hold ans1cannot say whichs1a result is for, so the panel asks for names first (name={rack}{id}). -
The cables carry exactly what the model shows. iperf paces and reports TCP payload, and every packet also carries 52 bytes of TCP/IP headers (timestamps on, as Linux sends them) and 38 of Ethernet framing — 1448 bytes of payload in every 1538 on the wire at MTU 1500. So
-basks for that share of the model's rate: 941.5m for each 1 Gb/s on the cable at 1500 (94.1%), 990m at 9000 (99.0%). The comparison knows it: an iperf metric is set against the model's payload, whichexpected.tsvcarries asmodel_payload_gbps, so a run that did as asked reads 100%. A stream slower than about 100 Mb/s is written 10 ms at a time (length=, iperf's-l): iperf2's own 128 KiB writes are refused under about 1 Mb/s, and overshoot a short run above it. mx needs no such allowance: its packets are counted with their framing on both sides. -
iperf's
-bis one rate a run. A load where every pair sends the same — all-to-all, random peers, a ring at one amount — is one plan. Pairs at different rates make a plan per rate (up to eight; more are binned), each on its own port so they run at once in separate terminals, and their exports go into oneiperf_results.tsv. Runs at once each keep a remote directory of their own (remote_dir=/tmp/iperf_orchestrator-dcviz-2in the plan): iperf-orchestrator ends a run by removing its--remote-diron every host, and a host in two rates' runs would otherwise lose the files of the one still running. iperf sends TCP, so a sized load's replies go back as TCP the other way at their own bit rate. -
mx's workers are sized to the load, or to the cores. A pair's connections are mx's
--streams— its rate split over that many sockets — but what does the work on each host is--workers, separate processes, one a core, that each sustain about 200k packets a second of request and reply. With cores per server set, that is the number: the run line says--workers N(even at 8 or fewer, sinceautotakes whatever the box has), and the notes say if the load would want more. Without it, the files size them asmx hintsdoes — twice the busiest host's requests a second (out, or in, which it answers) over 200k, plus one, and never fewer than the connections a flow — capped at the fewestcores=among the run's hosts, since mx runs one number everywhere; the notes say when the cap cut it.--workers autoruns one a core and stops at 8, which a full-frame load at tens of Gb/s outruns (50 Gb/s of 1472-byte requests is 4 million a second), so past 8 the run line says--workers N, the mx run under netmesh too. iperf has no cores setting:-Pis threads within each iperf and the kernel spreads them, so its notes count the connections the busiest host holds against the cores it has — fewer connections than cores leaves cores idle, and the notes say to raise connections a flow. netmesh is one light agent process a host, whatever the cores; they go to the mx run under it. -
netmesh makes no load. It measures the network: run alone it probes the idle network;
netmesh run --mesh mesh.csv --baseline 20 -- mx run …probes 20 s idle and then under the load, which is the latency the model's queues predict. Of the generator's settings it takes only which hosts talk to which (the pattern, the hosts, the peers), how long, and the connections, as probe flows a pair (flows=). Every pair gets ten 64-byte probes a second whatever the load's rate: the rate, its unit and the request and reply sizes go only to the mx run under it. Nor is the MTU given to it — it finds each path's MTU itself — and the MTU box is what the model expects it to find. Its notes andmesh.csvsay exactly this, rather than describing the load as if netmesh sent it. -
Flows to or from anything taken out are left out — nothing runs there — and the panel says how many.
-
expected.tsvgoes with every tool's files: the model's own figures for this load — what it expects each flow to carry (model_gbps, andmodel_ppsfor a packet tool), what each was asked to send (model_offered_gbps), where it holds one back (model_held), and what each host sends and takes in. The last command collects the run with it, stamped with when the run began and ended, the host it ran from and the tool's own--version:{ cat expected.tsv echo "#~ran started ${started:-unknown}" # set as the script begins echo "#~ran finished $(date -u +%Y-%m-%dT%H:%M:%SZ)" echo "#~ran host $(hostname 2>/dev/null || uname -n)" echo "#~ran mx $(mx --version 2>&1 | head -n 1)" cat mx_results.tsv } > mx-run.tsv # one file: the run and its model
The run carries its layout
A run is only as good as knowing what it was run against, so the layout
goes with it. The tar holds it as model.dc (kept with the run, never
needed to run it: the script does not wait for it), and expected.tsv —
so every mx-run.tsv made from it — carries it whole, as #~ lines that
every reader of a results file skips as comments, with its sha256 and how
the load was set up:
#~run made 2026-10-28T14:02:11Z
#~run viewer datacenter_visualization 1.42.0
#~run tool mx
#~run layout floor.dc
#~run load the random pattern to 1 peer, 1000 pps from each host
#~run seconds 60
#~run connections 1
#~run mtu 1500
#~run bind eth1
#~run out DC1/A/r3
#~model sha256=9f2c… lines=212 name=floor.dc
#~ dc DC1
#~ row A
…
#~end model
#~ran started 2026-10-28T14:03:00Z
#~ran finished 2026-10-28T14:04:05Z
#~ran host jumpbox
#~ran mx mx 1.11.1
The sha256 is sha256sum model.dc's, so a layout file on disk can be
checked against a run without the viewer.
Load a run file and its layout comes with it. Into an empty viewer it simply loads. Over the very same layout nothing changes, and the report says it matched. Over a different one the viewer asks first — the layout there now, and any edit not saved, would go — and Cancel keeps it and loads the readings onto it, with a warning that they may land on elements other than the ones they were measured on. A layout loaded in the same drop as the run file is your choice and stays. A carried layout that no longer matches its sha256, or was cut short, is not loaded, and the report says why. The report also gives each stamped file's line: made when and by which version, ran when, where, and with which tool.
Measured against the model
Load a run's results — mx-run.tsv, or expected.tsv and the tool's export
side by side — and Measured against the model, under Findings, sets
what was measured beside what the model said that load would do. It
appears whenever a rate is loaded, per flow (peer=: iperf_mbps_out,
mx_peer_pps) or per host (mx_egress_gbps), and compares it against
the expected.tsv loaded with it, or the load generator as it is set now
when there is none.
43Gb/s measured, of the 50Gb/s the model expects on the same flows: 86%
30 flows, within 15%: 20 as modeled, 5 short of it, 5 not measured
6 hosts: 4 as modeled, 1 short, 0 over, 1 with nothing measured
Where to look: what the short flows have in common
r1h1 sending: 5 of its 5 flows short
1 host off the model by more than 15%
r1h1 sent 3Gb/s of 10Gb/s (30%) · 5 of its 5 flows, every one
1 host with nothing measured
r2h2
- Flow by flow, within the tolerance either side of the model (15%
unless changed) is as modeled, below it short, above it beat it; a
flow the model has and the run does not is not measured, one the run
has and the model does not not in the model. A bit rate is compared in
Gb/s; requests a second (
mx_peer_pps) in requests a second — or, against a model in Gb/s, as bits at the size the run says its packets were. - Against the model, not against what was asked. A flow the model expects to be held back at a full cable, measured so, is as modeled, and listed under As the model said with where it was held. What falls short of that is the surprise.
- Where to look: what the short flows have in common that the flows that kept up do not. A place — a server, a rack, a row — nearly every flow out of (or into) fell short: one slow server names the server, four racks at half rate name the four racks. A switch or a cable most short flows cross and almost no good one does: a rack's uplink, when what leaves the rack falls short and what stays inside it does not. The widest place is kept over everything inside it — a rack, not its ToR, its uplinks and its sixteen servers. Each side is routed one flow a pair, so below the layer where routes fork it is as exact as the layout; above it, as good as the model's guess of which way each flow hashed. Hosts are named as the run named them where that name is theirs alone.
- Hosts: one that is off the model as a whole is one line, not a line per flow; one the model has sending and the run has nothing from is listed apart — usually a host that never ran its part.
- Loss: with a loss metric per flow loaded too (
mx_peer_loss, netmesh'sloss), flows losing more than 1% are counted, and those where the model has room called out by pair: loss the load does not explain. - Round trips — netmesh's
rtt_p50andrtt_p99, or any per-pair metric in a time unit (ns,us,ms,s) whose name says round trip, RTT or latency — are set against the fabric's own round trip for each pair: there and back, 64-byte probes, every switch'slatency=and every cable's time to send. That needs no load and noexpected.tsv— the layout alone says it, so Against offers the layout — and netmesh'sexpected.tsvfrom Run it for real carries it too, asmodel_rtt_us(andmodel_rtt_load_us, with the generator's queues). A measured round trip is mostly the hosts' stacks and the prober, much the same for every pair, and rarely in the model; so that part is found, not assumed: the median of what each pair measured beyond the fabric is common to every pair, added to each pair's expectation, and a pair more than the tolerance past it is slower — a path, not the hosts. The panel says what to put in the model for the common part, alatency=on the servers (a quarter of it: a round trip crosses each end's stack twice). Measured under the load sets it against the fabric with the generator's queues, for a netmesh run beside an mx one. - Path MTU: with netmesh's
path_mtuloaded, the pairs whose path carries less than the fabric's MTU (the one in Run it for real, orexpected.tsv'smodel_path_mtu) are listed, with where to look — a port, a switch or a tunnel set lower. Itslossis listed as a rate tool's is. - Requests alone. With replies on, what a host sends is its requests
and the replies it owes the hosts that call it. mx's
mx_request_gbpscounts the requests alone, so it is set against the model's requests —expected.tsvcarries them asmodel_request_gbpswhen there are replies — and the panel says so;mx_egress_gbpsis the figure with the replies. - An agent at its ceiling. With the run's
mx_agent_cpu(or netmesh'sagent_cpu) loaded, the hosts that fell short with their agent at 90% of a core or more are named, in amber, above the rest: mx's agent is Python, a core a worker, and short for want of CPU looks just like short for want of a cable. More workers (mx start --workers), more--streams, or bigger packets. - Show on the floor plan loads the comparison as overlays of its own —
vs_model_pct, each host's sending as a share of what the model expects, red to blue around 100%;vs_model_flow_pct, each flow's; andvs_model_verdict— and Save vs-model.tsv keeps them as a file to load again later, its summary and suspects in its header.
Hovering a cable
The pointer on a cable outlines it, and its tooltip says:
- its net and its two ends —
TOR to spine: R01/tora ↔ SP1/spine1; - what it can carry —
1 cable · 100G, or for parallel cables drawn as one,4 cables · 100G each, 400G in all; - with load on the cables: what it carries each way, and how full it is the busier way — or, comparing, now against before;
- and, once the pointer rests on it a moment, the flows that cross it, biggest first, with how much of each: routed exactly as the drawing was, so they add up to the load shown. A test flow is marked as one.
Where several cables are drawn as one line — the stub out of a ToR carries
every cable it has, a rack's lane every server's — the pointer is on all of
them, and the tooltip says so: NIC a: R01/tora ↔ 4 spliced (s1…s4), …, in
and out of the ToR. Inside a device's own box, the device is what is shown
unless the pointer is right on a cable.
Every cable is full duplex, and its load is told each way: a NIC carrying 88 Gb/s out and 88 Gb/s in is 88% full, not 176%. A flow listed over a cable shows the part of it that crosses that cable — under an even split over two NICs, half of it each.
The headline
At the top of the Cable load panel, one number: what the hosts deliver to
each other in all — with contention, of what they offered — and over how
many flows. Under it, each net's load, both ways, against what its cables
that carry any of it can carry: plane-1 57.6Tb/s of 115.2T · 50% says how
much crosses the superspines. The nets are not added up, since a flow
between two rows crosses NICs, uplinks and planes and is counted on each.
Hovering a server
With load on the cables, a server's tooltip says what it sends and receives
in all, and how full that is of its cables' capacity, the busier way —
sending 25Gb/s · receiving 25Gb/s — 50% of its 50G — then each of its
cables, out and in, and how many of the flows start and end there. Shared
out, it also says what it offered where a full cable held it back, and how
many of its flows were: an incast target reads receiving 50Gb/s, of 775Gb/s offered — 100% of its 50G … 31 held back by a full cable. A switch
says how much it switches instead (see How full a switch is).
Per-host totals: an estimate
A metric measured per host, with no peer= — mx_egress_gbps, or any
egress_gbps <host> 30 — says how much each host sent but not to whom. Its
card offers load the cables (estimated): each host's total is sent to the
other hosts measured, then routed like flows.
- to each host by its own total (the default) sends more to the hosts that are busy themselves — the gravity model, the standard first guess at a traffic matrix from per-host volumes. A host sending 10 among hosts totalling 10 and 20 sends them a third and two thirds.
- evenly to every other host sends each the same share.
- to its peers, as the test paired them sends each server's total to its random peers instead, in the layers set under it — the same setup explaining uses: × peers, within or outside its own rack (row, room, …), and a share for each layer. A few flows a server rather than one to every other, so it shares out and its replies come back however many servers there are. It is taken by itself where every pair would be too many flows to share out — a thousand servers are a million pairs — and the panel says so.
- Every host still sends exactly its total; what is a guess is only where it goes. The panel says the load is estimated and from how many hosts, and an estimate is always shared as a fluid — hashed or adaptive routes it as an even split, since one path per guessed flow would read as a measurement.
- Only devices count as hosts: a reading on a rack or a room is left out, and the panel says how many were.
- The metric must be a rate (
unit=Gb/s,pps, …, or a name likemx_egress_gbps). A temperature or a bare count is not offered. - treat as duplex reads each total as both directions added — what
iperf --bidirreports — so half of it is what the host sent and half what it received. It is ticked by itself when the metric's name saysduplexorbidir, or its!testline saysduplex=yes(duplex=nosays it is not). It is offered unticked when hosts read past what their cables carry one way — a measured 309 Gb/s from a server with two 100G NICs cannot be what it sent, and the panel says so in red, naming the worst. Ticked or unticked by hand, it stays as set for that metric. The same box sits in the load generator when it reads the metric.
examples/dual-plane-hosts.tsv gives each server of examples/dual-plane.dc
a total: a busy rack, and one hot server in a quiet one.
Explaining the results by the network
The estimate above asks where would these totals go? Ticking explain these results by the network, under the same per-host load, asks the question the other way round: if every server tried to do exactly the same thing and the network is what stopped them, what would the network have to look like to produce these results?
-
What every server tried is set in the panel itself, to match how the test was run, at the best server's result:
- the pattern — random peers (the default, one each), all-to-all, a ring, a shift;
- × peers for random peers: how many each server sent to;
- where — anywhere, within its own rack (row, room, …) or outside its own — the same choices as the Load generator's to hosts. A test run rack by rack is within its own rack: left at anywhere, every server is assumed to have sent across the whole floor, over the superspines.
- + layer adds another set on top, with % of it — the share of each server's traffic that layer carried. Two tests run at once, iperf to 4 peers inside the rack and mx to 8 anywhere, are two layers; each draws its own peers. "to 4 random peers within its own rack (25%) + to 8 random peers (75%)" says what was assumed.
- New peers draws every layer's peers again; From the Load generator starts from its settings as one layer.
The setup matters: explaining two-layer results as one layer anywhere finds the right cable at the wrong speed and reproduces fewer servers. Change the pattern there, and the routing (packet spraying, hashed, …) in the routing box, to match the test that was run.
-
Each server that fell short of that is a server the network has to explain. The search looks for the cables those servers share, and for each one the capacity that holds every server crossing it back to what it measured — running the same max-min contention as the load itself, so a cable shared with a fast server is not blamed for a slow one.
-
A cable that comes out under its
gbps=is degraded — a lane down, a bad optic, a speed mis-negotiated — and says what it runs at: "runs at 30G of its 100G · 8 servers". One that only needs to be full at its rating is full: simply more offered than it carries. Click either to select its sending end; hover for the servers it explains. -
A server no shared cable explains is held back by itself — its NIC, CPU or stack — and is named as such rather than blamed on a cable.
-
As the layout already has it: a short server that the layout as it stands already holds to its result — cables full at their
gbps=, or its ownstream=,cpugbps=less what it was busy with — is named as such, with how busy it was, and no cable is changed for it. -
A packet rate with its packets sized (mx's) is read in bits: each server's requests and the replies it sends back.
-
The cables in explained by the network carry the load as the explanation shares it out, so none reads past its capacity but a suspect, against what the results give it.
-
The servers themselves: when most of them stop at the same result, and the network as the layout has it would carry more of the same load, the limit is theirs — "32 of 32 servers stop at about 155Gb/s, where the network would carry 200Gb/s". It gives the
stream=(the result over each server's flows and connections) orcpugbps=(sending and receiving the result at once) that would reproduce it; write either on the servers and the panel has nothing left to explain. Where cables explain the results too, it is offered as the other answer, after them. -
When even the best server sits on a degraded path (all-to-all, where every server crosses the bad cable), the best result is not what was tried: the search tries higher targets and says so — "above the best result of 31G: even the best was held back".
-
The panel ends with how well it fits: how closely the model, with those cables changed, reproduces every server's result, and how many come out within 5%. A poor fit is in red — the pattern, peers or routing likely do not match the test.
-
The cable view gains explained by the network: the load as it would be, with each suspect cable's share read against the capacity the results give it.
-
It is worked out in turns, the page answering between them, and on thousands of servers stops after twenty seconds with the cables it was surest of — the panel says when it did.
It is an inference, not a measurement: it finds the fewest cables that explain the results, and a different set may explain them as well. Switch counters, below, are what settle it.
Switch counters: measured, on the cable
Interface counters say what each cable actually carried. Write one sample per
switch port, naming the neighbour the port faces with link=:
!test port_gbps unit=Gb/s label="Switch port counters"
port_gbps R01/torb 200 link=SP1/spine1 net=fabric # sent towards spine1
port_gbps R01/torb 40 link=R01/s3 net=nicb dir=in # received from s3
- The value is what the device sent towards that neighbour;
dir=inmakes it what it received from it. net=picks the net when two cables of different nets join the same two elements (a server's data and mgmt to one ToR). Parallel cables of one net — four uplinks to the same spine — share the port's value in proportion to their capacity.- Both ends of a cable often report it, one as sent and one as received: those are two readings of the same traffic, and are averaged, not added.
- Samples over time reduce by the card's aggregation, as everywhere.
- A counter naming two elements with no cable between them is listed in the panel rather than drawn somewhere wrong.
The card offers draw on the cables: the counters, measured rather than routed, drawn in the same colours and widths as the model. With a routed metric loaded too, the panel's Show choice offers the model, the counters, and measured − model: each cable coloured by how far the measurement is from what the model said — red where it carried more, blue where less, as a share of the cable's capacity — with the biggest disagreements listed. That is where hashing put more than its share, or where traffic runs that the flow metric never mentioned.
examples/dual-plane-counters.tsv is the counters for the run in
examples/dual-plane-flows.tsv, as ECMP hashing really placed it plus two
flows the flows file does not know about: one R01 uplink carries 200 Gb/s
where the model has 80.
What-if: take something out
Select an element and press Take out in the inspector, or press the ✕ on a cable in the panel's lists, and it is out of service:
- A cable taken out carries nothing; an element taken out carries nothing and neither does anything inside it — take out a rack and its servers' flows go with it, listed as taken out.
- The model routes round what is left, and the Show choice switches to change since taking out: every cable coloured by how its load moved — red where it took more, blue where it lost it — with the biggest changes listed. The panel says how many flows lost their route and how many cables are now past full against before.
- What is out stays on the floor, marked: an element crossed through, a cable as a dashed red line between its two ends. Routes between picks go round it too.
- Each is a chip in the panel's what-if box; click one to put it back, or Put all back. Taken-out cables and elements are remembered by name, so editing the layout keeps the what-if.
The results file (.tsv)
One sample per line — test target value [key=value …] — separated by tabs,
commas or spaces. It is append-only by design: to add results from a new
test run, append lines. cat run47.tsv >> results.tsv is a fully supported
workflow.
Already have a table? A file of <stamp> host var1 var2 … loads as
it stands, no conversion and no layout file — see
A table you already have. This format stays
exactly as it is described here; that is a second reader, chosen per file.
Converting data from somewhere else into this format — by hand or by handing
the job to an AI — is written up in
docs/converting-data-to-overlays.md:
the target-resolution rules, what each !test key does, and the mistakes that
produce a file which loads cleanly and paints nothing.
temp_c DH1/A/R01/u05 61.2 run=nightly-01
iperf_gbps DH1/A/R01/u05 94.7
burnin DH1/A/R01/u05 PASS
- Targets are element paths, but any unique suffix works (
A/R01/u05, or just a hostname if node ids are hostnames). Matching is case-insensitive. A target that fits several elements goes to the first and the metric's card says so — N ambiguous targets, naming each one and where its reading went. Slot and role names repeat across racks (forty racks, fortyu01s), so write enough of the path to be unique when they do. - Values may be numbers or words (
PASS/WARN/FAILget traffic-light colors; other words get stable categorical colors). - How big can a results file be? There is no limit in the viewer. A file
is read as one string, so the hard wall is the browser's maximum string
length — about 512 MB, past which
file.text()throws. Measured on a laptop-class browser: 5 MB (142k samples) loads in under a second, 25 MB in ~3 s, 100 MB (2.8M samples) in ~12 s, 300 MB (8.5M samples) in ~34 s. Past that, split the run across several files and load them together — they merge exactly as appending would, and the panel groups them by file. - The same test + target may repeat freely (many runs, many instances). All samples are kept, and the UI reduces them with the aggregation you pick: mean, median, min, max, sum, count, first, last, harmonic mean, geometric mean, p95, p05, stdev, range. A metric of verdicts is not averaged and offers no choice: the worst wins, so a failure under a collapsed rack stays visible, and where nothing is worse than anything else the most common value does. The card says so in place of the picker.
- Optional
!testlines set display metadata:
!test temp_c unit=C higher=bad min=18 max=78 short=TMP label="Inlet temp"
Standardizing a metric
An overlay's colours normally span the smallest and largest value present, so the same colour means different things on different metrics, and a floor where everything is fine still paints something red. standardize on the overlay card colours by the z-score instead — how far each value sits from that metric's own mean, in standard deviations:
| Setting | Values shown | Colour from |
|---|---|---|
off |
the metric's own values | the min…max range (as before) |
colour by z-score |
the metric's own values | z, across ±σ |
values as z-score |
the z-score, in σ | z, across ±σ |
Use the first when the numbers matter and the colours should say "unusual for this metric"; the second when comparing metrics in different units side by side, where +2σ on temperature and +2σ on latency should look alike.
standardize all, above the cards, switches every metric at once. It
overrides the per-metric settings rather than rewriting them: each card goes
on showing (and still lets you edit) the setting it was given, greyed to say it
is not what is being drawn, and switching the override back off puts every
metric back on its own setting — including the ones you had deliberately left
raw. off there means "no override", not "force every metric raw". A metric
loaded while it is on is standardized too. The ramp's spread stays per metric,
since a shared one would have to overwrite what each card holds.
one scale — beside it, and on by default once anything is standardized —
is what makes two metrics actually comparable by eye. Spanning ±σ is not
enough on its own: a per-metric palette=, a per-metric spread, or a
higher=good inversion each paint the same z-score green on one metric and
red on the next. Ticked, every standardized metric is drawn from one palette,
one spread and no inversion, so the same z-score is the same colour
everywhere; each card still shows the palette and spread it would use on its
own, greyed. The default ramp is rdbu, diverging, because a z-score is
signed and the mean belongs in the middle. Untick it to go back to per-metric
colours.
The mean and spread are measured over one aggregated value per measured
element — the devices being compared, not the raw sample rows, which repeat
per run — so changing the aggregation re-measures them. The overlay card shows
what they came out as (mean 40.5C ± 8.73C over 480), the ramp ends default to
±3σ and are adjustable, and a metric with no spread at all reads 0σ everywhere
rather than dividing by zero.
A z-score says how unusual, never how much. +2σ is 17 °C on one metric and 0.4% on another, and sharing one scale across metrics is exactly what makes that worth knowing. Three things say it:
- The legend carries two rows:
≤ -3σ · mean · ≥ +3σover what those three points are worth here —14.31C · 40.50C · 66.68C. - The inspector and the tooltip show both numbers:
38.8Gb/s -4.82σ. The floor plan still shows whichever single number the mode asks for, because a slice is too narrow for two. - Hovering the units row says what one σ is worth on this metric.
Is σ a fair yardstick here? A shared scale assumes one σ means the same thing on every metric, which holds only while their distributions are a similar shape — so each card says how much of its metric sits beyond ±2σ against the 4.5% a normal distribution puts there. Close to it and the comparison is sound. It is highlighted only when the tail is further from normal than chance explains at that sample size: on twenty elements the expected tail is 0.9, so one element either side doubles the percentage, and a mark that fires on that is a mark you learn to ignore. A split population (σ set by the gap, nothing in the tails) and a fat tail (σ set by the bulk, too much outside it) are both caught once there are enough elements to tell.
The ends are where the colour stops, not where the data stops. Everything
past ±3σ paints the same colour, so an outlier at -4.8σ is indistinguishable
from one at -3.1σ — at exactly the extreme worth looking at. The card counts
what ran off (20 past the ends, worst -4.82σ), those elements carry a bar on
the edge they left by, and fit widens the range until nothing is clamped.
On a shared scale, fit reaches past the furthest point on any standardized
metric, since one range has to hold them all.
(higher=bad|good picks the green↔red ramp direction; palette= chooses any
ramp: viridis, magma, plasma, turbo, health, cool, ember, gray, rdbu;
agg= presets the aggregation; decimals= fixes formatting, 0 to 10 places;
finding=drops|lockups|errors|verdict says what trouble a metric reports, for
the findings
list, where its name does not — and finding=no that it reports none;
duplex=yes|no says whether a per-host rate adds both directions, where its
name does not say duplex or bidir; bytes= and reply_bytes= size a
packet rate's requests and replies, in bytes of payload, so it loads the
cables as a bit rate.
min=/max= must be numbers with min below max; anything else is
reported in the load report and ignored rather than quietly applied.)
Fields split on tabs, commas or runs of spaces — except inside quotes, so a
value that needs a space is written label="Inlet temp" (single quotes work
too) and the quotes are not part of it.
dcadd — appending made even easier
Optional helper; plain >> works too. tools/dcadd runs it from a clone
with nothing installed; pip install puts it on PATH as dcadd.
dcadd results.tsv temp_c DH1/A/R01/u05 61.2 run=nightly # one sample
my_test | dcadd results.tsv --stdin temp_c # target value per line
dcadd results.tsv --merge run1.tsv run2.tsv # concat other files
dcadd results.tsv --csv fio.csv --test iops --target host --value write_iops
dcadd results.tsv --meta temp_c unit=C higher=bad min=15 max=95 'label=Inlet temp'
A metadata or key=value field whose value carries a space or a comma is
quoted on the way out, so label=Inlet temp writes label="Inlet temp" and
reads back whole. The format has no escape, so the quote used is the one the
value does not contain; a value holding both is refused rather than written
as something that reads back differently.
JSON results
Anything that starts with { or [ is read as JSON instead — no flag, no
separate extension, and a # comment above it does not count. The overlays
that come out are identical to the text form's, so the two are
interchangeable and can be loaded together.
NDJSON — one object per line — is the form to generate. It keeps the
append-only property that makes cat run47.ndjson >> results.ndjson work,
which a top-level [ ... ] array would lose:
{"!test":"rtt_p50","unit":"us","higher":"bad","palette":"turbo"}
{"test":"rtt_p50","target":"wr12r06u15","value":184.2,"meta":{"peer":"wr12r06u16"}}
{"test":"burnin","target":"wr12r06u15","value":"PASS"}
An object with !test is the JSON spelling of a !test line. Everything
else needs test, target and value; meta is optional and shows up in
the inspector. A number stays a number and a string that reads as one is
treated as numeric, so a value quoted by whatever generated the file behaves
the same as an unquoted one.
A whole document also works, for tools that would rather emit one value:
{"tests": {"rtt_p50": {"unit": "us", "higher": "bad"}},
"samples": [{"test": "rtt_p50", "target": "wr12r06u15", "value": 184.2}]}
A bare [ ... ] array of samples is accepted too. Malformed lines are
reported with their line number rather than dropped, so a broken generator
shows up as a warning instead of an empty overlay.
mx export — matrix_orchestrator writes this format itself
matrix_orchestrator
needs no importer: it exports overlay samples directly, so there is nothing
in between to guess at its reports.
mx run --for 120 # measure
mx export --window 120 >> results.tsv # colour the floor plan with it
The numbers are the ones mx summarize prints, reduced by the run's own
rules — a blank cell is not measured and never zero, a layered run's rates
come from the host rows, latency is the worst peer's rather than a
percentile of percentiles. Those rules are not visible from outside a
reports/ directory, which is why this direction is an export and not an
import.
What arrives on the floor plan
One sample per host per run, targeted at the host's name. Every overlay carries its own units and palette direction, so the first render is already readable.
| Overlay | Unit | What it is |
|---|---|---|
mx_pps |
pps | requests this host sent |
mx_rep_pps |
pps | replies it got back |
mx_served_pps |
pps | requests that arrived here from its peers — the receiver-side truth a sender cannot see |
mx_request_gbps |
Gb/s | its own requests on the wire, Ethernet/IP/UDP framing included |
mx_egress_gbps |
Gb/s | everything it puts on the wire: those requests plus the replies it owes the hosts that call it |
mx_rel_median |
% | its packet rate against the fleet's own median — 100% is "normal for this fabric" |
mx_line_util |
% | egress against the NIC's line rate (only with mx export --nic-gbps) |
mx_loss |
% | round-trip loss |
mx_forward_loss |
% | of what it sent, the share that never arrived |
mx_return_loss |
% | the share that arrived but whose reply never came back |
mx_achieved |
% | delivered rate against the matrix's target |
mx_coverage |
% | layered runs only: the share of its peers measured so far |
mx_rtt_avg |
µs | mean latency over its flows |
mx_rtt_p50 mx_rtt_p99 mx_rtt_max |
µs | latency of its worst peer |
mx_cpu |
% | the whole box, averaged over its cores |
mx_cpu_core |
% | its busiest single core |
mx_agent_cpu |
% | the busiest mx worker, as a share of one core — near 100% means mx itself is the ceiling |
mx_peers |
flows this host sends | |
mx_workers |
agent worker processes, which is what makes mx_agent_cpu readable |
|
mx_intervals |
report intervals it contributed to the window | |
mx_state |
REPORTING; SILENT for a host that reported earlier in the run but nothing inside the window; NO-DATA for one in the matrix that never reported at all — the reading a missing report cannot give you |
mx export --peers adds a per-flow overlay — mx_peer_pps, mx_peer_loss,
mx_peer_rtt_p99 — one sample per flow, each tagged peer= (and layer=
on a layered run). They live under their own names so a mean over the
per-host overlays can never quietly include per-flow rows, and they ask for
max by default, which reads as the worst peer of this host.
The matrix on the cables, in packets and in bits. mx export --peers
writes mx_peer_pps, one sample per flow tagged peer= — the matrix itself.
Tick load the cables on it and every flow is routed, src to peer. A
packet rate alone cannot be set against a cable's gbps=, so the panel
asks for its packets' size: packets of [256] B, replies of [4096] B,
filled in by itself from the # run shape: 256 bytes out -> 4096 bytes back
line mx export writes at the top of the file. With a size:
- every request is routed out at its payload plus 66 bytes of framing
(Ethernet, IP, UDP, preamble and gap — mx's own
mx_request_gbpsis exactly that), and its reply is routed back at the reply's size, so a matrix of small requests and big replies loads the return path hardest; - the load is a bit rate: cables read as a share of their
gbps=, contention shares out the full ones, and switches say how full they are; - the headline gives both:
168.7Gb/s delivered between hosts · 132 flows, then4.7 Mpps requests · 4.7 Mpps replies — 256 B out, 4096 B back.
Change either size in the panel to ask what a different payload would do;
empty the reply size to route the requests alone. bytes= and
reply_bytes= on a !test line size any packet-rate metric, mx's or not.
Per-host totals (mx_pps) work the same way, with each host's requests
spread as the estimate spreads them and every reply sent back to whoever
asked — up to about 380 hosts, the most that can be paired here; past that
the panel says the replies are left out, and mx_peer_pps is the one to
load.
Rates you can read without knowing the hardware. mx_rel_median is the
one to open first on an unfamiliar fleet: it puts every host against the
run's own median on a diverging ramp, so 100% is "normal here" and a slow
rack stands out whatever the absolute numbers are. It aggregates by
median deliberately — every host in a mesh talks to the sick host, so
min would redden the whole floor and hide it, while a host that is itself
slow has all of its flows slow. That is "I am slow" against "I have a slow
peer". (iperf_rel_median below is the same idea for throughput, and reads
the same way.)
What a relative overlay cannot see is a floor that is uniformly slow —
every host reads 100% of a median that is itself wrong. mx export --nic-gbps 25 fixes the scale to the hardware: it adds mx_line_util and
pins the throughput overlays absolutely, so half speed looks like half
speed.
Every overlay arrives ready to read. Units, palette direction, decimal
places, a 0-100 range pinned where the value really is a percentage of
something (auto-fitting makes a 30% CPU peak look alarming for no reason
but being the highest), and the aggregation that answers each overlay's own
question when a rack or room is collapsed: max for the worst peer's
latency and the busiest agent worker, min for coverage and intervals,
median for the two that diverge around 100%. All of it is overridable in
the overlay panel.
An overlay is present only when its number was measured. A host whose
receive side nobody reported gets no mx_served_pps and no
mx_egress_gbps — rather than a zero, which the viewer would average into
the rack and room above it — while mx_request_gbps is known from the
host's own rows and is always there. The loss split needs every peer's
receive rows, mx_achieved needs a paced matrix, and mx_coverage only
means anything on a layered run. Sparse overlays are normal here and are
the format working as intended.
Reading mx_rtt_p99. mx's latency histogram holds four buckets per
octave and a percentile reports its bucket's upper edge, so mx_rtt_p99
rounds up — by up to ~25%, and it can read a little above mx_rtt_max,
which is exact. That is a property of mx's report, not of the export;
mx summarize prints the same pair.
Making the names line up
Targets are the mx host names. The viewer resolves a bare name, a full
path, or any unique tail of one, so hosts named wr12r06u15 in
servers.txt land on the right node with nothing else to do. A server list
of bare IPs will not resolve — the overlay panel shows them as unmatched
targets — so either name the hosts in servers.txt (name=addr) or map
them on the way out:
mx export --names hosts.map --target-prefix DH1/A/ -o results.tsv
The rest of the switches
--raw adds one sample per host per report interval for the columns a
host row carries, so the viewer can show min/max/p95 over a run rather than
the window's mean; the overlays derived from more than one row are still
written once per host. --json writes the same samples as NDJSON, one
object per line. --window 0 reduces the whole report history instead of
the last 60 s. --run LABEL tags every sample run=LABEL, and
--test-prefix renames the overlays if mx_ would collide with something
else in the same file.
export-overlay — iperf_orchestrator writes this format itself
iperf_orchestrator
2.2+ needs no importer either. A mesh test answers "which host is slow"; this
floor plan answers "which rack", and the orchestrator exports the samples that
join them:
iperf-orchestrator run # measure
iperf-orchestrator export-overlay # -> results/latest/iperf_overlay.tsv
iperf-orchestrator run --overlay # or as part of the pipeline
Throughput
| Overlay | Per | What it is |
|---|---|---|
iperf_mbps_out / iperf_mbps_in |
direction | one test's rate, credited to the sender and to the receiver |
iperf_mbps_duplex |
host | what that host carried at once, both directions |
iperf_gbytes |
host | total data carried across the run |
iperf_line_util |
direction | rate as a % of the NIC's line rate (only with --overlay-line-rate) |
iperf_achieved |
direction | rate as a % of the -b target the run asked for |
Relative — the overlays that say something a rate cannot
| Overlay | Per | What it is |
|---|---|---|
iperf_rel_median |
direction | that direction against the run's own median, in % |
iperf_asymmetry |
pair | the gap between a pair's two directions, as a % of the faster one |
iperf_rel_median is the one to open first: 100% is "normal for this fabric",
45% is half speed, and you need not know what good looks like on this hardware.
It arrives with a diverging palette pinned at 0–200% and aggregates by
median, which matters more than it sounds — on a full mesh every host talks
to the sick host, so every host's worst direction is the one to it, and a
min aggregation reddens the whole floor while hiding the host that is
actually slow. The median separates "I am slow" (all my directions are) from "I
have a slow peer" (one is). Switch it to min when you do want the worst link
anywhere. iperf_asymmetry is the shape a duplex mismatch, a one-way policer
or a congested return path makes, credited to both ends and aggregating by
max.
Reliability
| Overlay | Per | What it is |
|---|---|---|
iperf_status |
direction | OK; FAIL with the status, error text and log file to open; or NO-DATA |
iperf_fail_kind |
direction | the failures only, valued by why (NO_SUMMARY, DIRECTION_MISSING, …) |
iperf_state |
host | the roll call: TESTED, or NO-DATA for a host that never answered |
iperf_ok_pct |
host | success rate, the worse of the host's send and receive sides |
iperf_peers |
host | how many distinct peers it exchanged data with |
iperf_coverage |
host | those peers as a % of the ones it was planned to reach |
iperf_tests |
host | how many directed tests it took part in — sample count is confidence |
iperf_state is the roll call, and NO-DATA is a host that was in the run's
server list and produced no row at all — SSH refused, iperf2 missing, box down.
Without it that host has nothing to paint and reads here as "not part of this
test", which is the one reading that is certainly wrong; it comes with 0% on
iperf_ok_pct so it lands on a numeric overlay too. The roll call is a
per-host overlay kept apart from the per-direction iperf_status for the same
reason mx export keeps mx_state apart from its per-peer overlays: two
granularities in one overlay would reduce into each other. iperf_ok_pct reports the worse side
rather than the pooled rate because a host that receives fine and cannot send
anything is broken, not half-well — the sent=/recv= metadata names which
side failed.
The host itself
| Overlay | Per | What it is |
|---|---|---|
iperf_cpu_peak, _mean, _softirq, _sys, _user, _idle_floor |
host | from cpu_summary.csv; softirq and idle floor are per-core, and say which core |
iperf_bind_iface |
direction | which NIC the traffic rode, when the run used --bind |
Every overlay arrives with its unit, ramp direction, short name and — for the
percentages — its real 0–100 range, because an auto-fitted scale makes a 30%
CPU peak look alarming purely for being the highest number present. Sample
metadata carries the peer, the test's timestamp, and on a failure the error
text and log file; the header records the run id, its mode (a parallel
run is the whole fleet under load at once, sequential-pair is one link's
uncontended maximum — the numbers are not comparable) and the shape every
sample shared.
Two things are deliberately never invented. A direction that produced no number
is never exported as 0 Mb/s — zero is a measurement, and averaging it in
makes a broken link read as merely slow — so it leaves as a FAIL verdict and
pulls its host's coverage down instead. And iperf_mbps_duplex is not a plain
sum: parallel and sequential runs fire a round's flows together so those rates
genuinely add, while rolling mode probes one pair over and over and adding
those reports more than the NIC can carry. Rows are clustered by overlapping
test window, summed inside a cluster and averaged across clusters; when the CSV
carries no test windows the overlay is left out rather than guessed at.
This export and mx export are built to the same rules, so an mx run
and an iperf run can share one results file and read as one system:
iperf_achieved is mx_achieved against a target rate, iperf_coverage is
mx_coverage, iperf_tests is mx_intervals, iperf_state is mx_state,
both keep per-host and per-peer values in separate overlays, and both write
numbers to four significant digits in fixed notation — %g renders a host's
aggregate as 1.163e+06, which is correct and unreadable in a file people
grep. --overlay-test-prefix (like mx's --test-prefix) namespaces every
overlay, so two runs exported with different prefixes load side by side for
comparison instead of averaging into each other.
One measured row becomes one sample, so the aggregation menu here does the
reducing (mean across peers, max the best, min the worst, stdev how
steady). Every sample carries run=<run-id>, so a nightly
export-overlay --overlay-append accumulates history in one file that first
and last read as then-and-now. --overlay-map and --overlay-prefix rename
hosts onto whatever the layout calls those nodes, and --overlay-format ndjson
writes NDJSON instead.
reckon — binnacle grades a run against the hardware
The overlays above are relative: mx_achieved to the target the run asked
for, mx_rel_median and iperf_rel_median to the fleet's own median. A
fleet running at half its NICs sits at 100% of both. The absolute reading
needs the hardware, and the hardware can live in the layout as ordinary
attributes:
row A +compute nic_gbps=25
rack r[01..04] u=42 uplinks=4 uplink_gbps=100
binnacle's
reckon reads
that, works out what every flow of an mx or iperf run could have had — its
fair share of the NICs and rack uplinks it crosses, or its target if lower —
and writes this format itself:
reckon floor.dc --mx reports/ --overlay reckon.tsv
dcviz serve --layout floor.dc --results reckon.tsv
| Overlay | Per | What it is |
|---|---|---|
reckon_efficiency |
host | its flows' achieved vs expected, median, on a ramp pinned at 0–200% |
reckon_expected / reckon_achieved |
host | pps for an mx run, Mb/s for iperf |
reckon_limit |
host | what the hardware limits it to: TARGET, NIC, UPLINK, … |
reckon_verdict |
host | OK, WARN, FAIL, or NO-DATA for a host that never reported |
reckon_nic_gbps |
host | the speed it was graded against, source= layout, measured or flag |
reckon_peer_efficiency |
flow | one flow, with peer=, so it draws as a measured flow |
reckon_change / reckon_peer_change |
host / flow | with --baseline: efficiency now minus then, in points, on a ramp from −50 to +50 |
reckon_efficiency is the one to open beside mx_rel_median: the median
overlay says which hosts are unlike the rest, and this one says whether the
rest are where the hardware puts them. A host above 100% is a finding too —
it means the layout's hardware is wrong. Every assumption the model made is
written into the file's # header.
reckon --baseline takes an earlier run's reckoning — its --json, or the
overlay you kept for the viewer — and adds reckon_change: how far each host
moved since then. Efficiency compares across runs where raw rates cannot,
because each run is graded against its own expectation, so the change layer
shows a host slipping before any absolute layer turns red.
reckon floor.dc --mx reports/ --run wed --baseline tue.tsv --overlay wed.tsv
dcimport — netmesh output, directly
dcadd --csv already imports any CSV with a target column and a value
column. dcimport is for netmesh, whose output does not have that shape,
because what it measures is a pair — (src, dst) — while the viewer
paints elements.
dcimport results.tsv --tidy reports/ # netmesh
tools/dcimport results.tsv --tidy reports/ # ... or straight from a clone
| Source | What arrives |
|---|---|
netmesh reports/ |
rtt_p50 rtt_p99 jitter loss path_mtu per peer, agent_cpu per host |
Nothing is averaged on the way in. One measured row becomes one sample,
so the viewer's own aggregation menu does the reducing: mean reads as
"across peers and intervals", max as "the worst peer", last as "right
now". --reduce collapses to one sample per host per metric (median) for
meshes big enough that the raw row count matters.
Only while every agent was running. netmesh start and stop reach
the hosts over ssh a batch at a time, so a run's first and last seconds are
not a mesh: a probe to a peer not yet up, or already gone, is never
answered and reads as loss on a network that lost nothing. dcimport
imports the rows written while every agent was running, read off the
reports the way netmesh's own summary reads them — from the first row of
the last host to start to the last row of the first host to stop — and
says what it left out, on stderr and as a # line in the file. A host
that starts or stops 15 s (three report intervals, if longer) or more
apart from the rest came late or died, and its loss stays in.
--whole-run imports every row.
A blank cell means "not measured" and is skipped rather than read as zero — averaging a blank as 0 is the one mistake that quietly makes every one of these numbers look better than it is.
iperf_orchestrator is not imported here either, for the same reason as mx:
it writes this format itself — see export-overlay above.
tests/fixtures/iperf-overlay.tsv is that export, kept as the contract the
suite checks.
Targets are whatever the tools called the hosts, so generate the server
list from the layout and the names already match. binnacle's manifest
reads this project's .dc format for exactly that:
manifest floor.dc 'room[1]' | netmesh gen --servers -
netmesh run --for 60 && dcimport results.tsv --tidy reports/
A table you already have
Most monitoring already writes a wide table: a timestamp, a host, and a column
per thing measured. The viewer reads that directly — no conversion step. Load
your .dc file beside it as usual, or, if the layout is not written yet, ask
for one to be built from the names in the data.
Timestamp host rtt (us) loss % cpu %
2026-09-16T12:00:00 wr01r01u05 184.2 0.01 37
2026-09-16T12:00:00 wr01r01u06 191.0 0.00 41
2026-09-16T12:00:10 wr01r01u05 186.9 0.00 39
Columns are separated by tabs where the row has them — that is what lets a
heading be rtt (us) rather than three columns — and by runs of spaces
where it has none, so a table typed by hand or written by awk reads too. Try
it, with a floor plan read out of the hostnames:
http://localhost:8000/?build=1&results=examples/live/room-wr01.tsv
-
The first column is the stamp, and it is only the stamp. A date, a time, an epoch, or a plain number counting the passes —
1,2,3is as much a first column as an ISO date is. It is kept on every sample and it never names anything: a file like1 host_1 5 2 host_1 6 3 host_1 3is one metric, on one host, with three samples — not three metrics called 1, 2 and 3.
-
The name does not matter. Which reader a file gets is worked out from what is inside it, so a table written to
today.log,metrics.dator a file with no extension at all is the same table it would be inrun.tsv. The Files panel lists the usual data extensions by default and everything else behind all files — but it will open anything that could be text, and only refuses what cannot be (.png,.gz,.so). Going the other way, a floor plan calledfloorthat opens withdc DC1is read as a layout. -
The header is optional. Without one the columns are called
A,B,C… — so give the file a header line, or a!testline per column, when the metric deserves a name. A header may be commented out (# Timestamp<TAB>host<TAB>…), so the file stays a table toawk, and a second header part-way down the file is read as a new one — concatenated days each bring their own. -
A heading carries its unit when it is bracketed (
rtt (us),loss [%]) or ends in%. The metric keeps the whole heading as its label. -
A blank cell is "not measured", never zero. Averaging a gap as zero is the one mistake that makes every number on a floor look better than it is.
-
A value may be a word (
PASS/FAIL), exactly as in the results format. -
A host may be a flow:
wr01r01u05 -> wr01r02u09measures the path between two machines rather than a property of one. The sample belongs to the host it started at, with the far end as itspeer=, which is what the viewer already draws as a measured flow and filters withpeer=.->,=>and→all work, anda -> b -> ckeeps its hops. -
A column can still be given display metadata. A table may carry the results format's own
!testlines, checked exactly as they are there, so a column gets a unit, a palette, a range or a short name without a second syntax for it — and without stopping the file being read as a table:!test rtt unit=us higher=bad min=0 max=500 short=RTT !test loss unit=% higher=bad slug=loss_pct Timestamp host rtt loss 2026-09-16T12:00:00 wr01r01u05 184 0.01 -
Several files combine. Tables in the same folder are one dashboard: one file per host, or per metric, or per hour — a column of the same name in two of them is one metric carrying the samples of both. (Every other format keeps a file's metrics to itself, deliberately; this is the format where the opposite is what you meant. Two folders stay two dashboards.)
Building a floor plan from the data
The .dc file is what says where the machines are, and normally that is the
file you load. But the names in a results file often carry their position
anyway, and sometimes the layout is not written yet — so Build from data,
in the Structure panel, reads a floor plan out of them. It is a button and
never anything else: nothing here invents a floor plan because a file arrived.
It works on any loaded results, not only a table: a results target is already a path through a floor plan somebody wrote, rows and all.
DH1/A/R01/u05 -> room DH1, row A, rack R01, machine u05
wr12r06u15 -> room wr12, rack r06, machine u15
dc1-hall2-r03-u05 -> room dc1, row hall2, rack r03, machine u05
rack01-server05 -> rack rack01, machine server05
mailserver -> one rack of hosts
web01.dc.acme.com -> the domain the hosts share is dropped, not read as racks
The last part of a name is the machine, the part before it its rack, then its
row, and the rest its room. -, _, /, : and . split a name; a name
with no separator is split at its letter-and-digit runs, which is what makes
wr12r06u15 three levels. Racks that name no row are dealt into rows of
twelve, so a flat fleet is a floor plan rather than one very long line.
- It is an ordinary layout:
dc DATA … +generated, which Edit layout opens and Download .dc saves to correct by hand. - While it is the floor plan it follows the data — a host that starts reporting gets a place without being asked for again — and Clear takes it away, leaving the data loaded.
- Loading a
.dcfile replaces it, every overlay still bound to the same hosts. Going the other way, replacing a floor plan you loaded, takes two clicks: that file is what the data is meant to be read against. - When a layout is loaded, the panel also says how many targets in the data are not on it — the number that otherwise only reads as "nothing was measured".
&build=1in the URL does the same thing on load, which is what makes a link to a folder of tables a dashboard somebody else can open.
dcprobe — how every host is doing
dcprobe reaches every host over ssh and writes how each is doing as
results the viewer paints. One shell script goes to each host; nothing is
installed there and nothing is left behind. tools/dcprobe runs it from a
clone; pip install puts it on PATH. Host status, in the right
panel, writes its hosts.txt and commands for the layout's hosts, and
reads a loaded scan back into the hosts worth a look.
dcprobe preflight --servers hosts.txt -o preflight.tsv --tools iperf --ports 5300,5310,5001 --bind data
dcprobe scan --servers hosts.txt -o host_status.tsv --nic data --mtu 9000 [--why-slow]
dcprobe watch --servers hosts.txt -o host_status.tsv --every 30 --history host_history.tsv
dcprobe diff host_status-before.tsv host_status.tsv -o changes.tsv
Save probe-hosts.tar puts it all in one archive, as Run it for real
does for a load: hosts.txt, the layout as model.dc, and
probe-hosts.sh, executable, in a folder of their own. The layout goes
with the results: every command passes --model model.dc, which writes
it, whole, into host_status.tsv as the #~ lines a run file carries
(see The run carries its layout), so
loading the scan brings back the floor plan its hosts were probed in —
asking first when another is in the viewer. The script says so and stops if dcprobe is not on PATH;
otherwise it runs preflight and then one scan (the watch and diff lines are
there commented, to swap in). Save probe-hosts.sh saves the script
alone.
tar -xf probe-hosts.tar && ./probe-hosts/probe-hosts.sh
hosts.txt is the grammar mx and netmesh read — name[=addr] a line —
with what the host should be beside it: r1h1=10.1.1.11 gbps=25, and
optionally mtu= and nic=. Host status writes it from the layout: each
host by the layout's name, at its addr=, with the speed of its fastest
NIC. The hosts are the load generator's: those its hosts filter
picks (every host when it is empty), less anything taken out — the hosts a
load would run on, so a probe looks at the machines the run will use. The
pane, and the script's comments, say which filter picked them and how
many there are; a filter that picks none says so instead.
| Verb | What it does |
|---|---|
scan |
one pass: everything below, for every host |
watch |
a pass every --every seconds; the file is rewritten each pass, so it always says how things are now and Live follows it; --history keeps every pass |
preflight |
can every host run the load? ssh, python3 (mx and netmesh copy their agents over), the tools the hosts need (iperf), --bind matching an interface, the run's UDP ports free, the clock against this machine's, open files allowed |
--model FILE |
(scan, watch, preflight) the layout, written whole into the results, named by hosts.txt's # layout: line |
diff |
what changed between two scans — kernel, OS, CPU, memory, NIC driver and firmware, NIC speed, MTU and link — host by host, and -o as probe_changed |
Hosts are probed in parallel, 256 at once (--jobs): a scan of 1,000
hosts is four waves of about two seconds each — the ssh connection and the
one-second sample. Each host at once is an ssh child holding a few open
files here, so dcprobe raises its own open-file limit to fit, as far as
the system lets it; where ulimit -n will not stretch that far it probes
as many at once as fit, says so, and still probes every host.
What a scan measures, over a one-second sample (--sample):
| Metric | What it is |
|---|---|
probe_reach, probe_why |
answered the probe, PASS or FAIL, and ssh's reason when not — a host that does not answer is a sample, never a gap |
host_load1, host_load_per_core |
load, and load a core |
host_cpu_pct, host_iowait_pct, host_steal_pct |
CPU busy, waiting on I/O, stolen by the hypervisor |
host_mem_pct, host_swap_pct, host_disk_pct |
memory in use (from MemAvailable), swap in use, the fullest real disk |
host_softnet_drop_ps, host_softnet_squeeze_ps |
the kernel's receive backlog: packets dropped, and passes out of budget, a second |
nic_link, nic_speed_gbps, nic_mtu |
each physical NIC (no lo, veth or bridges) |
nic_rx_gbps, nic_tx_gbps |
what it moved |
nic_rx_drop_ps, nic_tx_drop_ps, nic_err_ps, nic_ring_drop_ps |
what it dropped and errored, and its ring and buffer drops (ethtool -S), a second |
nic_check, nic_why |
the NIC as the layout says: FAIL when none is up or none is at the layout's gbps=, WARN for one slower or at another MTU than --mtu |
host_kernel_errors, host_kernel_last |
kernel errors logged in the last hour (--dmesg-since; watch: since the last pass), and the last of them |
host_top |
the busiest processes |
inv_kernel, inv_os, inv_cpu, inv_cores, inv_mem_gb, inv_nic_driver, inv_nic_fw |
what the host is |
whyslow_verdict, whyslow_top |
with --why-slow: binnacle's why-slow, piped to each host over the same ssh, its worst finding as PASS, WARN or FAIL and its first serious one |
Pass --nic the run's --bind (Host status does): only the NICs it
matches are judged, so the run's NIC is held to the layout's speed and the
run's MTU, and a port nobody cabled is not a WARN. Without it, a NIC that
is down is listed but not held against the host.
Drops, errors and verdicts go to Findings
on their own (finding= on their !test lines). Host status lists, from
a loaded scan, who did not answer, who is not ready to run, whose NIC is not
as the layout says, what why-slow found, kernel errors, and who is busy
(CPU 90%+, load a core 1.5+, iowait 20%+), nearly out of memory (90%+),
swapping (25%+ of swap) or nearly full (a disk at 90%+) — each host a chip
with what it said, a click away. Folded, its title says how many answered
and how many need a look.
Live: reload on a timer
The Live panel (left, under Files) turns loaded files into a dashboard:
-
reload every N s — re-reads every loaded results file on a timer. The same reader and the same rules as opening one by hand, so a file being appended to replaces what it brought last time instead of counting its rows twice. A pass still running when the next is due is simply the pass that is running: they never overlap.
-
last N records per file — reads only the end of each file, the way
tail -ndoes, so a log that grows all day stays the size of what is on screen. Headers, comments and!testlines are kept whatever their age — losing them would take the column names and units with them. Empty means the whole file. -
replace what it brought / add the rows since last time — what a pass does with what it reads.
Replace reads each file again from scratch: the file is the truth, which is what a file rewritten in place needs, and what stops a growing log from counting its rows twice.
Append takes only the records that were not in the last read and adds them to what is loaded — so the view accumulates past the tail, which is the point: read the last 500 rows every ten seconds, keep the whole hour. Where the new read starts is found by looking for the end of the last read inside it (the last few records, matched as a block, since one row is not an identity — a table on a timer writes the same line twice the moment two passes measure the same values). A file that cannot be lined up — rewritten from the top, or grown by more than the tail being read — is replaced instead and the report says so, because taking rows that cannot be lined up would count some of them twice. Past 400,000 accumulated samples on a metric the oldest are dropped as new ones arrive, making it a rolling window.
-
Reload now — one pass, on demand.
-
Load all, in the Files panel, loads every results file in the open folder matching the name filter (
*.tsvby default) and then follows that folder: on each pass it is listed again, so a file written while the dashboard is up joins it, and one that disappears takes its samples with it. The × beside the watched folder in Live stops that; nothing already loaded is unloaded.
Files opened through the folder browser reload best — the browser hands over a handle that reads the file as it is now. Files chosen through the plain file input are re-read by path, which works until the file is replaced rather than appended to; a file that cannot be re-read is named in the panel rather than quietly left stale.
The viewer
-
Every titled part of the side panels folds: a section (Files, Structure, Overlays, Networks, Load generator, Cable load, Findings, Measured against the model, Inspector), a list inside one (Busiest cables, Run it for real, Where to look …) and each part of the inspector (Readings, Cable load, Findings, Measured flows). A click on a title folds it to that title, and what is folded stays folded across redraws and reloads. A folded section still says in a few words how things stand — the cables' total, how many servers have findings, how many flows are off the model — and the buttons in its title (Run/Stop, Off, mark on the floor) work without unfolding it. A section with nothing to show is not shown, and a section's title stays at the top of the panel while its contents scroll under it. Alt+click a title to open that one alone, every other beside it folded; Alt+click it again to open them all. Collapse all, at the top of each side panel, folds every section of that panel to its title; once they all are, it reads Open all and opens them again. The parts inside a section keep what they were. The Inspector is the right panel's first section — what was clicked is what is being looked at — and its own parts sit a step below the panel's sections.
-
Light or dark: the Light button in the top bar turns the panels and the floor plan light, for a bright room or a printout, and Dark turns them back. The choice is kept for next time.
-
What connects these? Shift-click (or Ctrl/⌘-click) elements on the floor plan or in the tree to pick them; each pick gets a numbered tag. With two or more picked, only the cables between them are drawn: for each pair, every network that joins the two on its own, by its shortest routes — and every equally short alternative, so two servers in different rows show all the spines they could cross. The inspector lists the picks and, per pair, each network and how many hops it takes (
u05 ↔ u07 · data: 4 hops). A route crosses from one network to another only when no single network joins the two, as in a layout that gives each tier its own net; it never borrows a server's mgmt cable to reach the data fabric. A pick can be a container — two racks show the uplinks between them. Untick a network to route without it; Shift-click a pick again, or its chip, to drop it; Clear or Esc forgets them all. With cable load on, the picks box can also count only the flows between the picks, and send a test flow from the first to the second — see Between picks. -
Hover a cable to see it: its ends, what it can carry, what it carries and the flows over it — see Hovering a cable. With nothing to load, the load generator makes a load up.
-
Files — the panel's Open folder… keeps a directory open beside the canvas, instead of a dialog that shows one and forgets it. Layouts and results are listed with their sizes (worth seeing before clicking a 300 MB file); a click loads one — a
.dcas the floor plan, anything else as overlays added to what is already there — and a ✓ marks what is in. The extension only decides what is listed, never what can be opened: tick all files and atoday.log, ametrics.dator a file with no extension opens exactly the same way, because the format is read from the file. Subdirectories open, the breadcrumb goes back up,⟳re-reads a folder that has gained a file, and the name filter takes globs (mx*.tsv). Dropping a folder on the window opens it here. In Chromium the folder and the path inside it come back after a reload, behind one click for the permission the browser will not keep; elsewhere it falls back to a directory input, which lists the same tree but has to be chosen again each session. Nothing is read until it is clicked, and clicking a results file that is already loaded re-reads it — replacing its overlays rather than counting every sample twice, since the format is append-only. A file is known by its path below the open folder, somonday/results.tsvandtuesday/results.tsvare two files and stay two sets of overlays; where a load genuinely cannot tell two files apart (two named the same, dropped together, with no path between them) they are numbered and the report says so. -
Restart — empties the viewer: no floor plan, no overlays, no filter, no selection, and a blank canvas. Remove all in the Overlays panel only ever cleared the overlays and left the floor plan drawn; this clears the lot. What survives is the workspace rather than its contents — panel widths and folds, and the folder held open in Files, so the next thing to load is one click away.
-
The load report — every load says what each file did, in a chip at the right of the top bar (
✓ 3, or⚠ 1when something did not go as asked). Click it for the report; a load with a problem opens it by itself. It sits in the top bar rather than a panel because a panel collapses, and the one thing saying a file did not arrive must not be collapsible. It names the quiet outcomes in particular: a file with no data lines in it, a file whose lines are not intest target valueform, two layouts at once (a viewer holds one floor plan — it says which won and which were ignored), and a file re-read over itself (it replaces what it brought before, rather than counting its samples twice). Warnings inside a file name that file, soline 5:is never ambiguous between two. -
Panels — either side panel collapses to a slim rail (the
‹/›in its heading, and the rail brings it back) and resizes by dragging the edge beside the canvas; double-click that edge to reset a width. Every section inside a panel — Files, Structure, About, Overlays, Networks, Inspector — folds to its heading when the heading is clicked. Widths, collapsed panels and folded sections are all remembered between visits. Sections are wired from the markup, keyed by their heading, so one added later folds with no extra code. -
Zoom / pan — wheel and drag (level-of-detail keeps hundreds of thousands of elements smooth).
ffits selection,0fits all. Labels grow with the zoom — each is sized from its element's on-screen height — so zooming in to read something makes it bigger, up to a ceiling. -
Reloading cannot empty it by accident. A reload loses everything loaded, built, edited or set, so once you have done anything since the page opened, Ctrl+R (⌘R, F5) does nothing and a note says so, and the browser's reload button or closing the tab asks first. A page just as its URL loaded it, untouched, reloads as ever: the reload brings it all back. Restart empties the viewer too, so it asks first while there is a floor plan or a metric to lose, and says which.
-
Expand / collapse — double-click any container, use the tree, or the Collapse racks/rows/rooms buttons. Collapsed containers show their aggregate results computed from all raw samples inside them.
-
Filter — the top bar matches anything: bare words search ids, names, tags and attributes, and a bare glob (
*serv*,r76?) searches exactly the same fields, plus the full path soDH1/A/*is a query;+gputags (globs too:+stor*,+gp?);kind:rack;model=r76*(?matches exactly one character);net:storage; result queries liketemp_c>70,burnin=FAIL,has:iperf_gbps(a test name can be loaded from more than one file, and the element matches when any of them reads over the threshold);!negates,|ors, space ands. Matches keep their ancestors visible; "hide non-matching" prunes everything else, otherwise non-matches are dimmed. Sections, and boolean logic.in:A01is everything inside A01 (and A01 itself) — by id or name, or by path,in:H1/A01for the A01 in hall H1. Values take ranges the way a layout writes them:in:A[01..04],in:R[01..08,12],name=u[01..10]— and so does a bare name:ht11r01u[7..11],ht11r[01,05,07]u[7,9], each bracket a range (7..11), a list (1,5,9) or both (1..4,7), spaced as you like ([07, 09]). A number in a range matches however it is padded, sou[7..11]findsu07tou11; a range matches whole names, sou[5..6]is notu50. Terms combine withand,orandnot(any case) and parentheses:notbinds tightest, thenand(or a plain space), thenor, sorole=server (in:A[01..04] or in:B07) not +gpu role=server in:H1/A01 or role=server in:H2/A[05..08]
each pick out servers in several sections at once. A
|still joins only the two terms beside it (+gpu a | bis+gpuand either), and a quoted"or"is a word to search for, not an operator. The load generator's hosts box reads the same language, and says how many hosts it takes as you type. A metric with a space or a slash in its name has a second name that the box can read:iperf Mb/s (out)answers toiperf_mb_s_out, and every overlay card prints its own under filter as — click it to filter by that metric, or type it into a comparison (iperf_mb_s_out>18000). The original name still works where it can be typed at all;slug=on a!testline sets a different one. -
Overlays — check any number of tests. With N enabled, every element is split into N side-by-side slices, each colored by its test with the test's short name and value printed on it (2, 3, 4, 5… all work). Per overlay you pick the aggregation, palette, and value range live. Hide all unticks every overlay without unloading it, and Remove all drops them and their samples — both at the top of the panel, since a results file can carry twenty-odd overlays. standardize all there standardizes every metric at once, overriding each card's own setting without overwriting it (see Standardizing a metric). Overlays are grouped by the file they came from, and each group's header collapses it, so an mx export and an iperf export loaded together stay tellable apart; the header counts the group's overlays and how many are shown, and its × removes that file's overlays alone. sort A–Z orders the metrics alphabetically within each file; unticked they keep the order the file wrote them in, which the exports choose deliberately (mx and iperf write theirs in reading order). Files never combine: every overlay belongs to exactly one file, so two files that both carry a test called
temp_care two overlays with two sets of samples, two domains and two cards — one under each file's group. To accumulate a metric over time, concatenate the runs into one file, which is what the append-only format is for. Loading the same file again re-reads it, replacing what it brought before rather than counting its samples twice. -
Networks — toggle each named fabric, or Show all / Hide all them at once; opacity slider for dense views.
-
Measured flows —
mx export --peersandiperf_orchestratorwrite one sample per flow, taggedpeer=. Those overlays offer draw measured flows, which paints each measured pair as a dashed curve coloured by its own value, and the inspector lists a host's flows per peer (worst first) rather than only the aggregate that hides them.peer=wr01r01u02in the filter bar selects the hosts that measured a flow to that host, globs included. A flow is not a cable: mx and iperf measure end to end, so a flow between two servers in one rack really crossed server → ToR → server, and neither tool can say how the traffic divided across the hops. load the cables models it — routes every flow over the fabric and draws each cable by what lands on it (see Cable load); measured per-cable numbers still need switch interface counters. -
Cable load and capacity — with a metric loaded onto the cables, the Networks panel holds what the cables show (model, counters, the two compared, or the change since taking something out), the colour key, the routing (hashed per connection with Re-roll and the connections a flow, polarized, adaptive, by capacity, or packet spraying), the spread for an estimate, the what-if, and the busiest cables or biggest changes; width by capacity draws each cable by its
gbps=instead. See Capacity and Cable load. -
Inspector — click an element for its tags (click a tag to filter by it), attributes, U-slot, link counts, and all its readings. Cables hang off the leaf devices, so a rack, row or room reports links below instead: every cable in its subtree per fabric, split into the ones that stay inside it and the ones that leave. Show only this element's cables then hides every cable that neither starts nor ends inside the selection, at any level — one server's two cables, or everything entering and leaving a room. The elements at either end of the surviving cables are marked with a faint wash and outline (a dot where they are too small to outline), so the far end of a hairline is visible without tracing it.
Tests
node tests/run.mjs # 1,003 assertions over the modules
node tests/browser.mjs # 109 more, driving the page in Chromium
Both skip what they cannot run — tests/run.mjs needs python3 for the
dcadd/dcimport sections, tests/browser.mjs needs Playwright — so either
runs anywhere. --strict turns every skip into a failure, which is what CI
passes: a green run that quietly tested nothing is indistinguishable from a
green run that tested everything, and that is the one result a test suite must
never give.
GitHub Actions runs both on every push and pull request, plus a third job that builds the Python package, installs the wheel into a clean environment and fetches a page from the server it provides — the wheel's layout is not the checkout's, so nothing else here would notice a package that installs and then serves nothing. Three jobs, so a red mark says which layer broke.
Repository layout
index.html, css/, js/ the viewer (ES modules, no dependencies)
examples/small.dc starter layout (~1,200 elements)
examples/small-results.tsv two nightly runs of four tests
examples/mega.dc scale test (~256k elements on one page)
examples/hostnames.dc flat hostname naming (wr12r06u15 style)
examples/splice.dc four servers spliced into each cable to the ToR
examples/dual-plane.dc dual-homed servers, spliced to TOR a and TOR b,
spines per pod and two planes of superspines, with
every net's capacity
examples/dual-plane-flows.tsv flows for it: one rack across the planes at 80%,
and an incast at 160% -- tick "load the cables"
examples/dual-plane-counters.tsv switch port counters for the same run, as
hashing placed it: compare them with the model
examples/dual-plane-hosts.tsv per-server totals, for the estimated load
examples/dual-plane-health.tsv drops, soft lockups, an EDAC error and verdicts,
for the findings beside the flows
examples/iperf/ a floor plan using every layout feature, with a real
export-overlay run painted over it (see its README)
examples/hostnames-results.tsv results addressed by flat name
examples/mx/ every layout construct, painted by a real mx run
examples/live/ three wide TSV tables in one folder: the format that
needs no layout file, and what "Load all" loads
js/traffic.js cable load: flows routed over the fabric (shortest
routes, through +switch elements, hashed per
connection, adaptively, by capacity or evenly),
per-host totals estimated by
the gravity model, switch counters onto their
cables, comparisons, and what-if
js/tsv.js the wide-table reader: detection, the arrow that
makes a host a flow, tail -n, and the floor plan
read out of hostnames
js/version.js the project's version; python/dcviz/__init__.py
carries the other half and tests/run.mjs pins them
python/dcviz/ the Python side: dcadd, dcimport and `dcviz serve`,
which is what `pip install` ships. Stdlib only.
tools/dcadd, tools/dcimport
two-line shims that run those from a clone with
nothing installed, so the copy that ships and the
copy that runs are one file
pyproject.toml packaging (hatchling; no runtime dependencies)
CHANGELOG.md what changed, per version
docs/releasing.md how a release is cut, and the one manual PyPI step
docs/ the results-format guide for converting data in
tests/fixtures/ real tool output, as the contract the suite checks
tests/run.mjs headless test suite (node tests/run.mjs)
tests/browser.mjs the same for the page itself (node tests/browser.mjs) —
drives Chromium through the interactions that module
tests cannot reach. Skips with a message where
Playwright is not installed; the viewer still has no
dependencies.
.github/workflows/ both suites plus a package build on every push and
pull request; release.yml on a v* tag
.github/release_notes.py the release notes: every changelog section since
the last release
LICENSE GNU General Public License v3
License
Copyright (C) 2026 Martin J. Gallagher
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.
Full text: the LICENSE file at the repository root, or
https://www.gnu.org/licenses/gpl-3.0.html — SPDX identifier
GPL-3.0-or-later. This repository follows the
REUSE Specification: source files carry SPDX
copyright and licence headers, and the licence text lives in LICENSES/.
Metadata
Release files for datacenter-layout-viewer 1.54.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| datacenter_layout_viewer-1.54.2.tar.gz | 736.3 kB | With provenance | Details for datacenter_layout_viewer-1.54.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| datacenter_layout_viewer-1.54.2-py3-none-any.whl | Python 3 | none | any | Details for datacenter_layout_viewer-1.54.2-py3-none-any.whl |
Total release size: 1.3 MB
Release files / datacenter_layout_viewer-1.54.2.tar.gz
| Download URL | datacenter_layout_viewer-1.54.2.tar.gz |
|---|---|
| Size | 736.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f585d488a32787ff18d94b37818f49cecfd4eba65ac704533b16774b887a8922
|
|
BLAKE2b-256 checksum How to use checksums |
8d553d0a11a4c35389c3e3a8cb522bbcf786db7ba20f2e889e873ee6dedb95b1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.
Transparency logRelease files / datacenter_layout_viewer-1.54.2-py3-none-any.whl
| Download URL | datacenter_layout_viewer-1.54.2-py3-none-any.whl |
|---|---|
| Size | 578.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
10c16ee74a66310d1c469d3ebe5e0ebab24b23e2eec1f47d1359b8b3c36745d6
|
|
BLAKE2b-256 checksum How to use checksums |
fd41d1384f5835cddc6d3e0d8c8221621129293ad40fd92c9abe3f20935ad7ed
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.
Transparency logSecurity for datacenter-layout-viewer 1.54.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for datacenter_layout_viewer-1.54.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.