chklib

A read/write library for StarCraft map data, and chkdiff — a CHK-aware diff tool that makes git diff say something useful about a .scx file.

Status: working end to end. Container, typed views, terrain (including ISOM), string-table editing (STR and STRx), MPQ reading and writing, chkdiff inspect, chkdiff diff, pack/unpack, and the git integration are all implemented and tested. A map can be opened, edited — including its name, description and any trigger text — and saved back to a playable archive.

Why

Everything that can read a StarCraft map today is one of three things:

• read-only — bw-chk parses scenario.chk in JavaScript, triggers included, but has no writer at all and does not parse locations.
• a section bag — eudplib models a CHK as dict[bytes, bytes]. It collapses duplicate sections and its writer injects a junk ISOM section, so it does not round-trip.
• welded into a Windows GUI — Chkdraft has the only complete read/write CHK engine that exists, including a trigger text compiler, but it is C++ inside a Win32 application with no bindings.

There is no library anywhere that reads and writes a CHK losslessly with a typed trigger model. That is the gap this fills.

See docs/first-consumer-scope.md for the scope and the acceptance gate, and docs/contribution-targets.md for a survey of the surrounding ecosystem.

The container

A scenario.chk is a flat sequence of sections: a 4-byte name, a signed 32-bit little-endian length, then that many payload bytes. Chk preserves three things other readers discard:

• Order — StarCraft applies sections in file order, later overriding earlier. A mapping has already lost the information needed to know which wins.
• Duplicates — deliberately duplicated sections are a standard protection technique.
• Raw bytes — unknown sections, and sections whose declared length disagrees with reality, survive untouched.

from chklib import Chk

chk = Chk.from_bytes(open("scenario.chk", "rb").read())

len(chk)                    # 37
chk.last("DIM").data        # b'@\x00@\x00'  - the section StarCraft would use
chk.find("VER")             # every section with that name, in file order
chk.duplicated_names        # [b'VER '] if the map is protected that way
chk.diagnostics             # problems, reported rather than raised

chk.to_bytes() == raw       # True, always

Malformed input is never raised on. Half the interesting maps in the wild are malformed on purpose, and a parser that refuses them is useless for exactly the maps people care about.

Typed views

Views are layered over the raw bytes, never in place of them, so a read/write cycle reproduces the input exactly — including short sections, undocumented flag bits, and the fields the format sources only call unused.

from chklib import Chk
from chklib.views import view_for
from chklib.enums import Tileset, ActionType

chk = Chk.from_bytes(raw)

view_for(chk, "DIM")            # Dimensions(64x64), .pixel_width, .tile_count
view_for(chk, "VER").name       # 'Hybrid (Brood War compatible)'
Tileset(view_for(chk, "ERA").value).name          # 'Badlands'

strings = view_for(chk, "STR")
sprp    = view_for(chk, "SPRP")
strings.text(sprp.name_string_id)                 # 'Tutorial 1'

for unit in view_for(chk, "UNIT"):                # 36-byte records
    unit.owner, unit.xc, unit.yc, unit.type

for trigger in view_for(chk, "TRIG"):             # 2400-byte records
    for action in trigger.used_actions():
        ActionType(action.action_type)            # CreateUnitWithProperties, ...

Covered — all 41 sections:

DIM VER ERA OWNR IOWN SIDE SPRP FORC          map and players
TYPE IVER IVE2 VCOD                           versions and validation
UNIT THG2 MRGN DD2                            objects, locations, doodads
MTXM TILE MASK ISOM                           terrain
STR STRx                                      strings
TRIG MBRF UPRP UPUS                           triggers, briefings, unit properties
WAV SWNM UNIS UNIx UPGS UPGx TECS TECx        settings
PUNI UPGR PUPx PTEC PTEx                      player restrictions
COLR CRGB                                     colours

Nothing documented is left as opaque bytes. Verified rather than asserted: across the 488 maps this suite runs against, view_for builds 16,505 views over all 41 section kinds with zero round-trip mismatches and zero failures. The only section names left unclaimed are deliberate forgeries — protected maps ship TlLE, PUNl, FORc and SlDE, which swap a lowercase l for an I to mislead editors that match loosely.

Three behaviours worth knowing, each of which a naive implementation gets wrong:

• TRIG and MBRF share a byte layout but not an action id space. Ids 0–9 mean entirely different things in each, so a view carries is_briefing.
• String ids are 1-based, and strings end at the next NUL — never at the following slot's offset. In 30 of the 65 corpus maps the string data is not in ascending id order, so offset-differencing yields negative lengths.
• Location ids are 1-based too: Anywhere is id 64, which is file record 63.

The format work behind this is in .research/SPEC.md (gitignored, regenerable): six independent implementations cross-checked against each other and validated against the corpus, with every claim carrying a confidence tier and every unresolved disagreement listed rather than guessed at.

Settings tables

WAV, SWNM, UNIS/UNIx, UPGS/UPGx and TECS/TECx are all the same shape: a fixed-size section of parallel arrays, one entry per unit, upgrade, technology, switch or sound. This is where a map's custom unit statistics live.

from chklib.settings import UnitSettings, settings_for

units = settings_for(chk, "UNIS")
units.customised_units()            # [151]
units.custom_name_id(151)           # 30  -> "Cerebrate Zasz"
UnitSettings.displayed_hitpoints(units["hitpoints"][151])   # 1500

Four traps, each producing a wrong map rather than an error:

useDefault is inverted from the obvious reading. No = 0, Yes = 1, so a set flag means use the game's built-in stats and ignore the custom data here — which makes the custom name id dead. Read as "this entry is customised" and you invert every unit in the map. custom_name_id returns 0 behind a set flag rather than a name the game never shows.

The unset value is 1, not 0. These structs are built with memset(&useDefault, Yes, ...), so an entry a short section never reached uses defaults. Zero-filling the gap — the obvious way to pad — asserts the exact opposite, and the first edit writes that inversion to disk.

UPGx has one pad byte after its flag array and UPGS does not — and TECS/TECx have none, so the rule doesn't generalise. Its position is the whole trap: move it to the end of the layout and the section still totals 794 bytes while all six cost arrays shift.

Unit hitpoints are stored at 256× the displayed value. Both directions are provided; reading with displayed_hitpoints and writing the value straight back would otherwise store 1/256th of the intended health, silently.

How the layout is actually checked

A round-trip test cannot verify field order. _pack and _unpack walk the same layout in the same order, so swapping two same-width fields still round-trips every byte perfectly, still totals 4048, and still puts nameStringId at 3192. Re-deriving an offset by summing the layout is no better — it only asks the code to agree with itself.

So the field order is transcribed from Chkdraft's REFLECT declarations, and the tests pin each field at a literal byte offset that is never computed from the code under test. A golden test reads a known fixture and asserts the actual values (Cerebrate Zasz, 1500 hp, armor 1), which a transposed layout or a two-byte drift cannot satisfy.

Measured across the 423 installed maps, and recorded as tests rather than assumed:

• 166 carry both UNIS and UNIx. Which one wins is formally unresolved, but their shared arrays are byte-identical in all 166, so the ambiguity has no practical effect here. If that ever stops being true, precedence starts to matter.
• Duplicate settings sections are real — one map ships two WAV sections. They resolve last-wins: only MTXM takes the prefix-patch merge the terrain grids use.

Player restriction tables

Where the settings tables say what a unit is, PUNI, UPGR/PUPx and PTEC/PTEx say what each player may do with it.

from chklib import restrictions_for

puni = restrictions_for(chk, "PUNI")
puni.buildable(player=3, unit=45)        # follows the uses-default flag
puni.customised_units()                  # units some player restricts

upgr = restrictions_for(chk, "UPGR")
upgr.max_level(player=0, upgrade=0)      # 3

Every one of these is player-major, with twelve players. player_unit_buildable is u8[12][228] flattened row by row, so player 3's entry for unit 45 is at 3 * 228 + 45. Index it the other way and you read a real value belonging to a different player — use .at() and .set_at() rather than doing the arithmetic. Twelve, not eight: COLR and FORC have eight playable slots, these follow OWNR.

The unset value is 1 for most of these arrays. As with useDefault, the flags are constructed set, and PUNI additionally starts with everything buildable. Zero-filling a short section asserts that nothing is available and nothing uses defaults — the exact opposite of an untouched map.

Reading the per-player array alone reports a value the game ignores. A player whose uses_default flag is set follows the global row instead. The accessors above apply that rule; the raw arrays are still there when you want what is literally on disk.

What is settled and what is not

The layout is as well attested as anything here: openbw reads all five independently of Chkdraft, llvm-bw's map of StarCraft.exe pins the counts 61 and 44 a third time, and the corpus agrees exactly — all 488 PUNI, 328 UPGR, 391 PUPx, 328 PTEC and 391 PTEx sections are precisely nominal size, with not one short or oversized.

The default tables are weaker, and are labelled that way rather than presented as constants. Only Chkdraft states them. Measured against real maps they are a common map's contents, not a universal truth:

Both readings are legitimate; a map saying "no technology starts researched" is making a choice. Chkdraft's values are kept for synthesising a section from nothing, because they are traceable to a cited source — and since no section in the corpus is short, that is the only time they are ever used.

Trigger unit properties, doodads and the rest

UPRP holds 64 twenty-byte property slots that CreateUnitWithProperties actions index into; without it such an action can be named but not explained. UPUS says which slots are real — seven corpus maps declare it zero-length, which reads as "none in use".

DD2 is editor-only doodads, 8 bytes each. Its enabled byte is inverted: 0 means enabled, so bool(enabled) is exactly backwards and a doodad written with 1 is silently switched off. Use Doodad.is_enabled. Seven corpus maps carry a 60-byte DD2 — seven whole doodads and half of an eighth — so the trailing partial record is preserved rather than dropped or padded out.

COLR is eight colour bytes. They are not validated against a named colour: the corpus carries 14 and 15, and a Remastered map can go higher, so anything rejecting an unnamed value would refuse real maps.

CRGB is the least corroborated section in this library and says so in its docstring. Only Chkdraft describes it, and unusually does not annotate its size the way it does every neighbour. Exactly one map of 488 carries one; it is 32 bytes with eight uniform setting bytes, which is what the 24/8 split predicts.

VCOD is the 1040-byte checksum seed table. It is very nearly a constant — 483 of 488 maps hold byte-identical payloads — so is_standard lets pack check it rather than copy it blindly. The five that differ are the interesting ones: a non-standard VCOD is how a map is made to refuse to open in an editor that recomputes it.

chkdiff inspect

A deterministic textual rendering of a scenario, designed so that git diff can use it as a textconv driver.

chkdiff inspect scenario.chk
chkdiff inspect --stable scenario.chk    # omit the filename, for git

The point is that a small change to a map becomes a small change in the text. Moving one unit and dropping its hitpoints — 3 changed bytes in the binary — reads as:

-p12   type=188   at=(224,448)  hp=100%  resources=5000
+p12   type=188   at=(288,448)  hp=55%   resources=5000

Three rules make that work, and they are what the tests pin down:

• Determinism. The same bytes always produce the same characters — no paths, no timestamps, no iteration-order accidents. Verified over all 65 corpus maps.
• One fact per line. A moved unit is one changed line, not a reflowed block.
• Canonical ordering where order carries no meaning. Units and sprites are sorted by content and carry no index, so inserting one produces a constant-size diff regardless of how many units the map has. Triggers, locations and strings keep file order, because for them the index is the identity.

Nothing is invented: unit and sprite types print as numbers, because naming them needs units.dat from a StarCraft installation, which this library does not read.

chkdiff diff

Compares two scenarios by meaning rather than by bytes. Exit status follows diff(1): 0 when identical, 1 when they differ.

chkdiff diff a.chk b.chk
chkdiff diff --json a.chk b.chk

~ map  name
    - #1 "Z6) The Dark Templar"
    + #1 "Z8) The Dark Templar"
~ players  p6 race
    - 2 Protoss
    + 0 Zerg
~ forces  force2 members
    - p2,p3
    + p2

A byte diff can't do this — inserting one string shifts every offset after it, and one new trigger moves 2400 bytes of everything downstream. So each section is compared using whatever notion of identity it actually has:

Triggers are the hard case: no ids, but their order is execution order, so they can't be treated as a set either. A positional comparison reports every later trigger as modified the moment you insert one at the top.

The fix is an LCS alignment (difflib.SequenceMatcher) over per-trigger content hashes, with survivors inside each replace-block paired greedily by similarity. Inserting a trigger at position 0 of a 20-trigger map reports one addition. Insert one and edit a later one, and you get:

+ TRIG  trigger 0
    + owners=[18] | flags=0 | if Bring(...) | do CreateUnit(...)
~ TRIG  trigger 6->7
    - do RemoveUnit(group=17, type=101, flags=0x14)
    + do RemoveUnit(group=17, time=12345, type=101, flags=0x14)

The 6->7 records that the trigger both moved and changed. It degrades gracefully: below the similarity threshold a replacement is reported as an add plus a remove, which is correct, just less informative.

Editing

from chklib import Chk, StringTable
from chklib.mpq import MpqArchive, write_scenario, SCENARIO_PATH
from chklib.views import view_for

chk = Chk.from_bytes(MpqArchive(open("map.scm", "rb").read()).read_file(SCENARIO_PATH))

# terrain, players, units, triggers - edit the typed view, write it back
view_for(chk, "ERA").value = 4
chk.replace_section("ERA", view_for(chk, "ERA").to_bytes())

# strings - the map name, description, location names and all trigger text
strings = StringTable.from_view(view_for(chk, "STR"))
strings[view_for(chk, "SPRP").name_string_id] = "Blood Bath (Remix)"
new_id = strings.add("a string that did not exist before")
chk.replace_section("STR", strings.to_bytes())

open("edited.scx", "wb").write(write_scenario(chk.to_bytes(), compress=True))

StringTableView reads; StringTable writes. Two things about it are load-bearing:

String ids are positional and gaps are preserved. Id 7 is referenced as 7 from SPRP, MRGN, FORC and every trigger, so compacting the table would silently repoint every reference in the map without touching a single trigger.

Offsets are 16-bit, and the limit is enforced honestly. Chkdraft's own guard sums string lengths without the terminating NUL its writer then emits, so it accepts payloads a little over 64 KB and writes offsets that wrap modulo 65536 — a corrupt map with no error raised. This counts the NULs and raises instead. The id ceiling (32766) is likewise derived from the offset table filling the addressable space, and named as such rather than inherited: the sources give four different ceilings across five orders of magnitude, none of which is a format limit.

Rebuilding the string table of all 65 corpus maps preserves every string at its own id, and a rename leaves every location name still resolving.

Terrain

from chklib.views import terrain_for

game = terrain_for(chk, "MTXM")     # what StarCraft reads
editor = terrain_for(chk, "TILE")   # the editor's ISOM-derived layer
fog = terrain_for(chk, "MASK")

game[5, 42] = 0x0864                # (x, y), row-major
chk.replace_section("MTXM", game.to_bytes())

Three things here corrupt terrain silently rather than raising, so each is pinned by a test:

Indexing is row-major, y * width + x. Chkdraft's own header comments declare these arrays column-major — and the same wrong comment appears verbatim on MTXM, TILE, ISOM and MASK — while every accessor in that codebase uses row-major. On a square map the two are indistinguishable, so the tests use non-square maps.

MTXM and TILE are distinct layers, never aliases. They are byte-identical in only 1 of 65 corpus maps and differ in the other 64 by a mean of 4.8% of tiles, worst 13.2%.

A set bit in MASK means the tile is fogged for that player, bit 0 being player 1.

Short, long and odd-length sections are read rather than refused — the game itself tolerates them, and blackvrice's hard error would reject real protected maps. to_bytes() preserves the original length so an unmodified short section stays short; normalize=True emits the full grid, which is what Chkdraft always does.

ISOM

The editor's isometric terrain, on its own grid: isom_width = tileWidth // 2 + 1, isom_height = tileHeight + 1, 8 bytes per record.

from chklib.views import isom_for

grid = isom_for(chk)                 # 33x65 records for a 64x64 map
grid[1, 2].values()                  # the four sides, editor flags masked off

Its bit layout is the least corroborated thing in the format — Chkdraft is the only witness — so each side stays a raw u16 and the accessors are an interpretation over it. Measuring 488 real maps supports the layout well:

That last row matters: the layout says Chkdraft merely clears those bits by convention and nothing guarantees it. Real maps carry them, so masking on read is necessary rather than defensive.

Two traps avoided. Chkdraft's scenario.cpp pads with expected - actual computed in size_t after testing !=, so an oversized ISOM underflows into an astronomical insert; padding here is short-only. And eudplib writes a decoy ISOM with a length past 0x80000000 as a protection marker — the container already stops at a negative section length, so such a map simply has no ISOM rather than a fabricated one.

STRx

Remastered maps use STRx — exactly STR with the count and every offset widened from u16 to u32, nothing else changed. string_table_for(chk) returns whichever table a map's references actually resolve against, because STRx supersedes STR in either file order and that rule is not expressible as a section lookup. Chkdraft, bw-chk and eudplib agree on it; blackvrice abstains when both are present and so fails to open ordinary Remastered maps that kept a legacy STR.

Verified against the 24 STRx maps in a real installation: every scenario name and description resolves, and rebuilding each table preserves every string at its id. Reading one of those sections at the wrong width yields empty strings rather than an error — which is why the width is decided by the section name and never sniffed.

An empirical note the format sources don't carry: of 423 installed maps, 24 use STRx and none uses both, so the precedence rule is real but unexercised by those maps.

Reading map files

.scm/.scx maps are MPQ archives, so the library reads those directly — that is what makes chkdiff usable on files as they actually exist in a repository.

from chklib.mpq import MpqArchive, SCENARIO_PATH

chk_bytes = MpqArchive(open("(4)Blood Bath.scm", "rb").read()).read_file(SCENARIO_PATH)

Reading is deliberately partial: MPQ v1 with encrypted hash/block tables, multi-sector and single-unit files, FIX_KEY, and the compressions maps actually use. MPQ v2 is refused rather than guessed at, because a v2 archive parsed as v1 produces plausible-looking wrong bytes.

Saving

from chklib.mpq import write_scenario

open("edited.scx", "wb").write(write_scenario(chk_bytes, compress=True))

chkdiff unpack "(4)Blood Bath.scm" scenario.chk
chkdiff pack --compress scenario.chk rebuilt.scx

Writing produces a plainly laid out v1 archive and encrypts nothing — encryption exists to make files hard to extract, which buys a map editor nothing.

On compression, two tools whose maps demonstrably load in StarCraft disagree, and both are fine: euddraft writes zlib (MPQ_COMPRESSION_ZLIB), while sc64-maps stores everything plainly. So the default is stored — the option nothing can refuse — and --compress opts into zlib on the strength of euddraft's production use. Writing PKWARE implode, what Blizzard's own maps use, would need a compressor; only the decompressor is implemented here.

Round-tripping every installed map through the writer and back out via StormLib:

chkdiff pack refuses a scenario that fails to parse, since it would fail in StarCraft too; --force overrides.

A permissive off-the-shelf reader wasn't an option: mpyq (BSD) cannot open a single genuine Blizzard map — it has no decryption, and no PKWARE implode, which is compression method 0x08 and what Blizzard maps overwhelmingly use. So both the archive layer and a PKWARE exploder are implemented here from the format description.

Verification is by ground truth, not inspection. Every scenario is compared byte-for-byte against the same file extracted by StormLib:

Eight of those maps are deliberately protected — seven declare a hashTableSize of 0x10000400 instead of 0x400, and one hides its hash entry away from its home slot. Both are handled (the declared size is clamped to what the file can hold; a failed probe falls back to a full scan), and both are recorded on the archive object rather than silently absorbed, so a caller can tell a protected map from a clean one.

One honest gap: the literal-mode byte was 0 in all 488 maps, so the 256-symbol Huffman table for coded literals has never been exercised against ground truth. It is marked unverified in the source.

Git integration

git diff on a map file says this today:

Binary files a/map.scx and b/map.scx differ

Two lines of setup replace that with the actual change:

chkdiff install-textconv          # prints the commands; --write applies them

git config --global diff.starcraft.textconv "chkdiff textconv"
git config --global diff.starcraft.binary false

plus, in .gitattributes:

*.scm diff=starcraft
*.scx diff=starcraft
*.chk diff=starcraft

After which the same commit reads:

 [map]
 version      205  Brood War
 tileset      5  Desert
-dimensions   128x96 tiles  (4096x3072 px)
-name         #1 "Dust Bowl"
+dimensions   128x128 tiles  (4096x4096 px)
+name         #1 "Hot Zone"

chkdiff textconv exists as its own subcommand rather than being an alias for inspect --stable, because a textconv driver has a requirement ordinary commands don't: it must never fail. Git runs it over every blob on both sides of a diff, including historical ones that may be truncated, protected, or not maps at all — and a driver that exits non-zero makes git diff fail outright, which is worse than no driver. So it always exits 0, always prints something, and degrades unreadable input to a short deterministic note that never contains the randomised temp path git passes in.

The tests drive a real git repository rather than mocking it, and assert the status quo (Binary files ... differ) as well as the improvement.

Development

py -3.13 -m venv .venv
.venv/Scripts/python -m pip install -e ".[dev]"
.venv/Scripts/python -m pytest

That runs everything except the tests that need real maps — and those are the ones that found most of the bugs here, so they are worth enabling. Point two environment variables at map corpora you already have:

export CHKLIB_SC_MAPS="/path/to/StarCraft/Maps"        # a StarCraft installation
export CHKLIB_SC64_MAPS="/path/to/sc64-maps/gamedata"  # optional, see below

CHKLIB_SC_MAPS is the valuable one. A StarCraft install ships hundreds of maps including protected ones, Remastered STRx maps and current ladder maps — the corpus that turned up duplicate MTXM, short and odd terrain sections, and every MPQ edge case the reader handles.

CHKLIB_SC64_MAPS points at output from sc64-maps, scenarios extracted from a StarCraft 64 cartridge. They are useful precisely because StarEdit never touched them, so they catch assumptions a PC-only corpus cannot.

Some tests additionally need extracted scenario.chk fixtures, which are gitignored because map archives are copyrighted and are not redistributed here:

pip install -e ".[fixtures]"          # adds eudplib, Python 3.10-3.13 only
python tools/extract_fixtures.py "path/to/maps/*.scm"

Fixtures are produced with eudplib's MPQ reader rather than our own, deliberately: a gate generated by an independent implementation cannot pass by agreeing with our own bugs.

Everything is optional. Anything not found makes the tests that need it skip, never fail, so a clean clone with no StarCraft installed still has a green suite.

The name

Named after the CHK file format, following what the rest of this ecosystem does: Chkdraft, ChkForge and bw-chk all name the format, and StormLib and CascLib name a technology. None of them names a Blizzard product.

This project was briefly called openstaredit, which was a mistake worth not repeating: StarEdit.exe is a Blizzard product that ships with the game, and prefixing a product name with open reads as "the open replacement for it" — exactly the association to avoid. A file format's four-character code carries no such baggage.

License

MIT — see LICENSE.

StarCraft is a trademark of Blizzard Entertainment, Inc. This project is not affiliated with or endorsed by Blizzard Entertainment, and redistributes none of its data: the test corpora are read from your own installation and are gitignored.