Skip to main content

pySAGE

A collection of Python tools for reading, editing, linting and visualising the data formats of the SAGE engine - the engine behind Command & Conquer: Generals and The Battle for Middle-earth. It grew out of an ini parser and now spans ini game data, binary maps, replays, and UI, with a domain overlay for the Edain mod.

Everything installs as one package with optional extras (see Install). Each subproject has its own README with the details; this page is the map.

Can be downloaded pre-packaged from here: https://www.moddb.com/games/battle-for-middle-earth-ii/downloads/pysage-tools

Projects

Ini game data

Project What it is
sage_ini The foundation: a typed, comment-preserving .ini parser, a whole-game loader, the cross-reference graph, and a lossless AST. Everything else builds on it.
sage_lint Formatter and linter over sage_ini - canonical reprint plus judgment rules (dangling references, out-of-range values, duplicates, unused definitions) and meta-analysis.

Binary formats

Project What it is
sage_map Reader/writer for BFME .map files, plus a game-aware overlay that resolves script arguments and object references and lints maps.
sage_asset Reader/writer, builder and combiner for asset.dat, the BFME2/RotWK art cache index - parse, build one from an art tree, merge a base with a mod overlay, and check it against the art on disk.
sage_w3d Lossless reader/writer for .w3d model files - meshes, materials, skeletons, uncompressed and compressed animation, HLOD level-of-detail data, collision boxes, and dazzles - plus a scene-assembly + PyQt6/OpenGL viewer (sage-w3d view, the w3d-view extra).
sage_cah Reader/writer for .cah, the BFME2/RotWK Create-a-Hero file - identity, class, colors, powers, "bling" customization/attributes, GUID, and the validating checksum - plus a PyQt6 editor (sage-cah-ui, the cah-ui extra) that completes over a loaded game's own classes, powers and bling.
sage_replay Reader for SAGE replay files (Generals .rep, BFME / BFME2 / RotWK) - the recorded order stream, decoded into build orders, APM and command timing.
sage_save Reader and editor for BFME2/RotWK save games (.BfME2Skirmish and friends) - a lossless container round-trip and a JSON view of the decoded chunks. Work in progress: most per-object state is still being reverse-engineered.
sage_live Python API for a running BFME2/RotWK game - attach to the process, read players, objects and upgrades as typed snapshots, and issue orders through the engine's own message stream.
sage_apt Converter, viewer and editor for .apt UI movies (the Flash-derived format behind BFME's menus and HUD). Work in progress, not yet fully functional.
sage_patch Binary patches for the ROTWK game.dat engine, with the reverse-engineering behind them - raising the CommandSet button limit from 33 to 64, opening the Create-A-Hero faction enum to mod sides, and making the AI respect a disabled hero-revive button; verify re-checks a patched binary without a disassembler.

Domain overlays & apps

Project What it is
sage_wiki Desktop tool that updates Edain wiki infoboxes from parsed game data through the MediaWiki API.
sage_ui PyQt6 desktop browser for SAGE game data: load sources, search an object, see its resolved stats.
sage_verify Maphack detection: follows a replay playing back in a live client and checks every targeted order against the engine's own shroud grid - you cannot click what you cannot see. Also compares a running client's binary-attest hash against the game.dat it should be running.
sage_test System tests that run against the real engine: declare a match and its starting objects, compile it into a .map, launch the game and assert on what it actually did.

Shared

Project What it is
sage_utils Helpers shared by more than one front end: the Qt-free data layer (sources, textures, views, the faction-graph types) and the shared Qt chrome.

Install

Requires Python ≥ 3.12. The project is pySAGE; on PyPI it is published as pysage-tools.

Note: the bare pysage name on PyPI is an unrelated, abandoned messaging library last released in 2011, and py-sage is likewise taken by another project. pip install pysage will not get you this project - install pysage-tools.

pip install pysage-tools             # core library + linter
pip install "pysage-tools[ui]"       # + the PyQt6 desktop apps (sage-ui)
pip install "pysage-tools[wiki]"     # + the wiki updater
pip install "pysage-tools[apt]"      # + reading .const/.apt out of .big archives
pip install "pysage-tools[asset-ui]" # + the SAGE Asset desktop app (build/combine asset.dat)
pip install "pysage-tools[w3d-view]" # + the OpenGL model viewer (sage-w3d view)
pip install "pysage-tools[patch]"    # + the game.dat disassembly/RE tools
pip install "pysage-tools[all]"      # + everything above at once

From a clone, for development, swap the name for an editable install of the checkout:

pip install -e ".[ui]"

The extras (ui, lint-ui, wiki, apt, asset-ui, cah-ui, w3d-view, patch, and all for the lot) pull in the optional dependencies each peripheral tool needs. The ini, map, replay, asset, w3d, cah and patch layers are stdlib-only and always ship, so no extra is needed to parse, lint or diff a map, to build and combine an asset.dat from the command line, or to apply and verify a game.dat patch. The one non-optional dependency is reversebox on Windows, the native RefPack compressor that makes saving large maps fast (its DLL is Windows-only, so other platforms use the byte-identical pure-Python compressor).

Console scripts are installed for the CLI tools: sage-ini, sage-lint, sage-live, sage-replay, sage-apt, sage-map, sage-save, sage-asset, sage-w3d, sage-cah, sage-patch, sage-verify (and the GUI scripts sage-ui, sage-wiki, sage-lint-ui, sage-asset-ui, the SAGE Asset builder/combiner window, and sage-cah-ui, the SAGE Custom Hero editor).

Mod overlays

pySAGE stays engine-generic. Knowledge of one mod's data - which faction owns which structure, which powers field permanent units, which map conventions a mod expects - lives in its own overlay package, wired in through the hooks this repo exposes (sage_utils.factiongraph, sage_replay's aggregate hooks, sage_map.checks, sage_lint's rule interface). The Edain overlay is pySAGE-edain (sage_edain): faction graphs, map checks, patch notes, horde formations, a Worldbuilder launcher and a skirmish bot.

Tests

pytest            # fast, data-free core suite
pytest --full     # + corpus acceptance gates and peripheral-package suites

Contributing

Contributions are welcome - bug reports, fixes, new checks and rules, format coverage, and documentation all help. See CONTRIBUTING.md to get set up and CONVENTIONS.md for the coding rules. AI-assisted contributions are welcome too, with one expectation: you have read, understood, and can stand behind every line you submit.

Metadata

Release files for pysage-tools 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pysage-tools 0.4.0
File Size Uploaded Provenance Actions
pysage_tools-0.4.0.tar.gz 2.2 MB Without provenance Details for pysage_tools-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pysage-tools 0.4.0
File Interpreter ABI Platform Actions
pysage_tools-0.4.0-py3-none-any.whl Python 3 none any Details for pysage_tools-0.4.0-py3-none-any.whl

Total release size: 4.6 MB

Release files / pysage_tools-0.4.0.tar.gz

Download URL pysage_tools-0.4.0.tar.gz
Size 2.2 MB
Tags Source
SHA-256 checksum
How to use checksums
10aa39f0c9b9c2576de71ca5e048d95722da5ce7463babe6aadfd6e2977a6300
BLAKE2b-256 checksum
How to use checksums
4c8e80bb779c52fd51158f3c794cd8e14b66592fa45d0ad3a0ff3e7dead73860
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.7

Release files / pysage_tools-0.4.0-py3-none-any.whl

Download URL pysage_tools-0.4.0-py3-none-any.whl
Size 2.4 MB
Tags Python 3
SHA-256 checksum
How to use checksums
fbb480c94b92d76e147597627be426d29f660c8774913152191b89c52463c102
BLAKE2b-256 checksum
How to use checksums
e9b4f3fdb2ba787b99da641344a29989850ce212172f38e8d65eae9dd32baa41
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.7

Security for pysage-tools 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page