Skip to main content

pySAGE

A collection of Python tools for reading, editing, linting and visualising the data formats of the SAGE engine - the engine behind Command & Conquer: Generals and The Battle for Middle-earth. It grew out of an ini parser and now spans ini game data, binary maps, replays, and UI, with a domain overlay for the Edain mod.

Everything installs as one package with optional extras (see Install). Each subproject has its own README with the details; this page is the map.

Can be downloaded pre-packaged from here: https://www.moddb.com/games/battle-for-middle-earth-ii/downloads/pysage-tools

Projects

Ini game data

Project What it is
sage_ini The foundation: a typed, comment-preserving .ini parser, a whole-game loader, the cross-reference graph, and a lossless AST. Everything else builds on it.
sage_lint Formatter and linter over sage_ini - canonical reprint plus judgment rules (dangling references, out-of-range values, duplicates, unused definitions) and meta-analysis.

Binary formats

Project What it is
sage_map Reader/writer for BFME .map files, plus a game-aware overlay that resolves script arguments and object references and lints maps.
sage_asset Reader/writer, builder and combiner for asset.dat, the BFME2/RotWK art cache index - parse, build one from an art tree, merge a base with a mod overlay, and check it against the art on disk.
sage_w3d Lossless reader/writer for .w3d model files - meshes, materials, skeletons, uncompressed and compressed animation, HLOD level-of-detail data, collision boxes, and dazzles - plus a scene-assembly + PyQt6/OpenGL viewer (sage-w3d view, the w3d-view extra).
sage_cah Reader/writer for .cah, the BFME2/RotWK Create-a-Hero file - identity, class, colors, powers, "bling" customization/attributes, GUID, and the validating checksum - plus a PyQt6 editor (sage-cah-ui, the cah-ui extra) that completes over a loaded game's own classes, powers and bling.
sage_replay Reader for SAGE replay files (Generals .rep, BFME / BFME2 / RotWK) - the recorded order stream, decoded into build orders, APM and command timing.
sage_save Reader and editor for BFME2/RotWK save games (.BfME2Skirmish and friends) - a lossless container round-trip and a JSON view of the decoded chunks. Work in progress: most per-object state is still being reverse-engineered.
sage_live Python API for a running BFME2/RotWK game - attach to the process, read players, objects and upgrades as typed snapshots, and issue orders through the engine's own message stream.
sage_apt Converter, viewer and editor for .apt UI movies (the Flash-derived format behind BFME's menus and HUD). Work in progress, not yet fully functional.
sage_patch Binary patches for the ROTWK game.dat engine, with the reverse-engineering behind them - raising the CommandSet button limit from 33 to 64, opening the Create-A-Hero faction enum to mod sides, and making the AI respect a disabled hero-revive button; verify re-checks a patched binary without a disassembler.

Domain overlays & apps

Project What it is
sage_wiki Desktop tool that updates Edain wiki infoboxes from parsed game data through the MediaWiki API.
sage_ui PyQt6 desktop browser for SAGE game data: load sources, search an object, see its resolved stats.
sage_verify Maphack detection: follows a replay playing back in a live client and checks every targeted order against the engine's own shroud grid - you cannot click what you cannot see. Also compares a running client's binary-attest hash against the game.dat it should be running.
sage_test System tests that run against the real engine: declare a match and its starting objects, compile it into a .map, launch the game and assert on what it actually did.

Shared

Project What it is
sage_utils Helpers shared by more than one front end: the Qt-free data layer (sources, textures, views, the faction-graph types) and the shared Qt chrome.

Install

Requires Python ≥ 3.12. The project is pySAGE; on PyPI it is published as pysage-tools.

Note: the bare pysage name on PyPI is an unrelated, abandoned messaging library last released in 2011, and py-sage is likewise taken by another project. pip install pysage will not get you this project - install pysage-tools.

pip install pysage-tools             # core library + linter
pip install "pysage-tools[ui]"       # + the PyQt6 desktop apps (sage-ui)
pip install "pysage-tools[wiki]"     # + the wiki updater
pip install "pysage-tools[apt]"      # + reading .const/.apt out of .big archives
pip install "pysage-tools[asset-ui]" # + the SAGE Asset desktop app (build/combine asset.dat)
pip install "pysage-tools[w3d-view]" # + the OpenGL model viewer (sage-w3d view)
pip install "pysage-tools[patch]"    # + the game.dat disassembly/RE tools
pip install "pysage-tools[all]"      # + everything above at once

From a clone, for development, swap the name for an editable install of the checkout:

pip install -e ".[ui]"

The extras (ui, lint-ui, wiki, apt, asset-ui, cah-ui, w3d-view, patch, and all for the lot) pull in the optional dependencies each peripheral tool needs. The ini, map, replay, asset, w3d, cah and patch layers are stdlib-only and always ship, so no extra is needed to parse, lint or diff a map, to build and combine an asset.dat from the command line, or to apply and verify a game.dat patch. The one non-optional dependency is reversebox on Windows, the native RefPack compressor that makes saving large maps fast (its DLL is Windows-only, so other platforms use the byte-identical pure-Python compressor).

Console scripts are installed for the CLI tools: sage-ini, sage-lint, sage-live, sage-replay, sage-apt, sage-map, sage-save, sage-asset, sage-w3d, sage-cah, sage-patch, sage-verify (and the GUI scripts sage-ui, sage-wiki, sage-lint-ui, sage-asset-ui, the SAGE Asset builder/combiner window, and sage-cah-ui, the SAGE Custom Hero editor).

Mod overlays

pySAGE stays engine-generic. Knowledge of one mod's data - which faction owns which structure, which powers field permanent units, which map conventions a mod expects - lives in its own overlay package, wired in through the hooks this repo exposes (sage_utils.factiongraph, sage_replay's aggregate hooks, sage_map.checks, sage_lint's rule interface). The Edain overlay is pySAGE-edain (sage_edain): faction graphs, map checks, patch notes, horde formations, a Worldbuilder launcher and a skirmish bot.

Tests

pytest            # fast, data-free core suite
pytest --full     # + corpus acceptance gates and peripheral-package suites

Contributing

Contributions are welcome - bug reports, fixes, new checks and rules, format coverage, and documentation all help. See CONTRIBUTING.md to get set up and CONVENTIONS.md for the coding rules. AI-assisted contributions are welcome too, with one expectation: you have read, understood, and can stand behind every line you submit.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pysage_tools-0.4.0.tar.gz (2.2 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pysage_tools-0.4.0-py3-none-any.whl (2.4 MB view details)

Uploaded Python 3

File details

Details for the file pysage_tools-0.4.0.tar.gz.

File metadata

  • Download URL: pysage_tools-0.4.0.tar.gz
  • Upload date:
  • Size: 2.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.7

File hashes

Hashes for pysage_tools-0.4.0.tar.gz
Algorithm Hash digest
SHA256 10aa39f0c9b9c2576de71ca5e048d95722da5ce7463babe6aadfd6e2977a6300
MD5 0d77c7cad8f19c3db5f67873b8b292a1
BLAKE2b-256 4c8e80bb779c52fd51158f3c794cd8e14b66592fa45d0ad3a0ff3e7dead73860

See more details on using hashes here.

File details

Details for the file pysage_tools-0.4.0-py3-none-any.whl.

File metadata

  • Download URL: pysage_tools-0.4.0-py3-none-any.whl
  • Upload date:
  • Size: 2.4 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.7

File hashes

Hashes for pysage_tools-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 fbb480c94b92d76e147597627be426d29f660c8774913152191b89c52463c102
MD5 67306f624b2fbff3a6cc00531835d5a9
BLAKE2b-256 e9b4f3fdb2ba787b99da641344a29989850ce212172f38e8d65eae9dd32baa41

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

0.4.0 This release

2 files

0.3.0

2 files

0.2.0

2 files

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page