STB — See, Think, Build
Runtime UI maps for Android & native desktop apps, bridged to static reverse engineering — with a clone scaffold and a round-trip verifier.
STB is a pipeline that gives coding agents what pure static analysis cannot see: what an app actually looks like when it runs — and which code is behind every button. v0.4 closes the loop: See the original → map it → plan the parts → scaffold a compilable clone → run it → See the clone → score how well it matches.
TARGET ──> [SEE] runtime traversal (Android emulator / AT-SPI on X11)
│ screenshots + accessibility dumps of every screen
▼
[THINK] bridge UI elements to code evidence
│ Android: jadx (R.id/text/desc) · Native: source + ext-dep
▼
[MAP] appmap.json — screens + flows + states + functions
│ one agent-digestible JSON, machine-checkable
▼
[PLAN] clone_plan.json — build parts sorted easy → hard,
│ per-part toolchain + acceptance criteria
▼
[SCAFFOLD] compilable project (GTK3 meson/ninja · Android gradle)
│ generated .ui / layouts from the observed UI
▼
[VERIFY] See(clone) vs original — per-part coverage + verdict
Why
Static analyzers (REA, Ghidra, jadx) are blind to runtime UI: they see
MainActivity.java but not that the home screen has a drawer, a search bar, and
a bottom tab bar. GUI agents (AppAgent, ARTEMIS) see the UI but are task-driven —
they complete a mission, not map the app.
STB is the missing piece: exhaustive UI documentation as an artifact, with
every element bridged to the code that implements it. Ground-truth validated
against real app sources (see stb_validate.py).
Files
| Module | Phase | What it does |
|---|---|---|
stb/traverse.py |
See (Android) | Auto-explores an APK on a headless emulator: dumps the accessibility tree, taps untried elements, screenshots each unique screen, emits a UI-map JSON plus a traversal log (*_traversal.log) that records every trigger → screen transition. |
stb/see_native.py |
See (Linux) | Same contract for desktop apps: AT-SPI tree walk, xdotool clicks, scrot screenshots, anti-thrash BFS over menus/buttons. |
stb/bridge.py |
Think (Android) | Bridges each UI element to decompiled source (R.id.*, text, content-desc) and resolves the Activity per screen. |
stb/bridge_native.py |
Think (Linux) | Bridges to a source tree plus on-disk external dependency indexes (gtksourceview language-specs/styles), classifying each miss honestly: source-string / ext-dep / runtime-generated / unresolved. |
stb/appmap.py |
Map | Merges uimap + bridge into appmap.json: screens, flows (navigation graph from the traversal log), states (variants, Jaccard > 0.8), functions (per-element event catalogue with evidence). |
stb/clone.py |
Plan | Decomposes the map into build parts sorted easy → hard, with difficulty scoring, dependencies, per-target toolchain, and acceptance criteria per part. |
stb/scaffold.py |
Build | Emits a compilable project: GTK3 (meson + generated .ui with nested menu hierarchy, one notebook page per screen) or Android (gradle + layouts + activities). |
stb/verify.py |
Verify | Round-trip grader: runs (or loads) a See pass over the clone and scores per-part acceptance coverage + label-set similarity, verdict PASS / PARTIAL. |
stb/report.py |
Build (legacy) | Self-contained HTML report with embedded screenshots and code evidence. |
stb/sanitize.py |
Guard | Prompt-injection scanner for untrusted artifacts (stb-scan). |
stb/validate.py |
eval | Ground-truth validation against the app's real manifest & source. |
Clone round-trip (v0.4)
End-to-end proven on mousepad (Xfce text editor, source-cloned ground truth):
# 1-2. See + bridge the original (Xvfb XFCE session, AT-SPI)
stb-see-native --app mousepad --max-actions 25 --out uimap.json
stb-bridge-native uimap.json --source mousepad-src/ \
--ext-dep /usr/share/gtksourceview-4/language-specs --out bridge.json
# 3. Map — one JSON: screens, flows, states, functions
stb-appmap uimap.json bridge.json --traversal-log uimap_traversal.log \
--source mousepad-src/ --out appmap.json
# 4. Plan — parts easy→hard with toolchain + acceptance
stb-clone appmap.json # → appmap_clone_plan.json
# 5. Scaffold — compilable GTK3 project
stb-scaffold appmap.json appmap_clone_plan.json --out-dir clone/
cd clone && meson setup build && ninja -C build # → runs on X11
# 6. Verify — See the clone, score against the original
stb-verify appmap.json appmap_clone_plan.json --see --app stb-clone
# → per-part coverage, verdict PASS/PARTIAL, missing-label lists
Results (mousepad): bridge 99.0% resolve (513 source / 810 ext-dep / 23 runtime-generated, 14 honest unresolved); appmap 6 screens / 6 flows / 278 functions; scaffold binary runs on X11 with the full nested menu bar; verify correctly reports part 3 (Menus & actions) 20/20 PASS while unimplemented logic parts score 0 — the gap a builder agent must close.
Setup (VM, no KVM needed)
# Android 11 in Docker via redroid (binder on host kernel)
sudo modprobe binder_linux
sudo mkdir -p /dev/binderfs && sudo mount -t binder binder /dev/binderfs
docker run -d --name stb-android --privileged -v /dev/binderfs:/dev/binderfs \
-p 5555:5555 redroid/redroid:11.0.0-latest \
androidboot.redroid_width=720 androidboot.redroid_height=1280 \
androidboot.redroid_dpi=320 androidboot.redroid_gpu_mode=guest
sleep 45 && adb connect localhost:5555
# tools
sudo apt-get install -y adb openjdk-17-jdk-headless
# jadx: https://github.com/skylot/jadx/releases
Usage
# 1. SEE — traverse the app (a few minutes, ~40 taps)
python3 stb_traverse.py org.schabi.newpipe --max-screens 20 --max-taps 40
# 2. THINK — decompile & bridge
jadx -d decompiled/newpipe --no-res -j 2 newpipe.apk
python3 stb_bridge.py <uimap>.json decompiled/newpipe
# 3. BUILD — report
python3 stb_report.py <uimap>.json <bridge>.json
# → opens as a single HTML file, works offline
Validated results (Nov 2026)
Test subjects: NewPipe 0.29.3, Aegis (open-source, F-Droid).
- NewPipe: 17 screens mapped, 471 code references bridged
- Aegis: 4 screens; every button resolved to its exact implementing class
(
IMPORT VAULT→WelcomeSlide.javaL53, radios →SecurityPickerSlide.java) - Ground truth vs real NewPipe source: all verified screens MANIFEST-OK, drawer coverage 7/10 items within a 40-tap budget (9/10 with targeted deep-dive)
FLAG_SECUREapps (Aegis) block screenshots but not accessibility dumps — STB detects and reports this as a security signal
Known limitations
- Traversal is DFS with a tap budget — deep screens (Settings, About, Player) may need more budget or targeted navigation
- Text-based code evidence can be noisy on large apps;
R.idevidence is precise - Wizard screens requiring text input (e.g. password setup) are declared "dead" and backed out of
Roadmap
- v0.1 MCP server (
see-think-build) so any coding agent can drive the pipeline - v0.2 BFS-style traversal with scroll awareness (proactive swipes on long lists/drawers)
- v0.2 Text-input actions during traversal (tap + typed sample)
- v0.2 Layout-XML bridge: res-id → declaring layout/menu XML →
R.layout.*inflate site - v0.2 Labeled-row tapping (drawer items report
clickable=falsebut still navigate) - v0.3 One-command dissection (
stb-dissect), APK acquisition (F-Droid/Aptoide), bridge dedupe - v0.3.1 Injection guard:
stb-scanscanner, Security Notes in reports,[DATA-ONLY]MCP banner - v0.4 Native desktop support: See via AT-SPI, bridge with ext-dep evidence, honest miss classes
- v0.4
appmap.json: unified agent-digestible map (screens/flows/states/functions) - v0.4 Clone plan: parts easy→hard, per-part toolchain + acceptance
- v0.4 Scaffold: compilable GTK3/Android skeleton from the map (binary runs on X11)
- v0.4 Verify: round-trip grader, per-part coverage verdict
- Implement scaffold parts 2+ (real handlers) to lift verify verdict PARTIAL → PASS
- Windows/macOS See (UIA / AX API — same uimap contract)
- Blutter-class Dart snapshot parsing for richer Flutter appmaps
- Direct deep-link navigation between screens (skip back-walking)
- Coverage regression tracking per APK version
Install
From PyPI (recommended):
pip install see-think-build
stb-server # run the MCP server (stdio)
# CLI surface (Android pipeline + native round-trip):
stb-see org.schabi.newpipe --max-screens 15
stb-think uimap.json decompiled/
stb-build uimap.json uimap_bridge.json
stb-see-native --app mousepad --max-actions 25 # Linux AT-SPI
stb-bridge-native uimap.json --source src/ --out bridge.json
stb-appmap uimap.json bridge.json --out appmap.json
stb-clone appmap.json # parts easy→hard
stb-scaffold appmap.json appmap_clone_plan.json # compilable skeleton
stb-verify appmap.json appmap_clone_plan.json --uimap clone_uimap.json
stb-scan <file> # injection guard
From source:
git clone https://github.com/Kian1904/stb && cd stb
pip install .
Python 3.10+; only hard dependency is the mcp package. adb and jadx
are external binaries you install separately (see below).
Cross-platform setup
STB runs on Linux, macOS, and Windows. It talks to anything adb can reach: an official Android Studio emulator (AVD), a physical phone over USB, or a redroid container (Linux/Docker only). Requirements:
- Python 3.10+
- adb (Android platform-tools) — on PATH or via
STB_ADB - jadx — on PATH or via
STB_JADX(github.com/skylot/jadx; needs Java 17+)
Setup per OS:
| Step | Linux | macOS | Windows |
|---|---|---|---|
| adb | sudo apt install adb or platform-tools zip |
brew install android-platform-tools |
Download platform-tools zip, add to PATH |
| jadx | unzip jadx-x.y.zip → add bin/ to PATH |
same | same (jadx.bat) |
| Emulator | Android Studio AVD, USB phone, or redroid (Docker) | AVD or USB phone | AVD or USB phone |
| MCP config | python3 |
python3 |
python (or full path) |
Device targeting — set STB_ADB_SERIAL if more than one device is connected
(find serials with adb devices -l):
export STB_ADB_SERIAL=emulator-5554 # or 127.0.0.1:5555, or a USB serial
export STB_JADX=/path/to/jadx/bin/jadx
export STB_DIR=~/stb # where maps/reports/decompiled go
Run stb_status first — it reports exactly what is missing and how to fix it.
Using a physical phone (Termux-friendly)
- Enable USB debugging (Developer options), connect, accept the fingerprint
- From Termux:
pkg install python android-tools, clone this repo python stb_server.py— the phone you're debugging can be a second device; the controlling agent runs anywhere adb works
MCP server
STB also ships as an MCP server (stb_server.py) so any MCP-capable coding agent
(Claude Code, Codex, Hermes, …) can drive the whole pipeline:
| Tool | What it does |
|---|---|
stb_status |
Lab health: container, adb, boot state, packages, jadx |
stb_see |
Traverse an app → UI-map JSON (screens + elements + screenshots) |
stb_dump_ui |
Accessibility tree of the CURRENT screen (works on FLAG_SECURE apps) |
stb_screencap |
Screenshot of the current screen |
stb_jadx |
Decompile an APK (jadx) |
stb_think |
Bridge UI-map elements to decompiled code |
stb_build |
Generate the self-contained HTML report |
stb_validate |
Ground-truth check vs real app source |
Config (generic stdio):
{
"mcpServers": {
"stb": { "command": "python3", "args": ["/path/to/stb_server.py"] }
}
}
Env overrides: STB_DIR, STB_ADB_SERIAL, STB_ADB, STB_JADX,
STB_RUN_DIR, STB_CONTAINER (see Cross-platform setup above).
License
MIT
Metadata
Release files for see-think-build 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| see_think_build-0.4.0.tar.gz | 55.7 kB | Without provenance | Details for see_think_build-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| see_think_build-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for see_think_build-0.4.0-py3-none-any.whl |
Total release size: 119.5 kB
Release files / see_think_build-0.4.0.tar.gz
| Download URL | see_think_build-0.4.0.tar.gz |
|---|---|
| Size | 55.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2a5003cc1627adf9398db5317464bf33e85a2ee4dce51d3897e5120e7a6e6cd9
|
|
BLAKE2b-256 checksum How to use checksums |
73117dfa88d0e7343232b9f99a4bc6ae536130edca680417bcee5c2aaefc07cc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / see_think_build-0.4.0-py3-none-any.whl
| Download URL | see_think_build-0.4.0-py3-none-any.whl |
|---|---|
| Size | 63.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2734cdffc96263a5fe4db02127840df109ade53d790c84a053a0f5cc8e71f8ac
|
|
BLAKE2b-256 checksum How to use checksums |
fe99fc5b2e4ac7ccfd56333170d340f7f253dfcb8d8abab77e1d7cb427ea334e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Security for see-think-build 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.