SmartCLI

Read this in: English · 简体中文 · 繁體中文 · 日本語 · 한국어

A local Python toolkit for driving, perceiving, and rendering the terminal — three agent skills over one pluggable PTY + pyte core.

Let an AI drive, perceive, and render real terminal programs. SmartCLI reads the actual screen with a pyte cell model — not a byte pipe — so it knows which menu row is highlighted, presses the right keys, and waits for the screen to settle. Below: it drives the real lazygit TUI end-to-end (arrow-key navigation, opening a commit diff, highlighting a branch) — no script, no mock.

pip install smartcli-toolkit

Requires Python 3.10 or newer. The install includes the shared Python library, the persistent TUI driver, and the stdio MCP server.

What & why

SmartCLI is a workspace for terminal work that agents and humans both do: driving interactive terminal programs, perceiving what a screen actually shows, and rendering visuals and layouts back out. It is built on one shared, pluggable PTY backend plus a pyte screen model — chosen over screenshot/vision so a single structured screen model feeds both perception (read the screen) and rendering (draw the screen). The PTY layer is intentionally not tmux-bound: local dev runs on Windows via ConPTY (pywinpty), while target programs can run under POSIX ptys or tmux elsewhere. Three skills sit on that core, each a self-contained tool you run in place from the checkout.

Driving a real TUI

The demo above is SmartCLI driving lazygit — a real full-screen curses app — through its perceive → act → confirm loop: it reads the pyte cell grid (which row is selected, the alt-screen diff), moves with arrow keys, opens a commit's diff, and highlights a branch. Captured by driving the actual program in a Linux container, not scripted or mocked. A byte-stream matcher like pexpect can't perceive "which row is highlighted"; a screen model can.

Honest scope: CI runs a Windows + Linux + macOS matrix. The POSIX pty backend (spawn / read / drive / resize / zombie-free terminate) is verified on Linux and macOS in CI; the interactive DECCKM/SS3-arrow probe is skipped on CI runners (no controllable terminal) and still wants a real-host run. Real tmux is not yet verified — known edges are listed in skills/drive-tui/references/LIMITATIONS.md.

Verified on Windows 11, Python 3.14.6, pyte + pywinpty / ConPTY. This machine has no real tmux, so screenshot reports are honestly labelled pyte-simulation, not real tmux captures.

Live effects

Real captures of the cmd-art fx engine — each GIF is the actual effect rendered frame-by-frame through the project's own pipeline (no screen recorder). Reproduce any with python -m fx play <name> (see Quickstart).

_{solarsystem — an orrery: planets on elliptical orbits around a pulsing sun}

donut — the classic ASCII torus	fire — demoscene heat field	rain — Matrix digital rain

🌐 Explore the live showcase → — play with the effect engine, drive a menu with arrow keys, and poke the widgets, right in your browser.

Install

Primary — from PyPI:

pip install smartcli-toolkit

Distribution vs import name: the PyPI distribution is smartcli-toolkit (the names smartcli / smart-cli were taken or blocked), but the importable package is smartcli_core. So after pip install smartcli-toolkit you still write from smartcli_core import PtySession.

Alternative — reproduce the full dev environment from a source checkout:

git clone https://github.com/dwgx/SmartCLI SmartCLI
cd SmartCLI
python -m pip install -r requirements.txt

requirements.txt installs pyte, the MCP SDK, and pywinpty on Windows only (POSIX uses the stdlib pty backend). pip install . installs smartcli_core plus the smartcli-tui, smartcli-mcp, and smartcli-toolkit commands. The visual cmd-art and tui-ui skills still run in place from a checkout via python -m fx and python -m ui.

Optional extras (real FIGlet fonts, raster images, authoritative cell widths — all degrade gracefully to stdlib fallbacks when absent):

python -m pip install -r requirements-optional.txt
# or, from the checkout, via pyproject extras:
pip install ".[all]"        # pyfiglet + Pillow + wcwidth
pip install ".[art]"        # pyfiglet only
pip install ".[image]"      # Pillow only  (also: the PNG screenshot harness needs it)
pip install ".[width]"      # wcwidth only

Windows note: set UTF-8 output before running any skill so box-drawing and CJK glyphs encode cleanly (the CLIs also auto-reconfigure stdout, but set this to be safe):

set PYTHONIOENCODING=utf-8

Verified dep versions on the dev box (Windows 11, CPython 3.14.6): pyte 0.8.2, pywinpty 3.0.5, pyfiglet 1.0.4, Pillow 12.2.0, wcwidth 0.8.1.

Diagnostics. python -m smartcli_core prints your OS, Python, terminal, PTY backend, and dependency versions. smartcli-tui doctor reports where the core was loaded from and whether drive dependencies are present. Include both outputs when filing a terminal-sensitive bug.

Quickstart

cmd-art — terminal visual effects

cd skills/cmd-art
python -m fx list                          # list all 30 effects
python -m fx play donut --seconds 5        # play one effect (bounded)
python -m fx gallery                       # one frame of each effect
python -m fx show --seq "donut:fire:3,plasma::3"

tui-ui — cell-accurate terminal UI

cd skills/tui-ui
python -m ui widgets                       # list all 17 widgets
python -m ui gallery --width 100 --height 30
python -m ui demo table --width 80 --height 12 --theme dashboard

drive-tui — perceive & drive interactive programs

Persistent-session CLI (state survives across shell calls):

smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24
smartcli-tui wait-regex --id <SID> ">>> " --timeout-ms 15000
smartcli-tui send-line --id <SID> "print(6*7)"
smartcli-tui snapshot --id <SID>
smartcli-tui close --id <SID>

On Windows use py -i -q as the child command. From a source checkout, replace smartcli-tui with python skills/drive-tui/scripts/tui.py.

Or drive from any MCP client — the same verbs as MCP tools, with the per-session token attached automatically:

pip install smartcli-toolkit
smartcli-mcp     # stdio MCP server; smartcli-toolkit is an equivalent alias

As a library

The shared core is importable directly:

import sys
from smartcli_core import PtySession

s = PtySession()
s.start([sys.executable, "-q"])
s.wait_for(r">>> ")            # readiness sync, never a blind sleep
print(s.snapshot().to_text())  # pyte-backed structured screen
s.close()

For the full command reference, the screenshot/AGENTCLI harnesses, and the regression suite, see README-USAGE.md.

Features

cmd-art (skills/cmd-art) — a "living-template" effect engine: an Effect ABC + @register decorator + auto-discovery. 30 effects (donut, solarsystem, fire, plasma, rain, starfield, tunnel, text3d, cube, sphere, boids, life, fireworks, sparkle, decrypt, gradient_text, banner_scroll, image2ascii, typewriter, julia, mandelbrot, perlin, flames, water, nebula, text_flyin, text_converge, text_decrypt, spectrum_bars, cbonsai) across 8 themes (mono, fire, ocean, synthwave, viridis, pastel, matrix-green, rainbow). Effects are pure frame producers; play is bounded by default and always restores the terminal.

tui-ui (skills/tui-ui) — a web-like terminal layout engine emitting tmux-safe ANSI frames (SGR color runs + newlines only; no cursor moves, no alt-screen). 17 widgets (badge, banner, braille_chart, card, fuzzy_filter_list, gradient_rule, kv, meter, panel, preview_pane, progress, radial_glow, rule, slider_track, table, tabs, tree) over a real engine: field.py (shader compositors), raster.py (sub-cell half/quad/braille pixels), box_junction.py (edge-algebra box joins), color_model.py (honest truecolor → 256 → 16 → mono degrade). Display-cell accurate for CJK/emoji/ZWJ so columns never desync.

drive-tui (skills/drive-tui) — drives interactive terminal programs (REPLs, menus, pagers, y/N prompts, wizards) through a PTY via a perceive → decide → act → wait → confirm loop, never a blind sleep. A thin CLI (scripts/tui.py) offers a persistent detached session and a one-shot run mode, with an importable pattern library of 8 recipes (repl, menu_select, pager, search_filter, confirm, form, progress, wizard) that classify() a screen and drive() it.

Shared core (smartcli_core) — the pluggable PTY backend + pyte screen model + semantic snapshot + readiness sync (pty_backend / screen_model / snapshot / readiness / session). The reusable, importable foundation under all three skills.

Knowledge graph (knowledge/) — a wiki-link graph (140+ .md files) of exact rendering formulas, ANSI sequences, and measured constants, each note carrying a source and cross-links. See knowledge/INDEX.md.

Project layout

SmartCLI/
  smartcli_core/           shared PTY + pyte engine (importable package)
  skills/cmd-art/          fx effect package and CLI (30 effects, 8 themes)
  skills/drive-tui/        TUI pattern library and PTY driver CLI (8 recipes)
  skills/tui-ui/           terminal UI layout engine and widgets (17 widgets)
  tools/screenshot/        pyte -> PNG smoke-test harness
  tools/agentcli/          agent-CLI control validation harness
  knowledge/               wiki-link knowledge graph, 140+ .md files (see knowledge/INDEX.md)
  showcase/                rendered effect PNGs + demo GIFs (shown above)
  tests/                   direct script-style regressions
  research/                archived first-pass research notes

Documentation

• README-USAGE.md — the full usage cheat-sheet: every skill, the screenshot and AGENTCLI harnesses, and the regression commands.
• knowledge/INDEX.md — the knowledge graph (140+ .md files).
• AGENTCLI-VALIDATION.md — agent-CLI control test matrix.
• CHANGELOG.md — release history.

License

MIT — see LICENSE.