off-camber-open-frame 0.1.1

Open-source recognition and action for anything on a display.

Open Frame

Open-source recognition and action for anything on a display.

Open Frame is a platform for seeing what's on screen, locating targets, interacting with them, and verifying the result — without brittle selectors or vendor lock-in. Built under the Off-Camber identity.

Open Frame is an independent project and is not affiliated with or endorsed by any third-party automation vendor.

Status

Active development. v0.1.0 is live on PyPI as off-camber-open-frame.

MVP goalpost: a declarative flow that automates Outlook → Microsoft 365 → create document → email, repeatably across desktop apps. See docs/MVP_GOALPOST.md.

Documentation

Doc	Purpose
Vision	What we're building and for whom
Architecture	System layers, modules, and interfaces
Action plan	Phased tasks and checklists
Decisions	Record of key design choices
Capture manual test	Live macOS validation checklist for Phase 1 capture
OCR setup	Install and run the Tesseract recognizer
Accessibility setup	Configure macOS Accessibility permission for AX recognition
Act setup	Install and safely run click/type actions
Act manual test	End-to-end capture → find → click validation steps
Verify setup	Configure and use click verification with run artifacts
Flow setup	Define and run YAML flows
Acceptance runs	Record three consecutive MVP flow runs
API	Programmatic Session SDK usage and extension points
Contributing	Development workflow and macOS permission setup
Publish to PyPI	Trusted publishing setup and release checklist
v0.1.1 backlog	Short-term stabilization priorities after first release

Quick start (capture)

# Lower-level primitive available now
open-frame capture --out screen.png
open-frame capture --out crop.png --x 100 --y 200 --width 800 --height 500

# Capture active or named windows
open-frame capture --out active.png --window-title "Outlook"
open-frame capture --out active.png --window-id 123

# Inspect visible windows (and displays)
open-frame list-windows
open-frame list-windows --json --displays

# OCR find on a frame
open-frame find "Submit" --frame screen.png --json
open-frame find "Submit" --frame screen.png --overlay-out overlay.png --json

# Find and click
open-frame click "Submit" --dry-run --json
open-frame click "Save" --verify 'text-gone:"Save"' --json

# Run a flow file
open-frame run flow.yaml --dry-run --json

# Phase 5 smoke flow
open-frame run examples/flows/outlook-new-email/flow.yaml --dry-run --json
open-frame run examples/flows/outlook-browser-outlook/flow.yaml --dry-run --json
open-frame run examples/flows/outlook-m365-email/flow.yaml --dry-run --json

Planned next commands

# Run the MVP reference flow (prepared desktop: Outlook + browser signed in)
open-frame run examples/flows/outlook-m365-email/flow.yaml

open-frame find "New Email" --click

License

Apache License 2.0 — free to use, modify, and self-host. See LICENSE.