fsq-agent 0.1.0a2

Evidence-first AI UI automation you can inspect, replay, and verify.

Quickstart · 中文 · How it works · Platforms · Documentation · Contributing

[!IMPORTANT] FSQ v0.1.0 is an alpha release. It is ready for evaluation and contribution, but public APIs and Case authoring details may evolve before 1.0. See support and stability.

FSQ turns a natural-language UI goal into an observable automation run, saves screenshots, UI snapshots, events, and reports as evidence, and can turn successful actions into a reviewable Case for deterministic replay. It uses Playwright, uiautomator2, pywinauto, and Appium as platform backends; it does not replace or install their host prerequisites.

https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939

Watch the full demo on YouTube

Why FSQ

• Inspect the facts. Every run keeps screenshots, normalized UI snapshots, ordered events, metadata, and reports together.
• Separate exploration from regression. AI can explore a goal; reviewed YAML Cases replay authored actions deterministically.
• Use one workflow across UI surfaces. Web, Android, Windows, and macOS share the same Case, evidence, Run, and readiness concepts.
• Keep control local. Workspaces, evidence, Provider configuration, and the Control Plane are local by default.

FSQ complements platform automation libraries. Playwright, uiautomator2, pywinauto, and Appium perform platform interaction; FSQ adds goal-driven execution, a shared Case format, evidence capture, verification, Run history, and a local Control Plane.

Product tour

Describe a goal	Inspect evidence	Review a candidate Case

See the remaining approved screenshots in release media.

Five-minute quickstart

This public Web example uses Example Domain, requires an installed Chromium-family browser, and writes all project data locally. AI-driven commands additionally require a configured Provider.

1. Install

python -m pip install fsq-agent

The base package includes Python dependencies for all four supported platforms. Browsers, applications, devices, ADB, and Appium services remain system prerequisites. FSQ never installs them during init.

2. Create an empty Workspace

mkdir fsq-web-demo
cd fsq-web-demo
fsq init --platform web --browser-channel chrome
fsq doctor

Workspace root selection is exact:

• In an empty current directory, fsq init adopts that directory as the Workspace root.
• In a non-empty current directory, it creates an absent <current-directory>/<workspace-name> child. Use --name NAME to choose that name, then change into the child directory for Workspace commands.
• Other CLI commands never search parent directories; run them from the exact registered Workspace root.

3. Replay the public example without a planning LLM

Download the versioned examples/web/example-domain.fsq.yaml into the Workspace and run it:

mkdir -p cases/web
curl --fail --location --output cases/web/example-domain.fsq.yaml \
  https://raw.githubusercontent.com/microsoft/FSQ/v0.1.0/examples/web/example-domain.fsq.yaml
fsq case test --platform web cases/web/example-domain.fsq.yaml
fsq runs list --platform web

4. Explore with AI

Configure one supported user-level Provider from any directory:

fsq providers configure github_copilot
fsq providers status

Then return to the Workspace:

fsq case create --platform web \
  --goal "Open https://example.com and verify the Example Domain heading is visible."

fsq case test --platform web --suggest cases/web/example-domain.fsq.yaml
fsq runs show RUN_ID --open

--suggest executes the source Case exactly once, then asks AI to analyze only the persisted Case, report, and evidence. Suggestions and an optional candidate Case remain inside that Run; the source Case is not modified.

5. Open the local Control Plane

fsq ui

The installed wheel includes the compiled frontend. It listens on 127.0.0.1:8879 by default and does not require Node.js at runtime.

How FSQ works

Goal ──► AI exploration ──► evidence ──► verification ──► reviewable Case
                                                          │
Reviewed Case ──► deterministic replay ──► fresh evidence ─┘

Dynamic execution and deterministic replay share platform Harnesses and evidence contracts. The original execution result is immutable; later suggestion analysis cannot rewrite it or perform another UI execution. See the architecture overview.

Supported platforms

All Python backend packages are installed with fsq-agent; platform applications and host services are not. Run fsq doctor from the exact Workspace root for actionable readiness results.

Platform target options for fsq init:

Runs and local data

<workspace-root>/
  .fsq/config/config.<platform>.yaml
  .fsq/runs/<platform>/<run-id>/
  cases/<platform>/
  knowledge/<platform>/

Use fsq runs list, fsq runs show RUN_ID, and fsq runs logs RUN_ID. fsq runs show RUN_ID --open creates an offline static HTML report without calling a Provider or operating the UI. Evidence can contain visible application data; review it before sharing. Do not commit .fsq, credentials, reports, screenshots, or private target data.

Provider configuration is stored under ~/.fsq and shared by the CLI and local Control Plane. Supported first-release Providers are GitHub Copilot and Azure OpenAI.

Documentation

Contributing

Contributions are welcome across documentation, Cases, platform Harnesses, evidence, verification, and developer experience. Start with CONTRIBUTING.md, follow the Code of Conduct, and report vulnerabilities privately through SECURITY.md.

License

MIT — Copyright (c) Microsoft Corporation.