fsq-agent 0.1.2

Evidence-first AI UI automation you can inspect, replay, and verify.

Goal → Evidence → Candidate YAML → Strict Replay → Runs report → Portable HTML

Quickstart · 中文 · Coding agents · How it works · Platforms · Documentation · Contributing

FSQ turns a natural-language UI goal into an observable automation run, saves screenshots, UI snapshots, events, and reports as evidence, and can turn successful actions into a reviewable Case for deterministic replay. It uses Playwright, uiautomator2, pywinauto, and Appium as platform backends; it does not replace or install their host prerequisites.

Run existing Cases without an LLM

Use FSQ like a test harness once a Case exists. Run history and offline reports do not require a configured LLM Provider. Strict replay is also provider-free unless the authored Case contains an AI assertion.

python -m pip install fsq-agent
fsq init --platform web --browser-channel chrome
# Store reviewed Case assets in your repo, for example: cases/web/*.fsq.yaml
fsq case test --platform web cases/web/YOUR_CASE.fsq.yaml
fsq runs show RUN_ID --open

Create Cases with AI

Configure an LLM Provider, then use fsq case create --platform web --goal "..." when you want FSQ to operate the real UI and create a reviewable .fsq.yaml Case from the successful Run. Coding agents should provide the goal and context; FSQ proves the path through live execution and evidence.

See FSQ in action

Watch the full v0.1.0 demo after the 20-second current Control Plane tour above.

https://github.com/user-attachments/assets/aa9d0a12-2f93-4894-8349-52a013424939

Watch the full demo on YouTube

Why FSQ

• Inspect the facts. Every run keeps screenshots, normalized UI snapshots, ordered events, metadata, and reports together.
• Separate exploration from regression. AI can explore a goal; reviewed YAML Cases replay authored actions deterministically.
• Use one workflow across UI surfaces. Web, Android, Windows, and macOS share the same Case, evidence, Run, and readiness concepts.
• Keep control local. Workspaces, evidence, Provider configuration, and the Control Plane are local by default.

FSQ complements platform automation libraries. Playwright, uiautomator2, pywinauto, and Appium perform platform interaction; FSQ adds goal-driven execution, a shared Case format, evidence capture, verification, Run history, and a local Control Plane.

Product tour

Describe a goal	Inspect evidence	Review a candidate Case

See the remaining approved screenshots in release media.

Five-minute quickstart

This public Web example uses TodoMVC, requires an installed Chromium-family browser, and writes all project data locally. Steps 1-3 exercise the provider-free harness path. A configured Provider is only needed for AI-driven Case creation or post-run suggestions.

1. Install

python -m pip install fsq-agent

The base package includes Python dependencies for all four supported platforms. Browsers, applications, devices, ADB, and Appium services remain system prerequisites. FSQ never installs them during init.

2. Create an empty Workspace

mkdir fsq-web-demo
cd fsq-web-demo
fsq init --platform web --browser-channel chrome
fsq doctor

Workspace root selection is exact:

• In an empty current directory, fsq init adopts that directory as the Workspace root.
• In a non-empty current directory, it creates an absent <current-directory>/<workspace-name> child. Use --name NAME to choose that name, then change into the child directory for Workspace commands.
• Other CLI commands never search parent directories; run them from the exact registered Workspace root.

3. Replay the public example without a planning LLM

Download the current examples/web/example-domain.fsq.yaml into the Workspace and run it:

mkdir -p cases/web
curl --fail --location --output cases/web/example-domain.fsq.yaml \
  https://raw.githubusercontent.com/microsoft/FSQ/main/examples/web/example-domain.fsq.yaml
fsq case test --platform web cases/web/example-domain.fsq.yaml
fsq runs list --platform web

4. Explore with AI

Configure one supported user-level Provider from any directory:

fsq providers configure github_copilot
fsq providers status

For direct official OpenAI access, run fsq providers configure openai and enter the API key at the hidden prompt, then select an offered GPT-5-or-later model. DeepSeek uses fsq providers configure deepseek with its official Responses API and an explicitly selected eligible Flash/Pro model. Google Gemini uses fsq providers configure google_gemini with a Google AI Studio API key and an explicitly selected stable Gemini 3-or-later Flash/Pro model. Kimi uses fsq providers configure kimi with an explicit China or Global region and an eligible K3-or-later model. Azure deployments use fsq providers configure azure_openai. The browser's Settings page offers the same six Providers; see the setup guide.

Then return to the Workspace:

fsq case create --platform web \
  --goal "Open https://example.com and verify the Example Domain heading is visible."

fsq case test --platform web --suggest cases/web/example-domain.fsq.yaml
fsq runs show RUN_ID --open

--suggest executes the source Case exactly once, then asks AI to analyze only the persisted Case, report, and evidence. Suggestions and an optional candidate Case remain inside that Run; the source Case is not modified.

5. Open the local Control Plane

fsq ui

The installed wheel includes the compiled frontend. It listens on 127.0.0.1:8879 by default and does not require Node.js at runtime.

Coding agent workflow

Coding agents should not guess UI action steps or hand-author final Case YAML from code context alone. They should understand the product change, provide a precise goal to FSQ, and let FSQ operate the real UI before a Case is reviewed and committed.

# 1. Ask FSQ to prove a goal against the live UI and record evidence.
fsq case create --platform web \
  --goal "Open https://example.com and verify the Example Domain heading is visible."

# 2. Inspect the generated Run and candidate Case.
fsq runs list --platform web
fsq runs show RUN_ID
fsq runs logs RUN_ID

# 3. Replay the reviewed generated Case deterministically before committing it.
fsq case test --platform web cases/web/RUN_ID.fsq.yaml

The durable asset is the reviewed .fsq.yaml Case. The proof lives in .fsq/runs/<platform>/<run-id>/ as events, screenshots, UI snapshots, evidence manifests, and reports. The strict replay path is provider-free, so CI and coding agents can verify committed Cases without configuring another LLM.

How FSQ works

Goal ──► AI exploration ──► evidence ──► verification ──► reviewable Case
                                                          │
Reviewed Case ──► deterministic replay ──► fresh evidence ─┘

Dynamic execution and deterministic replay share platform Harnesses and evidence contracts. The original execution result is immutable; later suggestion analysis cannot rewrite it or perform another UI execution. Implementation-level architecture and behavior are defined by the root and module SPEC.md files.

Supported platforms

All Python backend packages are installed with fsq-agent; platform applications and host services are not. Run fsq doctor from the exact Workspace root for actionable readiness results.

Platform target options for fsq init:

Runs and local data

<workspace-root>/
  .fsq/config/config.<platform>.yaml
  .fsq/runs/<platform>/<run-id>/
  cases/<platform>/
  knowledge/<platform>/

Use fsq runs list, fsq runs show RUN_ID, and fsq runs logs RUN_ID, or open historical Runs in the Control Plane. fsq runs show RUN_ID --open rebuilds an offline HTML report. fsq runs export RUN_ID --format json|junit|html|bundle creates a non-interactive export without a Provider or UI execution. Reports link failure facts, steps, metrics, screenshots, snapshot differences, and recorded replay provenance. See CI evidence and the public evidence demo. Evidence can contain visible application data; review it before sharing. Do not commit .fsq, credentials, reports, screenshots, or private target data.

Provider configuration is stored under ~/.fsq and shared by the CLI and local Control Plane. Supported Providers are OpenAI (official API), Azure OpenAI, DeepSeek (official Responses API), Google Gemini (Developer API), and GitHub Copilot. One Provider is active at a time; successful replacement removes inactive credentials. DeepSeek keys live in ~/.fsq/auth/deepseek.json and Gemini keys in ~/.fsq/auth/google-gemini.json; custom DeepSeek endpoints, Vertex AI, custom Gemini endpoints, and Preview/specialized Gemini models are not supported.

Documentation

Contributing

Contributions are welcome across documentation, Cases, platform Harnesses, evidence, verification, and developer experience. Start with CONTRIBUTING.md, follow the Code of Conduct, and report vulnerabilities privately through SECURITY.md.

License

MIT — Copyright (c) Microsoft Corporation.