shinylive-check

Smoke-check an exported Shinylive site in headless Chromium: fail CI on boot failures, console errors and unrendered outputs.

An independent project, not affiliated with or endorsed by Posit.

$ shinylive-check site
OK: 1 output rendered in 5.7s at http://127.0.0.1:54143/

And when the export is broken, the check fails with the reason, not a spinner:

$ shinylive-check site
FAIL (error): 2 console/page error(s) at http://127.0.0.1:54318/
  preload error:Traceback (most recent call last):
  File "<exec>", line 239, in _start_app
  File "<exec>", line 121, in _install_requirements_from_dir
  ...
ValueError: Can't fetch metadata for 'shinylive-check-nonexistent-package-fixture'. Please make sure you have entered a correct package name and correctly specified index_urls (if you changed them).
$ echo $?
1

Why

shinylive export turns a Shiny app into static files, and static files are easy to deploy: GitHub Pages, S3, any web server. That is the appeal, and it is also the trap. A static host can only fail by not serving files, so every check you already have reports success no matter what: the deploy job is green, curl gets a 200, the uptime monitor is happy. Everything that can actually go wrong happens later, inside the visitor's browser, after pyodide boots:

• a requirements.txt entry micropip cannot resolve (a typo, a package with no wasm-compatible wheel);
• a Python exception while the app starts (an import that only breaks under pyodide, a file the export did not include);
• a Shiny output that renders as an error;
• a JavaScript error from an asset or an extension package.

Any of these leaves the page a permanent spinner while every server-side signal says the deploy worked. The only observer that can tell the difference is a real browser, so shinylive-check boots one: it serves the export (or probes the deployed URL), opens it in headless Chromium, waits for the app to render, and exits non-zero with the actual traceback when it does not.

Install

uv tool install shinylive-check   # or: pip install shinylive-check
playwright install chromium       # once, the browser it drives

Requires Python 3.10+. The only dependency is playwright.

Use

The target is either an exported site directory, served on a loopback port for the duration of the check, or the http(s) URL of a site that is already deployed:

shinylive-check site
shinylive-check https://example.github.io/my-app/

Passing means: the page loaded, no console or page error occurred, no Shiny output is in an error state, and at least one Shiny output rendered non-empty content. After the first successful render the page gets half a second to settle, so an error thrown right after the render still fails the check instead of racing it.

• --timeout SECONDS (default 180): how long the app gets to render. A cold pyodide boot loads tens of megabytes of wasm, so the default is generous; a warm CI cache usually finishes in seconds.
• --expect SELECTOR (repeatable): CSS selectors that must become visible, replacing the default at-least-one-rendered-output condition. For apps whose UI is inputs only, or when "booted" means something specific: --expect "#sales" --expect ".value-box".
• --screenshot PATH: write a full-page screenshot when the check fails, for the CI artifact that shows what the visitor would have seen.
• --port PORT: serve a directory target on a fixed port instead of an ephemeral one, for exports that reference themselves by absolute URL.
• -o, --output auto|text|json (default auto): the verdict format on stdout. auto emits JSON when stdout is not a TTY, the human-readable form otherwise.

Exit codes:

In CI

The check slots between export and deploy, so a broken export never replaces a working site:

- name: Export
  run: uvx shinylive export app site

- name: Check the export boots before it deploys
  run: |
    pip install shinylive-check
    playwright install --with-deps chromium
    shinylive-check site --screenshot boot-failure.png

- name: Keep the evidence when it fails
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: boot-failure
    path: boot-failure.png

The same command probes the live site after the deploy, or from a scheduled workflow:

shinylive-check https://example.github.io/my-app/

For scripts and agents

The CLI is built for machine consumption as well as terminals. Piped, the verdict is JSON:

$ shinylive-check site | jq .
{
  "target": "site",
  "ok": true,
  "url": "http://127.0.0.1:54535/",
  "seconds": 4.7,
  "outputs": 1,
  "errors": [],
  "output_errors": [],
  "expects": [],
  "failure": null,
  "screenshot": null
}

failure is null on a pass and one of error, output-error, timeout, navigation or http-status otherwise. Exit code 1 is a declared outcome, not an error: the check ran and the verdict is on stdout, like grep finding nothing. Real errors (a wrong invocation, a missing browser) go to stderr as a one-line JSON envelope, {"error": {"kind": ..., "message": ..., "hint": ...}}, when the output format is JSON.

shinylive-check schema prints the whole contract, commands, arguments, output fields, error kinds and exit codes, as machine-readable JSON in The CLI Spec v0.3 dialect.

How it works

• A directory target is served on 127.0.0.1 by a quiet HTTP server for the duration of the check; a URL target is loaded as-is. An HTTP status of 400 or above fails the check immediately.
• Headless Chromium loads the page with uncaught page errors and console errors collected from the first byte.
• Every frame is scanned, not just the main one, because Shinylive apps often run inside an iframe; a selector evaluated only on the top document would silently match nothing.
• A Shiny output counts as rendered when it has non-empty content and is not in an error state; outputs carrying shiny-output-error fail the check with the error text.
• The page is polled twice a second until the verdict is clear one way or the other, or the timeout passes.
• The screenshot, when requested, is captured only on failure and never masks the verdict.

Limitations

• The default pass condition needs at least one Shiny output. An app whose UI is entirely inputs or static content never satisfies it; give --expect a selector that means "booted" for that app.
• It judges the boot, not the app's behavior. It does not click through the UI; for that, write Playwright tests against your app.
• Packages that are not bundled into the export are fetched by micropip at boot, exactly as they are for a real visitor, so checking such an export needs network access.
• Chromium only, by design: one engine, deterministic in CI.

When the app outgrows a static export

Shinylive is a great fit for demos, docs and small tools, and its constraints are the flip side of the free hosting: the entire app ships to every visitor, so there are no secrets, no private data, no database connections and no server-side compute, and every package must have a wasm-compatible wheel. When an app crosses that line and needs a real server process, ShinyHub is a self-hosted platform for exactly that: deploy Shiny apps (Python and R) to your own server with one command, with OAuth/OIDC login, per-app access control, idle hibernation and scaling.

Related: shiny-plotly renders plotly figures in Shiny for Python without the shinywidgets layer, and runs under Shinylive; its Pages demo is deployed behind exactly this kind of pre-deploy boot check.

Development

make sync         # uv sync --all-groups
make browsers     # playwright install chromium, once
make check        # lint, typecheck, CLI tests, browser tests, wheel check, floor check
make schema-check # score the schema against The CLI Spec (needs clispec on PATH)

make test runs the CLI contract without a browser: usage errors, exit codes, the JSON envelope, the schema command. make test-browser exports three fixture apps with a pinned shinylive and drives the CLI against them in headless Chromium: the green one passes, and each broken one (an exception at boot, an unresolvable package) fails for its own reason, so the failure detection is proven against real exports rather than assumed. make check-wheel installs the built wheel into a throwaway venv and tests that, so what ships is what was tested. make check-floor does the same with playwright at the oldest version pyproject.toml allows.

License

MIT. See LICENSE.