Selfbrowse
A small browser agent. You describe a task in plain language; it drives a real browser to do it.
Instead of screenshots or raw HTML, the model reads each page as a short numbered list of what can be clicked or typed into, and Selfbrowse remembers what worked on each site so repeat visits need fewer model calls.
Status: early (0.1), not yet on PyPI. Install from source (below). Cost and speed claims are not yet benchmarked against other tools; see Benchmarks.
Install
git clone <your-repo-url> selfbrowse && cd selfbrowse
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
playwright install chromium # or use your installed Chrome with --channel chrome
Optional extras: pip install -e '.[stealth]' (Patchright), pip install -e '.[otel]' (OpenTelemetry export), pip install -e '.[mcp]' (MCP server), pip install -e '.[presidio]' (name and personal-data masking; also run python -m spacy download en_core_web_sm), pip install -e '.[bench]' (token benchmark).
Quick start
With Claude (set your key in your own shell; never paste it into a chat or commit it):
export ANTHROPIC_API_KEY=your-key-here
selfbrowse "Open news.ycombinator.com and tell me the title of the top story."
With a local, free model through Ollama (no key):
selfbrowse -m ollama:qwen3.6:latest "Open news.ycombinator.com and tell me the title of the top story."
Watch it work in a visible window:
selfbrowse --headed "Open news.ycombinator.com and tell me the title of the top story."
Each step prints as it runs. The last lines are the answer and a token summary.
If you cannot see the output, save it to a file and open that:
selfbrowse "your task" > result.txt 2>&1
Python
from selfbrowse import BrowseAgent
result = BrowseAgent("find the top story on Hacker News", llm="anthropic:claude-haiku-4-5").run()
print(result.output)
print(result.usage) # input/output/cache tokens and number of model calls
result has success, output, steps, usage, history and error.
Fourteen runnable examples (local models, secrets, guardrails, name masking, hand-off, real Chrome, tracing, learning, and two that need no key) are in examples/.
Command line reference
selfbrowse [options] "task"
selfbrowse sites list sites it has learned
selfbrowse forget <domain> delete what it learned about a site
selfbrowse trace list list recorded runs
selfbrowse trace show <id|latest> [--full] timeline of a run (--full adds prompts)
| Option | Meaning |
|---|---|
-m, --model |
provider:model (default anthropic:claude-haiku-4-5, or $SELFBROWSE_MODEL), or your own model as module:Name or ./file.py:Name (see Bring your own model) |
--headed |
show the browser window |
--max-steps N |
stop after N tool calls (default 100) |
--channel chrome|msedge |
use an installed browser instead of bundled Chromium |
--profile DIR |
keep cookies and logins in this folder between runs |
--stealth |
use Patchright to reduce common automation signals |
--page-text N |
also send up to N characters of on-screen text with every page list (more tokens, fewer read calls) |
--max-output-tokens N |
cap on each model reply (default 2048); raise it for reasoning models that think at length |
--vision |
let the model request screenshots (more tokens) |
--screenshots DIR |
save a screenshot after every step, plus an index.html, in a new folder under DIR, as evidence of what the agent did (see Screenshots as evidence) |
--permissions LIST |
grant browser permissions to every site, e.g. notifications,geolocation (default: none) |
--dialogs auto|accept|dismiss |
what to do with JavaScript alert/confirm/prompt (default auto) |
--no-handoff |
never pause for a person (CAPTCHA, login, 2FA) |
--trace |
record the run to ~/.selfbrowse/traces/ |
--show-prompts |
print every model request to stderr |
--trace-hashes |
trace sizes and hashes instead of prompt text |
--otel URL / --otel-content |
export spans to an OTLP endpoint (needs the otel extra) |
--allow-domain D / --block-domain D |
only visit / never visit these sites (repeatable) |
--confirm-risky |
ask before buy, pay, delete, send, sign-out clicks |
--redact-pii |
hide emails, phones, SSNs, cards from the model: it sees <EMAIL_1>, the real value is typed back for it, and your answer shows the real values (see Keeping personal data from the model) |
--redact-presidio |
also mask names and other personal data with Microsoft Presidio (needs the presidio extra; see Names and other personal data) |
--presidio-entities LIST |
which Presidio entities to mask, e.g. PERSON,EMAIL_ADDRESS,LOCATION |
--presidio-model NAME |
spaCy model for Presidio (default en_core_web_sm; en_core_web_lg finds more names) |
--no-restore |
with a redaction flag: mask for good as [email]. Nothing keeps the value, the model cannot type it, and the answer stays masked |
--watch-injection |
warn on instruction-like page text and confirm risky actions |
--max-tokens N / --max-seconds S |
stop a run that goes over |
--no-learn |
do not read or write site knowledge |
--json |
print the full result as JSON |
-q, --quiet |
do not print each step |
--version |
print the version |
Environment variables: ANTHROPIC_API_KEY, OPENAI_API_KEY, GROQ_API_KEY, GEMINI_API_KEY, DEEPSEEK_API_KEY, OPENROUTER_API_KEY, TOGETHER_API_KEY, SELFBROWSE_MODEL, SELFBROWSE_HOME (data folder, default ~/.selfbrowse).
What the agent can do
goto, click, type, select (native dropdowns), press, scroll, back, wait, read (text on screen), find (jump to a word on a long page), switch_tab, close_tab, remember, ask_human and done.
- Dropdowns: a native
<select>is listed with its current value and options, andselectpicks one by its text or value. - Frames and web components: controls inside iframes (same-origin and cross-origin) and open shadow roots appear in the same numbered list and are clicked the same way.
- Tabs: when a link or popup opens a new tab the agent follows it, and the page list starts with
[tabs: 1 "Shop" · 2 "Details" (current)]so the model knows. Not supported yet: closed shadow roots, drag and drop, file uploads, custom (non-<select>) dropdowns beyond clicking them. See ROADMAP.md.
Each tool schema is sent on every model call, so tools are kept few and short (about 790 tokens for all of them on a typical step).
Use it from Claude, VS Code and other MCP clients
Selfbrowse can run as an MCP server, so an assistant can use it as its browser:
pip install 'selfbrowse[mcp]'
playwright install chromium
It offers two kinds of tools:
browse(task)(task mode). Selfbrowse runs its own model loop and returns only the answer, so the assistant never reads the pages. This is where most of the token saving is. It needs--model, for exampleollama:qwen3.6(free, local) oranthropic:claude-haiku-4-5.browser_snapshot,browser_click,browser_type,browser_select,browser_goto,browser_read,browser_find,browser_scroll,browser_press,browser_back,browser_switch_tab,browser_close_tab(step mode). The assistant drives the browser and reads Selfbrowse's short numbered list instead of a screenshot. No model is needed on the server. We have not measured how this compares with other browser tools.
VS Code (Copilot agent mode): add to .vscode/mcp.json:
{
"servers": {
"selfbrowse": {
"command": "selfbrowse-mcp",
"args": ["--model", "ollama:qwen3.6", "--headed", "--channel", "chrome", "--allow-domain", "example.com"]
}
}
}
Claude Code: claude mcp add selfbrowse -- selfbrowse-mcp --model ollama:qwen3.6 --headed
Claude Desktop (claude_desktop_config.json): {"mcpServers": {"selfbrowse": {"command": "selfbrowse-mcp", "args": ["--model", "ollama:qwen3.6"]}}}
Client config formats change between versions, so check your client's MCP docs if one of these is rejected.
Things to know:
- Guardrails apply. Use
--allow-domain,--block-domain,--block-risky,--redact-pii,--redact-presidio(with--presidio-entitiesand--presidio-model),--watch-injection,--max-tokensand--max-seconds. There is nobody to ask in a terminal, so a risky click is refused, not confirmed; your client's own tool-approval prompt is a second check. - No terminal hand-off. Stdout carries the protocol. When a CAPTCHA or login needs a person, the tool result says so; start the server with
--headedso there is a window to finish it in. - Secrets: set
SELFBROWSE_SECRET_<name>in the server's environment; the assistant writes<secret:name>and never sees the value. - One browser per server. Task mode and step mode share it, so a
browserun starts from the page your step tools left open. - Only
mcp2.x was tested. Selfbrowse also tries the 1.x import, but that path is untested.
Models
Use provider:model:
| Provider | Example |
|---|---|
anthropic |
anthropic:claude-haiku-4-5 (prompt caching on) |
openai |
openai:gpt-4o-mini |
ollama |
ollama:qwen3.6:latest (local; the model must support tool calling) |
lmstudio |
lmstudio:<model> |
groq, deepseek, gemini, openrouter, together |
groq:llama-3.3-70b-versatile |
Your own provider (Azure OpenAI, AWS Bedrock, an internal gateway) works too; see below.
Bring your own model
Any provider works if you give Selfbrowse an object with a model name and a complete(system=, messages=, tools=, max_tokens=) method that returns an LLMResponse. Messages and tools arrive as Anthropic-style blocks, so you translate to and from your SDK.
If your SDK speaks the OpenAI chat format (Azure OpenAI, and other OpenAI-compatible services), reuse the built-in translation:
# my_llm.py
from openai import AzureOpenAI
from selfbrowse.llm.openai_compat import messages_to_openai, tools_to_openai, response_from_openai
class AzureLLM:
model = "my-gpt4o-deployment" # shown in traces
def __init__(self):
self.client = AzureOpenAI(azure_endpoint="https://RESOURCE.openai.azure.com",
api_key="...", api_version="2024-10-21")
def complete(self, *, system, messages, tools, max_tokens=2048):
resp = self.client.chat.completions.create(
model=self.model, max_tokens=max_tokens,
messages=messages_to_openai(system, messages),
tools=tools_to_openai(tools),
)
return response_from_openai(resp)
Use it from Python, the command line or the MCP server:
BrowseAgent("task", llm=AzureLLM()).run() # an object
BrowseAgent("task", llm="my_llm:AzureLLM").run() # the same, by name
selfbrowse --model my_llm:AzureLLM "task" # a module on your path, or in the folder you run from
selfbrowse --model ./my_llm.py:AzureLLM "task" # or a file
selfbrowse-mcp --model ./my_llm.py:AzureLLM
The name after the colon can be a class (created with no arguments), a function that returns a model, or a ready object. Any provider: that is not built in is looked up this way, so a mistyped provider says so instead of failing quietly.
For a service with its own format (AWS Bedrock's Converse API, Vertex, an internal gateway), build the LLMResponse yourself: content is a list of {"type": "text", "text": ...} and {"type": "tool_use", "id", "name", "input"} blocks, stop_reason is "tool_use" or "end_turn", and usage is a selfbrowse.types.Usage with your token counts so traces and budgets work. selfbrowse/llm/base.py has the full interface.
These routes are tested with a fake model. They have not been run against live Azure or AWS accounts.
Seeing what is sent to the model
selfbrowse --show-prompts "task" # prints every model request live (stderr)
selfbrowse --trace "task" # also saves the run to ~/.selfbrowse/traces/<run-id>.jsonl
selfbrowse trace list
selfbrowse trace show latest # timeline: model calls, tokens, actions, guardrail decisions
selfbrowse trace show latest --full # include the full prompts
A trace is one JSON line per event: run_start, llm_request (the exact system text, tool names and messages sent, with token estimates per part), llm_response (usage, timing, the model's reply), tool_call, tool_result, replay, guardrail, handoff, run_end.
- Secrets passed with
sensitive=never appear: the model only sees<secret:NAME>. - Prompts include page text, which may hold personal data.
--trace-hashes(Python:trace_content=False) stores sizes and hashes only, andredact_trace=fnscrubs every string before it is written. Task text, typed text, tool results, reasons and URLs are covered too. - Tracing never breaks a run: a failing sink or unwritable folder is ignored.
From Python:
agent = BrowseAgent("task", trace=True, on_llm_request=lambda system, messages, tools, n: print(n, len(messages)))
agent.run()
print(agent.trace_path)
OpenTelemetry
pip install -e '.[otel]', then point it at any OTLP/HTTP receiver, including one on your own machine (Jaeger, Grafana Tempo, Arize Phoenix on http://localhost:4318):
selfbrowse --otel http://localhost:4318 "task"
Each run becomes a selfbrowse.run span with child spans per model call (gen_ai.* attributes, token usage) and per tool call; guardrail decisions and hand-offs are span events. Spans carry sizes and timings only. Add --otel-content to attach what the model was sent and what it answered: each chat span gets gen_ai.system_instructions, gen_ai.input.messages and gen_ai.output.messages (OpenTelemetry GenAI conventions, still experimental). LLM viewers such as Phoenix, Langfuse and Opik show these as a conversation; Jaeger shows them as JSON strings under the span's Tags. Long text is shortened part by part so the JSON stays valid. These attributes contain page text, so only send them to a backend you trust. Spans go only where you point them.
Screenshots as evidence
To keep proof of what the agent saw and did, save a picture after every step:
selfbrowse --screenshots ./evidence "Add the cheapest USB-C cable to the cart on example.com"
# [screenshots: evidence/run-20261110-093015]
Each run gets its own folder with step-01.jpg, step-02.jpg and so on, and an index.html that lists every step (tool, arguments, URL, result) next to its picture. Python: BrowseAgent("task", screenshots="./evidence"), then agent.screenshots_dir; each Step has a .screenshot file name, and a trace records a screenshot event per image.
- The picture shows the page after the step ran, including the last page when the agent calls
done. - The model never gets these images. This is separate from
--vision, which sends pictures to the model and costs tokens. - They are not redacted. A screenshot shows the page as it really is: names, emails and anything the agent typed, even a password that is hidden everywhere else. The folder is created private to your user (mode 700). Do not share or commit it without looking, and avoid
--screenshotson pages with data you would not want stored. - A picture that cannot be taken (the page just closed, or a pop-up is blocking capture) leaves a gap, marked in
index.html; the run carries on. - The MCP server does not save screenshots.
Guardrails
Rules between the model and the browser. Pass them as a list:
from selfbrowse import BrowseAgent
from selfbrowse.guardrails import AllowedDomains, ConfirmRisky, RedactPII, PresidioRedact, InjectionWatch, Budget
BrowseAgent(
"task",
guardrails=[
AllowedDomains(["example.com"]), # also allows subdomains
ConfirmRisky(), # asks before Buy / Pay / Delete / Send / Sign out
RedactPII(), # hides emails, phones, SSNs, cards behind <EMAIL_1>-style placeholders
PresidioRedact(), # optional: also names, via Microsoft Presidio (needs selfbrowse[presidio])
InjectionWatch(), # warns on "ignore previous instructions"-style page text
Budget(max_tokens=50_000, max_seconds=300),
],
).run()
| Guardrail | What it does |
|---|---|
SafeUrls |
http(s) pages only; blocks file:, chrome:, data:, javascript:. Always on unless safe_urls=False |
AllowedDomains([...]) |
only these sites (judged on the real hostname, so https://good.com@evil.com is evil.com). Also undoes a click that lands elsewhere, and refuses a start page outside the list |
BlockedDomains([...]) |
never these sites |
ConfirmRisky() |
asks a person before a click whose label looks irreversible; with nobody to ask (headless) it blocks instead |
RedactPII(kinds=..., restore=True) |
hides emails, phone numbers, US SSNs and Luhn-valid card numbers behind placeholders like <EMAIL_1>; the real value is typed back on the same site. Names and addresses are not detected. restore=False masks for good as [email] |
PresidioRedact(entities=, threshold=, model=) |
optional. Uses Microsoft Presidio and a spaCy model to mask names as well as emails, phones, cards, SSNs, IBANs, passports and similar. Runs locally. Install with pip install 'selfbrowse[presidio]'; see Names and other personal data |
InjectionWatch() |
flags instruction-like page text, tells the model to treat it as data, and requires confirmation before navigating, typing or risky clicks from that page |
Budget(max_tokens=, max_seconds=, max_usd=, price=(in, out)) |
stops a run that goes over; max_usd needs the per-million-token prices you pay, and counts cached input at full price, so it is a ceiling |
Confirmations use the same prompt as a hand-off (--headed, press Enter to approve or n to deny). From Python, your handoff= function receives request.kind == "confirm"; return True to allow the action.
Your own rule is a small class; override only the hooks you need:
from selfbrowse.guardrails import Guardrail, Block
class NoSocial(Guardrail):
def check_url(self, url): # also: on_page, before_action, after_step
if "facebook.com" in url:
return Block("social sites are off limits")
Names and other personal data (Presidio)
RedactPII is regex-only: it cannot see names or addresses. For those, turn on Microsoft Presidio, which uses a small language model to find personal data in page text. It runs on your machine; nothing is sent to Microsoft or anyone else. It is optional, so the default install stays small.
pip install 'selfbrowse[presidio]'
python -m spacy download en_core_web_sm # one-time model download (about 12 MB)
selfbrowse --redact-presidio "Open my account page and tell me the order status"
If either piece is missing, Selfbrowse stops before the run and prints the command to fix it. It never downloads a model by itself.
Choose what to mask with Presidio's entity names, and trade speed for accuracy with a bigger model:
selfbrowse --redact-presidio --presidio-entities PERSON,EMAIL_ADDRESS,LOCATION "task"
selfbrowse --redact-presidio --presidio-model en_core_web_lg "task" # several hundred MB, slower, finds more names
from selfbrowse.guardrails import PresidioRedact
PresidioRedact() # defaults below
PresidioRedact(["PERSON", "LOCATION"]) # only these
PresidioRedact(threshold=0.7) # fewer false positives, more misses
PresidioRedact(model="en_core_web_lg")
PresidioRedact(analyzer=my_analyzer_engine) # your own AnalyzerEngine: custom recognizers, other languages
- Default entities:
PERSON,EMAIL_ADDRESS,PHONE_NUMBER,CREDIT_CARD,US_SSN,IBAN_CODE,US_PASSPORT,US_DRIVER_LICENSE,US_BANK_NUMBER,MEDICAL_LICENSE,CRYPTO.LOCATIONandDATE_TIMEare off because page text is full of both;IP_ADDRESSis off because it would mask the address of a local page in the URL line. Add any of them by naming them. All Presidio entity names. - What the model sees: a placeholder in place of the value:
Hello <PERSON_1>, your email is <EMAIL_1>(with--no-restore, a label:[person],[email]). The model cannot read a hidden value, so a task like "find the contact email" needsEMAIL_ADDRESSleft out of the list. - Use it with
--redact-piior alone. Both can be on; the regex layer runs first, then Presidio. - Accuracy: statistical name detection has false positives (a product called "Max Pro" may be masked) and misses, more so with the small model and on short fragments. It lowers exposure; it does not prove nothing leaked. Measure it on your own pages before relying on it.
- Cost:
en_core_web_smloads once per run (about 0.8 s on one laptop). After that a short page takes a few milliseconds and about 9,000 characters of page text took about 0.2 s. Those figures come from one machine and one synthetic page, so treat them as a rough guide, anden_core_web_lgwill be slower. - Scope: it masks the page text sent to the model. It does not scrub traces, screenshots (
--vision) or text you type; useredact_trace=for traces and avoid--visionwith sensitive pages. - MCP server:
selfbrowse-mcptakes the same--redact-presidio,--presidio-entitiesand--presidio-modelflags and loads the model once at start-up.
Keeping personal data from the model
Hiding a value has a cost: the model cannot use what it cannot see. With [email] on the page it cannot fill a form with that email. So --redact-pii and --redact-presidio do not just delete the value. Page text gets stable placeholders, <EMAIL_1>, <PERSON_1>, <CARD_1>. The model is told it can type a placeholder, and Selfbrowse swaps in the real value at the keyboard. This is the default; --no-restore turns it off.
pip install 'selfbrowse[presidio]'
selfbrowse --redact-pii --redact-presidio --allow-domain example.com \
"On my account page, subscribe the email address on file to the newsletter"
What the model sees and does: Email on file: <EMAIL_1> ... type #2 "<EMAIL_1>". What the page receives: priya.raman@acme.com.
from selfbrowse.guardrails import RedactPII, PresidioRedact
BrowseAgent("...", guardrails=[RedactPII(), PresidioRedact()]) # placeholders, restored on typing and in your answer
BrowseAgent("...", guardrails=[RedactPII(restore=False)]) # mask for good: [email]
BrowseAgent("...", guardrails=[RedactPII()], restore_answer=False) # keep the answer masked, e.g. if you forward it to a model or a log
- The same value always gets the same token, across pages and across both masking guardrails, so the model can tell two people apart.
- Only
type(andfind) restore values.goto,select,clickand the traces only ever carry placeholders. Learned site flows never store typed text, so no placeholder can be replayed into a later run with a different meaning. - Same site only. Each placeholder remembers which sites it was seen on and is refused anywhere else (
www.and port are ignored). A page that tricks the model into typing<EMAIL_1>into a form on another site gets the placeholder back, not your email. The model is told why. - Echoes are masked too. After the real value is typed, the page often shows it back (a text box now holding the name). Selfbrowse masks every value it already knows wherever it appears, even where the detector would not have recognised it.
- Your answer shows the real values. The model writes
Confirmation sent to <EMAIL_1>and you get the real address. The recorded steps and traces keep the placeholders. If you pass the answer on to another model or a log, userestore_answer=Falsein Python. Over MCP the answer goes to the calling assistant, so it is never restored there. - It needs the model to copy the placeholder exactly. Brackets are optional (
EMAIL_1works) andEMAIL_1never matchesEMAIL_10, but a model that rewrites a token into words (the email) cannot be restored. See the check below. - MCP server:
selfbrowse-mcp --redact-piiuses placeholders too;browser_typefills in the real value on the same site.--no-restoremasks for good. - When to use
--no-restore: when your rule is that the value must not exist anywhere in the process. By default a mapping from placeholder to real value sits in memory for the length of the run (never on disk, in prompts or in traces). With--no-restorethe value is discarded the moment the page is masked. - It does not remove the need to trust the page. A value the detector missed was shown to the model in plain text. Combine this with
--allow-domain.
Checked against local qwen3.6 (benchmarks/reversible/run.py, 5 trials per mode, a profile page with a name and an email and a form that asks for both):
| Form filled with the real values | Real value seen by the model | Model calls (median) | Tokens (median) | Time per run | |
|---|---|---|---|---|---|
| placeholders (default) | 5 of 5 | 0 of 5 | 6 | 8,287 | 25 to 39 s (one run took 9 calls) |
--no-restore (masked as [email]) |
0 of 5 | 0 of 5 | 10 | 21,676 | 187 to 225 s |
With --no-restore the model typed the literal text [email] into the form every time and still reported success, so the page received nothing useful and the model did not notice. This is one local model, one page and five trials, so read it as a working demonstration, not a rate. Raw rows are in benchmarks/reversible/qwen3.6-result.json; rerun with python benchmarks/reversible/run.py --model ollama:qwen3.6-bench --trials 5.
Learned-flow replays obey the same rules and never ask anyone: if a guardrail objects, the replay stops and the model takes over.
These reduce risk; they are not guarantees. Pattern matching (PII, injection) misses things. The domain allow-list, confirm-risky and budgets are the dependable ones. A hosted model still sees whatever page text you do not redact; only a local model keeps page content on your machine.
Passwords and secrets
Never put a password in the task text. The task goes to the model; a secret should not. Give the secret to Selfbrowse instead, under a name. The model is told only the name (<secret:NAME>), writes that placeholder when it types, and Selfbrowse swaps in the real value at the keyboard. The value never appears in the prompt, the page list, the result, the step history or traces.
Command line. A full example, logging in to a site and reading something behind the login:
# 1. Put the secrets in your shell. `read -s` keeps the password out of your history.
export SELFBROWSE_SECRET_user='me@example.com'
read -s "SELFBROWSE_SECRET_pw?Password: " && export SELFBROWSE_SECRET_pw # zsh
# bash: read -s -p "Password: " SELFBROWSE_SECRET_pw && export SELFBROWSE_SECRET_pw
# 2. Run the task. Say which site to use, and lock the run to it.
selfbrowse --headed --allow-domain example.com \
"Go to example.com/login, sign in with <secret:user> and <secret:pw>, then tell me my latest order number"
The name after SELFBROWSE_SECRET_ is the name the model uses: SELFBROWSE_SECRET_pw becomes <secret:pw>. Every SELFBROWSE_SECRET_* variable is made available, and the model is told the names even if your task does not mention them.
For one command only, set the variable in front of it (the value is still visible in your history, so use this for throwaway values):
SELFBROWSE_SECRET_pw='throwaway' selfbrowse "log in to the staging site and open settings"
Or keep them in a file outside the project and load it when you need it:
# ~/.selfbrowse-secrets (chmod 600; never commit it)
export SELFBROWSE_SECRET_user='me@example.com'
export SELFBROWSE_SECRET_pw='your-password'
source ~/.selfbrowse-secrets && selfbrowse "sign in to example.com and open my orders"
To be asked for a value each time, so it is never stored in your environment, name it with --secret:
selfbrowse --secret pw "Sign in to example.com as me@example.com with <secret:pw> and tell me my latest order number"
# Value for secret 'pw' (not shown):
Selfbrowse has no --secret NAME=value form on purpose: a value on the command line shows up in ps and your history.
Python.
import os
BrowseAgent("log in and open my orders", sensitive={"user": "me@example.com", "pw": os.environ["SITE_PW"]})
MCP server. Set the same SELFBROWSE_SECRET_<name> variables in the server's environment (the env block of your client config). With browser_type, the calling assistant passes <secret:pw> as the text.
Tips:
- Name the field in your task ("log in with secret:user and secret:pw") if the site has more than one form.
- Use
--profile DIRto log in once and keep the session, so later runs need no secret. - For 2FA codes, CAPTCHAs and one-off values you do not want stored, let the model call
ask_human; you type it in the browser window. - Secrets hide what the model sees from the agent's side. They do not stop a page from showing your own details back (an account name, an email address). Add
--redact-pii(or--redact-presidiofor names too) for that. - Pair secrets with
--allow-domain, so a credential can only be typed on the site you meant.
Pop-ups, notifications and permissions
New tabs and windows (target=_blank, OAuth login windows) are followed. The page list shows [tabs: ...] and the agent can switch_tab and close_tab.
Overlays a site draws itself (cookie banners, "Allow notifications?" cards, newsletter modals) are ordinary page elements, so the agent sees them in the list and clicks "No thanks" or "Accept".
Browser permissions (notifications, location, camera, microphone, clipboard) are off by default, so a site that asks is refused and nothing pops up. To allow some for every site you visit:
selfbrowse --permissions notifications,geolocation "Open weather.example.com and tell me today's forecast for my area"
from selfbrowse import BrowseAgent, Browser
BrowseAgent("...", browser=Browser(permissions=["geolocation"]))
Names: geolocation, notifications, camera, microphone, clipboard-read, clipboard-write, midi, midi-sysex, payment-handler, background-sync, accelerometer, gyroscope, magnetometer, ambient-light-sensor, accessibility-events, storage-access. A misspelt name stops the run with the list. Grant the fewest you need, and combine with --allow-domain so only the site you meant gets them. The MCP server takes the same --permissions flag.
JavaScript dialogs (alert, confirm, prompt, "Leave site?") are shown to the model. After an action that raises one, the result says what it was and what happened, for example [A confirm dialog appeared: "Delete everything?". It was dismissed.].
--dialogs |
alert / leave-page |
confirm / prompt |
|---|---|---|
auto (default) |
accepted | dismissed (counts as Cancel) |
accept |
accepted | accepted |
dismiss |
dismissed | dismissed |
With accept, guardrails still apply: --confirm-risky asks a person before accepting a dialog about deleting, paying or sending, and the MCP server (where nobody can be asked) dismisses it. A dialog the model cannot answer in advance is declined, so if a task needs one accepted, run with --dialogs accept.
Not visible to the agent: the browser's own prompts that are drawn outside the page ("Save password?", file pickers, print dialogs). In --headed mode a person can click them. Permissions above cover the common ones.
Sites that show a CAPTCHA (Walmart, Target and similar)
Large retailers run bot detection that scores each visit before showing the page. The default setup (bundled headless Chromium, fresh profile, no history) looks like automation and often gets a challenge. A tool attached to your everyday Chrome passes more often because it has your cookies, history and a normal browser fingerprint.
Try these in order:
- Your installed Chrome, visible, with a saved profile:
selfbrowse --headed --channel chrome --profile ~/.selfbrowse/profile "open walmart.com and tell me the heading"
- If it is still blocked, add the stealth option (
pip install -e '.[stealth]'):
selfbrowse --headed --channel chrome --profile ~/.selfbrowse/profile --stealth "open walmart.com and tell me the heading"
- Browse to the site once yourself in that same profile window, so it has real cookies and history.
Selfbrowse does not solve or defeat CAPTCHAs. Instead it hands the browser to you (next section). Also check each site's terms of service before automating shopping or scraping at scale.
Handing off to a person
When the agent cannot go further, it pauses and lets you act in the visible browser window, then continues from wherever you leave the page. It happens when:
- a bot check appears (CAPTCHA, "verify you are human", "press and hold", Cloudflare "Just a moment"). Detected automatically; the model is not called while the page is blocked.
- the model asks: for a login with no stored secret, a 2FA code, a payment step or a choice only you can make, it calls
ask_human.
selfbrowse --headed --profile ~/.selfbrowse/profile "open target.com and tell me the heading"
When it needs you, the terminal shows a box with the reason and URL and rings the bell. Do the step in the browser window, then press Enter in the terminal to continue (type q + Enter to give up). Using --profile keeps the cookies from a solved check for later runs.
- Without
--headedthere is nobody to hand off to, so the run stops at once with a clear error ("blocked: ... re-run with --headed") instead of wasting model calls. - Nothing you type or see is sent to the model; it is only told you finished.
- It gives up after 3 hand-offs in one run, or if the page is still blocked after you resume.
--no-handoffturns this off.
From Python, use the built-in terminal prompt or plug in your own (a desktop notification, a Slack message, a web UI):
from selfbrowse import BrowseAgent, Browser
def ask_person(request): # request.kind: "challenge" or "model"
print(request.reason, request.url) # notify however you like
input("Press Enter when done: ")
return True # False = give up
BrowseAgent("task", browser=Browser(headless=False), handoff=ask_person).run()
handoff="auto" (default) uses the terminal prompt when the window is visible; handoff=False disables it.
How learning works
Knowledge is stored per site in ~/.selfbrowse/sites/<domain>.json.
- Memo: short facts the model chose to save with the
remembertool. - Flows: click sequences that worked, keyed by page type (for example
/*/dp/*). A flow is replayed without a model call only when it is trusted:- generic: it succeeded for at least two different tasks (a cookie banner, a sign-in step), so it is replayed for every task;
- repeat: it succeeded at least twice for the exact same task, so it is replayed for that task only.
- Only successful runs are learned from. Steps after typing on a page are never recorded for that page. Risky clicks (buy, pay, delete, send, sign out and similar) are never recorded. Failed actions and detours (clicks followed by going back) are not learned. A flow that stops working is dropped, and flows unused for 90 days expire.
Known limit: research-style tasks learn little, because most of their clicks are specific to the question. The biggest wins are generic steps and routine jobs you repeat.
Tests
python -m pytest -q
Browser tests use a local fixture page and need Chromium or Chrome installed. The model is faked, so no API key is needed.
Benchmarks
pip install -e '.[bench]'
python benchmarks/step_tokens.py
This prints what is sent to the model on the first step for four real pages (system prompt, tool schemas, page list). It uses an approximate tokenizer and needs no key. It does not compare against other tools yet.
Renaming
The name lives in src/selfbrowse/_about.py, the package folder, pyproject.toml and the docs.
Contributing
See CONTRIBUTING.md. MIT licensed.
Metadata
Release files for selfbrowse 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| selfbrowse-0.1.0.tar.gz | 109.5 kB | Without provenance | Details for selfbrowse-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| selfbrowse-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for selfbrowse-0.1.0-py3-none-any.whl |
Total release size: 192.5 kB
Release files / selfbrowse-0.1.0.tar.gz
| Download URL | selfbrowse-0.1.0.tar.gz |
|---|---|
| Size | 109.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
83e7c391df92c236d9fac86382c6f11c6998cec12bedca941e719264f28fe269
|
|
BLAKE2b-256 checksum How to use checksums |
d573235537fcee75921cfbb248fb0a34bb0932a69a24867ad3e611264ba15e46
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.9
|
Release files / selfbrowse-0.1.0-py3-none-any.whl
| Download URL | selfbrowse-0.1.0-py3-none-any.whl |
|---|---|
| Size | 83.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
257803c6644e3a95158f094be786bfbf400ff09c2594b2c263a4e173e384694e
|
|
BLAKE2b-256 checksum How to use checksums |
3fcd1653e7f9ee2ad147206a822cdad32bb409370efb2734d23afb972bb7289d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.9
|
Security for selfbrowse 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.