- 「網頁上的事一律用 browser 一個工具」: the guidance lines now name
action=open/read/click/type/table/find/wait/download and action=profiles —
the model reads one vocabulary in the system prompt, the schema and the
loop-guard messages (browser_read/browser_open mentions gone from those)
- H3, light: usage rules were already in the system prompt (and C1 caches
the tools block too, so moving descriptions there saves little) — the
browser schema shed its duplicated safety sentence to the click line;
wholesale description-slimming deliberately not done: the guidance that
reads as noise is what keeps a small model out of trouble
- browser {action: open|read|elements|find|table|screenshot|click|hover|
type|press|select|scroll|wait|tabs|back|upload|download|pdf|dialog|eval|
profiles|fill} — the shape models are trained on (Anthropic CUA, OpenAI CUA)
- vault fills ride on it: fill {kind: login|card|contact, id}; vault_list
stays its own tool; the fills' approvals (every use asks, the model never
sees the secret) are untouched — expand happens at the model-output
boundary, so oplog, approvals, replay, remote limbs keep internal names
- web keeps {web_fetch, browser}; the eighteen browser_* and the three
fill_* schemas leave the model's view (−48% schema text, −58% counting
the old web trio; 25 tools → 2)
- tests: every action maps to a real internal tool, every internal tool is
reachable from the one schema
Measured on the bot's own box (8 tasks x 2, grok-4.7): 14/16 done, 120 steps, 1114 s before;
16/16, 102 steps, 574 s after (model time 1015 -> 471 s). A replayed skill: 49 s -> 15 s.
Knowing the machine
- facts.rs: 電腦現況 per computer (system, desktop, systemd, sudo, network, package manager,
tools, apps, a fixed TZ), read once by a small read-only probe, kept 30 min, in the
instructions at the start of a turn.
- The box's sudo is the bot's: only what follows sudo is checked against the always-denied
list (it was refused outright, while the instructions said it could sudo), and commands with
sudo run as the box's root (setuid does nothing under gVisor).
- Box image v8 drops ENV TZ=UTC (it overrode /etc/localtime). ANYBOT_BOX_IMAGE /
ANYBOT_BOX_VOLUME give a throwaway box its own image and volume, never adopting the real one.
- Mac shell sandbox: App Store app CLIs (they carry their own app sandbox) and ps/top run
outside it; plain `osascript -e` too, asked the first time even in direct mode, refusing do
shell script, terminals, keystrokes, file reads, ObjC and the secret folders. A command the
sandbox broke says so, instead of looking like a broken program.
Reaching what it sees
- The Mac menu bar's status icons are numbered (each app's AXExtrasMenuBar, named from inside
for the system's own), computer_use_zoom looks closer at full sharpness.
- Numbers stay with their controls from look to look (they were reassigned every look, so a
step's second click landed elsewhere); the box's front window is listed first; a control can
be named instead of numbered (name "4").
- Web page numbering walks open shadow DOM (Chromium settings, web components).
- A box on a Docker host elsewhere retries the next port when the usual one is taken (a retry
without it came up with no network).
Working like a person
- Instructions: pick the route, know what done looks like, one route, check once, say what's
stuck instead of faking it with a substitute, ask before cutting the connection.
- StepPlan: the bot's model takes the first step, the step after trouble (failed, refused,
repeated, nothing changed on screen, the quick model switching routes), one in four, and the
final check; a quick model (Bot.step_model, auto: grok-4.20-0309-non-reasoning on xAI) does
the planned steps in between, falling back to the bot's model if it can't be used.
- Skills keep the steps that worked (with what each acted on, not numbers); replay_skill does
them again with no model in between, finding each control again, handing back on a mismatch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
Firefox dropped Chrome's DevTools protocol, so bidi.rs speaks WebDriver BiDi and does each
DevTools command the browser tools use the BiDi way (real mouse and keys, typing, screenshots,
PDF, dialogs, uploads, downloads, tabs). The 18 tools themselves are unchanged and pass the same
dashboard test on Firefox as on Chrome.
- A computer with only Firefox runs AnyBot's own profiles in it (under the Snap/Flatpak corner
of the home folder when Firefox is packaged that way).
- The person's own Firefox profiles are choices too, by the names Firefox shows (profiles.ini
and the newer profile switcher's SQLite list): 「阿金（Firefox）」, or just 「阿金」. Firefox
can't be taken over once open, so a profile open the everyday way is refused with "close it
first"; AnyBot then opens it itself, logins intact.
- Settings lists them without a remove button; the messages name Firefox as an option.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- Bots share a browser (the box's Chromium has one profile). Every step acted on the tab in
front, so two jobs at once read and clicked on each other's pages and one finishing closed
the other's. A job (named, and the name rides along to another computer's browser) owns the
tabs it opens or lands on, popups it clicks open included; its steps use its latest one, and
another job's tabs are never used, listed or closed. When it ends it closes only its own.
Real run on the box: two jobs at once each read their own page with the other's in front,
both tabs survived both endings; a popup was read by the job that opened it.
- A Docker host that is a Lima VM's socket and doesn't answer (the Mac restarted): AnyBot
starts the VM (limactl start, at most once in 5 minutes), and a step that needs the box
waits for it. No login item needed. Real run: the VM stopped, AnyBot had it back in 10 s;
the box then woke on use, still gVisor with Chromium's sandbox.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
The tool only said it put the file in; on a page that shows nothing about it the bot spent five
more steps checking (shell, read, screenshot, two scripts). Now the result names the file the
field holds (and its size) and the page's new text, and an empty field is an error. Real run in
the gVisor box with Grok: the sign-up form went from 10 steps / 35 s to 6 / 20 s; the second
time it followed the skill it had learned, by names only, 6 steps / 21 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- box_numbers_only_on_what_is_seen is ignored again (a test inserted before it in cbada6b took
its mark: every cargo test drove the real box), and the path test runs
- gvisor-vm.sh restarts the VM's Docker only when runsc was just registered (a restart stopped
Bot 的電腦), and `${var}` before full-width text (bash read 「）」's first byte as part of
the name and stopped)
- a scheduled box update and a PDF's name use the person's time zone (a server keeps UTC)
- 容器隔離 lists 標準 before 強隔離
- screenshots of real data can open 設定 › 電腦 (ANYBOT_SHOT_VIEW=computers) and scroll
(ANYBOT_SHOT_SCROLL)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Every browser_open is a new tab and nothing closed them. When a job ends (answered, stopped
or failed) each bot browser it used keeps only its front page; a browser that isn't open isn't
started for this, and one another job is still using is left alone. The six-tab limit stays
for long jobs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- A job of five or more real steps is looked back on after the answer: the short way that
worked becomes a skill (or improves the one it followed); the reply says so. The prompt has
the bot check its skills before starting.
- Upload, download and PDF on Bot 的電腦's browser look for and save files in the box, not on
the brain (a picture in a Threads post took 24 steps).
- The bot's browser keeps the six tabs used last (19 Threads tabs had filled the box's 2 GB).
- Words name the control, not the box around it: a form's 「Customer name:」, 「Large」 and
「Submit order」 used to land on the <p> holding them and do nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
AnyBot Dev
王性驊