A new mobile/index.html in the desktop's (and Grok Bot's) shape: the sidebar
with its sections and folders, the chat with steps, thinking, reactions, quotes
and the approval card, 對話詳細資訊, Bot 設定, routines, pages, computers and
設定. Each action goes to the desktop's own code on the app's thread
(api/act/<what> -> App::phone_act): appctl, set_pinned, restart_bot,
toggle_reaction, box_task, channels_restart and the rest. No new rules.
Keys, channel tokens and the vault never reach the phone; a model key can only
be written. Admins keep what the phone always let them do; making, changing or
deleting bots, routines, memories, pages, folders and settings stays the
owner's. The state the phone polls now carries the sidebar's section order.
Checked by using it on a preview brain: routines, memories, templates, folders,
keys, signing another phone out, approvals and a chat with a fake model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- Bots share a browser (the box's Chromium has one profile). Every step acted on the tab in
front, so two jobs at once read and clicked on each other's pages and one finishing closed
the other's. A job (named, and the name rides along to another computer's browser) owns the
tabs it opens or lands on, popups it clicks open included; its steps use its latest one, and
another job's tabs are never used, listed or closed. When it ends it closes only its own.
Real run on the box: two jobs at once each read their own page with the other's in front,
both tabs survived both endings; a popup was read by the job that opened it.
- A Docker host that is a Lima VM's socket and doesn't answer (the Mac restarted): AnyBot
starts the VM (limactl start, at most once in 5 minutes), and a step that needs the box
waits for it. No login item needed. Real run: the VM stopped, AnyBot had it back in 10 s;
the box then woke on use, still gVisor with Chromium's sandbox.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
容器隔離 in 設定 › 電腦: standard (runc, the container is the only wall, Chromium runs
--no-sandbox) or hardened (runsc: a virtualized kernel, all capabilities dropped but the few
the desktop needs, ANYBOT_BOX_SANDBOX=1 turns Chromium's own sandbox back on). Box image v7.
deploy/mac/gvisor-vm.sh makes a Lima VM with Docker + gVisor for a Mac (started at login),
gvisor-migrate.sh moves the box's files and logins there.
Checked and finished before committing:
- the hardened choice is offered only on a Docker host that has runsc; a box set to it on a
host without it opens the standard way and says so (it used to not start at all)
- the settings row shows what the box really runs on, and when 更新 is needed
- no swap only under gVisor (a standard box keeps Docker's default)
- the migrate script sees the running Mac app ("AnyBot", not "anybot")
- the VM script starts an existing VM by name, checks runsc and the kernel from the Mac
- the version test says 7
Real run in the Lima VM: desktop, VNC, numbered controls, xdotool and screenshots work; no
Chromium process has --no-sandbox, renderers run with seccomp and their own namespaces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- A job of five or more real steps is looked back on after the answer: the short way that
worked becomes a skill (or improves the one it followed); the reply says so. The prompt has
the bot check its skills before starting.
- Upload, download and PDF on Bot 的電腦's browser look for and save files in the box, not on
the brain (a picture in a Threads post took 24 steps).
- The bot's browser keeps the six tabs used last (19 Threads tabs had filled the box's 2 GB).
- Words name the control, not the box around it: a form's 「Customer name:」, 「Large」 and
「Submit order」 used to land on the <p> holding them and do nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- A turn may take up to 100 steps (was 25, which cut off real work; Grok Bot sets no limit).
Every 25 steps the bot is asked whether it is still getting anywhere.
- The system prompt asks for independent tools in one step and combined commands.
- Browser steps for the box always go to the box's Chromium. They used to depend on whether
the brain itself had a Chrome, so a cloud brain sent them to "the person's browser".
- The box's DevTools forward passes bytes with a plain read/write loop. std::io::copy splices
on Linux and held back short requests, so the box browser never opened from a Linux brain.
- A gateway's HTML error page shows as plain words.
- Real test a_brain_without_chrome_browses_on_its_box (ignored): passes in a Linux container
with no Chrome against a real box, and fails without the routing fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates
from a picture, and the web tools' description told it they didn't work on its own computer.
- marks.rs: the picture a bot gets has every button, link, field and menu item boxed and
numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click /
typing / scroll / mouse_move take the number (ref) and land on the control's centre.
- Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's
computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's
box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets
no number (topmost window at its centre must be the app's; the Dock's invisible full-screen
window doesn't count). open_app waits until the app is really in front.
- No going round in circles: a step that changed nothing says so; the same step (clicks within
16 px) a third time in a turn comes with a warning, from the fourth time it isn't run.
- The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry
at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP,
renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look
(list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a
minute (files and logins stay); a new box is never made from an older image.
- Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the
per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt
says web work goes through browser_* by number, and never to read browser cookie or history
files.
Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new
image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed
into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on
TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28,
153 unit tests; Linux build + 157 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unused (found by taking away the dead-code allowances; nothing else used them, tests aside):
computer::specs, quiet_button, the never-made Poke::Spin, ten icons and their SVGs, the
theme's chrome/bubble_agent colours, ImportRow.line, voice's multipart/transcribe/speak
wrappers, and the old keychain writers vault::save/remove/index_add with their live test
(the vault store replaced them; reading old keychain entries for the import stays).
One copy each: percent-encoding (httpd::url_encode, was four), tool arguments (tools::args_of,
was three), the WebSocket handshake (httpd::accept_websocket, was three), the read pacing
(wire::Pace, was two: relay and app), docker with input and a time limit (dk_with_input, was
two). Docs: FEATURES §看畫面 (VNC), protocol (v* frames, /screen, tickets), architecture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Acting by an app's number (read_app's e7) is now a real click at that control's spot, the
same path as screenshot coordinates. agent-desktop only reads; its own actions (a second set
with its own key names and failures: no typing in terminals, "/" and "`" failing off the main
thread) are gone, with computer_use_select, background clicks and press_key's app.
- Every action comes back with the screen after it (or, with no picture allowed, the front
app's numbered controls), so the bot no longer spends a step on a screenshot to check.
- Coordinates map without remembering a screenshot: the picture's size depends only on the
screen. The phone's taps go straight to the screen as the person's, in the pixels of the
picture it was sent; they used to borrow the bot's last screenshot scale and landed 1.25x off
on the bots' computer, or failed with "screenshot first".
- Mac typing pastes (⌘V, clipboard put back): with 注音 on, text sent as key events vanished
while return arrived, so commands typed into a terminal ran empty. Bare printing keys
("slash", "r") are typed the same way; keys like "slash" and "grave" have names.
- The bots' Docker computer: one docker exec per step (action, settle, picture), never
`mousemove --sync` (it waits forever when the pointer is already there: a second click on
the same spot hung two minutes), open_app (chromium = the browser browser_* drive, xterm).
Tests: box_screen_is_exact (real Docker: bot and phone coordinates exact, same spot twice in
under 2 s, ASCII + Chinese + named key typed), Calculator by numbers and Terminal typing under
注音 through the signed app, remote_e2e 28/28, Linux build + 151 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A new install (desktop or anybot --serve) starts with no bots; the main window shows
「歡迎使用 AnyBot」: connect a model, make the first bot, or sign in.
- Old data whose five sample bots were never used starts empty too.
- Signing in syncs right away: an online brain of the account is used and this computer's
bots are moved there; otherwise the backup's missing bots are taken in and backed up again.
- The five sample bots remain only for screenshots and tests (State::samples).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
王性驊