- A job of five or more real steps is looked back on after the answer: the short way that
worked becomes a skill (or improves the one it followed); the reply says so. The prompt has
the bot check its skills before starting.
- Upload, download and PDF on Bot 的電腦's browser look for and save files in the box, not on
the brain (a picture in a Threads post took 24 steps).
- The bot's browser keeps the six tabs used last (19 Threads tabs had filled the box's 2 GB).
- Words name the control, not the box around it: a form's 「Customer name:」, 「Large」 and
「Submit order」 used to land on the <p> holding them and do nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Both cloud brains had 「Etc/UTC」 as the time zone (the server's, set when they were made), so pages
said 「今天 01:03」 for 09:03 and a routine for 9 in the morning would run at 5 in the afternoon. The
phone now says where the person is; a brain still on a server's zone takes it (a zone the person
chose stays). Page times use the person's zone, not the computer's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
The phone page was bare: one row per bot, a chat listing every step, a menu with four lines.
- Home: the main bot large at the top with its model and last words, the other bots with times,
then its computers (own box and the account's, online or not, 看畫面), the newest pages and the
routines with their next run.
- Chat: the bot's name opens its settings — model (chat models only), computer, past conversations
(restore), regenerate, new chat. A turn's steps fold into 「已完成 N 個步驟」 when done (the newest
three stay while it works); copy and regenerate under replies; four ready starts in a new chat;
a jump back to the newest message.
- Menu: 外觀 (follow the phone, light, dark).
- API: api/state carries pages and models; each bot its model, computers, routines, past
conversations; POST api/bots/{id}/set | regenerate | restore. The home's last line skips errors.
Checked on a local copy of the brain's data (cut off from the account and Docker) at iPhone size.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
From looking at the real phone page after the fixes: a long command squeezed its step's name into
one character per line (執/行/指/令), and sending a file showed as 「工具」. The name keeps its width,
the detail is cut with an ellipsis, and send_file, react and view_demo_step have names; the window
says 「正在傳檔案給你」 instead of 「正在記下來」 while a file is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
From the second round of real runs:
- Direct mode no longer asks the model to check "always ask" rules before steps that only look
(screenshot, showing the screen, reading a page or file): none of those rules can be about them,
and the check cost each look at the person's Mac 10-20 s. Clicks and typing are still checked.
- An error inside the model's answer stream ("The model is currently at capacity…") is retried after
8 s and said in Chinese; it used to become the bot's reply, in English, never retried.
- send_file says it sends from any computer, the person's included (the bot copied a file to its own
computer first, which only works for text).
- A shown screen's file name carries no time (a server's clock is UTC).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
From real runs on the cloud brain with the person's Mac (Threads patrol, a post draft, "show me my
screen"):
- computer_use_show: a plain picture of any computer's screen goes into the chat (the bot sees it
too). The bot used to describe a screenshot in words and fail to send it.
- The phone shows the bot's attachments: pictures in place, other files download on a tap (api/file,
only from the chats' file folder). It showed nothing for the bot and names only for the person.
- send_file works from the person's other computers (fetch_file over the relay, 20 MB, secrets stay).
- The same call with the same answer again and again is warned the 3rd time and stopped from the
4th (a blank tab was read 11 times, then the bot said Threads was logged out without opening it);
a blank tab says so; checking a login means opening the site.
- Screenshot lists: a nameless row around a named button isn't numbered, a nameless row takes the
words inside it, empty spacer rows aren't numbered (System Settings: 63 numbers -> 33, all named).
- Writing "in my voice" reads the person's own recent posts first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
- A turn may take up to 100 steps (was 25, which cut off real work; Grok Bot sets no limit).
Every 25 steps the bot is asked whether it is still getting anywhere.
- The system prompt asks for independent tools in one step and combined commands.
- Browser steps for the box always go to the box's Chromium. They used to depend on whether
the brain itself had a Chrome, so a cloud brain sent them to "the person's browser".
- The box's DevTools forward passes bytes with a plain read/write loop. std::io::copy splices
on Linux and held back short requests, so the box browser never opened from a Linux brain.
- A gateway's HTML error page shows as plain words.
- Real test a_brain_without_chrome_browses_on_its_box (ignored): passes in a Linux container
with no Chrome against a real box, and fails without the routing fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
box_real_usage_drill (ignored) drives the whole path on a page that reports what really landed:
one batch fills and sends a form (six steps, Chinese typing, 1.6 s), a bare-x/y click on a canvas
lands 1 px off its target, a drag ends where it was pointed, scrolling reveals a field that is
then clicked and typed by number, and the same spot clicked twice says so. Every step is timed:
look 477 ms, steps 650-880 ms.
Caught: the a11y tree is exposed after the screen settles (measured 0.2-0.35 s behind a scroll),
so a list read in parallel with the picture can miss a control the picture already shows.
box_look now starts the capture in the background and gives the a11y read 0.18 s before running;
the parallel capture pays for it, and a scrolled-in field is in the list every run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- computer_use_batch: up to 8 planned actions run in one docker exec (steps chained with && and a
done-marker each), the first failure stops the rest, one picture comes back at the end. Typed
text rides inside the script quoted (one stdin for the chain). The prompt tells the model when
to use it (a form whose fields are all visible) and when not to (step 2 needs step 1's screen).
- The box waits for the screen to stop changing, not the clock: two matching small probes (~60 ms
each) mean settled — ~160 ms on a still screen where every step paid a fixed 500 ms, and a page
mid-load is waited for (up to ~0.9 s) instead of captured mid-change. The picture is JPEG, and
the a11y read runs in parallel with the capture (look ~307 ms; two steps with pictures 1102 ms).
(SETTLE_SH is a group command: dash needs ';' after it — found by box_screen_is_exact.)
- The home volume (every bot's logins) is never one command from loss: snapshots keep generations
(last 7 days, then one per week; the old single snapshot migrates by file date), Reset and
Restore save the running box first and refuse to wipe if saving fails, a box built on an empty
volume re-fills itself from the newest snapshot, and a sleeping holder container keeps the
volume attached so `docker volume prune` can't take it. make doctor shows volume and snapshot
state; the 電腦 card shows the box's average step times and says resets save first.
- box_batch_in_one_round_trip (ignored): three typed steps, ASCII and Chinese, land in one trip.
On the real box after upgrading v3→v5: box_update_keeps_files (files survive, snapshot 2 代),
box_numbers_are_exact, box_screen_is_exact, box_end_to_end, box_traffic_via_mac pass; 155 unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the server's box after the update to 4:
- Chromium called its whole window "not showing" (it took itself for minimized) while its page
was on the screen; the reader (like XBot's) skipped anything not showing, so a page had no
numbers. Now: visible with a place on the screen counts, containers are walked through, and
only a page that isn't showing (a background tab) is left out.
- Two browser profiles: Chromium's own (~/.config/chromium, where a person logged in by hand)
and AnyBot's (~/.config/anybot-browser); the launcher used AnyBot's and Threads showed as
logged out. One profile now (anybot-profile → Chromium's own; a box with only AnyBot's keeps
that); both folders stay as they were.
- The launcher's lock fd went to Chromium, which then held it for its whole life: the next start
waited 30 s and failed. And its flags line had been lost in an edit.
box_numbers_are_exact passes twice on a box from this image; the launcher answers in 0.09 s when
the browser runs; box_screen_is_exact passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates
from a picture, and the web tools' description told it they didn't work on its own computer.
- marks.rs: the picture a bot gets has every button, link, field and menu item boxed and
numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click /
typing / scroll / mouse_move take the number (ref) and land on the control's centre.
- Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's
computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's
box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets
no number (topmost window at its centre must be the app's; the Dock's invisible full-screen
window doesn't count). open_app waits until the app is really in front.
- No going round in circles: a step that changed nothing says so; the same step (clicks within
16 px) a third time in a turn comes with a warning, from the fourth time it isn't run.
- The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry
at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP,
renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look
(list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a
minute (files and logins stay); a new box is never made from an older image.
- Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the
per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt
says web work goes through browser_* by number, and never to read browser cookie or history
files.
Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new
image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed
into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on
TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28,
153 unit tests; Linux build + 157 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unused (found by taking away the dead-code allowances; nothing else used them, tests aside):
computer::specs, quiet_button, the never-made Poke::Spin, ten icons and their SVGs, the
theme's chrome/bubble_agent colours, ImportRow.line, voice's multipart/transcribe/speak
wrappers, and the old keychain writers vault::save/remove/index_add with their live test
(the vault store replaced them; reading old keychain entries for the import stays).
One copy each: percent-encoding (httpd::url_encode, was four), tool arguments (tools::args_of,
was three), the WebSocket handshake (httpd::accept_websocket, was three), the read pacing
(wire::Pace, was two: relay and app), docker with input and a time limit (dk_with_input, was
two). Docs: FEATURES §看畫面 (VNC), protocol (v* frames, /screen, tickets), architecture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The core: the bot works the computer; a person watches it like a colleague, and takes over
when it needs them. VNC is made for that (only what changed, every mouse move and key, touch on
phones), so the pictures sent one by one go, and with them their own input path.
- vnc.rs: a VNC proxy to the box's x11vnc through one docker exec (a Python bridge that ends
with either side). It signs in to the box with the box's own password (its VNC_PASSWORD) and
offers the viewer no password, so the password never leaves the brain. The person's clicks
and keys make the bot's steps wait until 8 s after they stop; text typed in the viewer's box
(any language) is pasted in the box.
- Account server: /screen (the viewer page) and /novnc.js (noVNC 1.6.0, MPL-2.0, bundled);
/relay/vnc/<host> takes the browser's WebSocket (account session, or a 10-minute ticket the
app asks for with POST /relay/screen-ticket; owners and admins) and carries it over that
AnyBot's lasting connection (vopen / vdata / vtext / vclose). Both ends of that connection
wait 2 ms instead of 40 ms to read while things move: a request and its answer took up to
160 ms more than needed.
- Phone page: 看 Bot 的電腦 and 接管 open the viewer page (also on the local network: /api/vnc on
the AnyBot's own phone server). This Mac's own screen moved to the ⋯ menu.
- Mac app: 看畫面 opens the viewer in the browser (a screen of a brain: the brain's computer by
ticket; else this computer's box, noVNC straight from it). The in-app picture viewer, the
live JPEG stream, the person's click/move/scroll/key endpoints and the input shell are gone.
- Opening a Mac app uses LaunchServices (`open -a`): agent-desktop's launch waited on the window
through accessibility and failed without 輔助使用 now and then.
Measured on this Mac (made-up account, local relay + headless brain + the real box): viewer
signed in in 0.4–0.7 s, a request and its answer through relay + brain + box 25–58 ms median;
headless Chrome shows the desktop live in /screen. remote_e2e 28/28 ×2, 149 unit tests, relay 11.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One lasting docker exec takes the desktop as a JPEG inside the box and sends a picture only
when it changed (quicker while it changes, slower while still). Viewers ask for the picture
after the one they have and get it at once. It was one exec per picture (a 1 MB PNG decoded
and encoded again on the brain) and a fixed 0.7 s / 1 s pause: about one picture a second.
Measured on the Mac's box: first picture 45 ms, ~90 KB (was ~1.1 MB), 9–19 pictures a second
while it changes; viewers take at most ~8 a second over the network.
- The loop in the box ends the one before it, ends itself when a write fails ("0" lines while
still) and is ended when nobody watched for 10 s (ending docker exec on this side left it
running in the box).
- The person's clicks and keys go through one lasting shell in the box (text as base64), not a
docker exec each.
- Answers over the lasting connection keep their headers (the relay dropped them): a picture's
number reaches the app. JPEG/PNG/PDF/zip bodies are no longer deflated again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Acting by an app's number (read_app's e7) is now a real click at that control's spot, the
same path as screenshot coordinates. agent-desktop only reads; its own actions (a second set
with its own key names and failures: no typing in terminals, "/" and "`" failing off the main
thread) are gone, with computer_use_select, background clicks and press_key's app.
- Every action comes back with the screen after it (or, with no picture allowed, the front
app's numbered controls), so the bot no longer spends a step on a screenshot to check.
- Coordinates map without remembering a screenshot: the picture's size depends only on the
screen. The phone's taps go straight to the screen as the person's, in the pixels of the
picture it was sent; they used to borrow the bot's last screenshot scale and landed 1.25x off
on the bots' computer, or failed with "screenshot first".
- Mac typing pastes (⌘V, clipboard put back): with 注音 on, text sent as key events vanished
while return arrived, so commands typed into a terminal ran empty. Bare printing keys
("slash", "r") are typed the same way; keys like "slash" and "grave" have names.
- The bots' Docker computer: one docker exec per step (action, settle, picture), never
`mousemove --sync` (it waits forever when the pointer is already there: a second click on
the same spot hung two minutes), open_app (chromium = the browser browser_* drive, xterm).
Tests: box_screen_is_exact (real Docker: bot and phone coordinates exact, same spot twice in
under 2 s, ASCII + Chinese + named key typed), Calculator by numbers and Terminal typing under
注音 through the signed app, remote_e2e 28/28, Linux build + 151 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The screenshot buffer was a sparse file in XDG_RUNTIME_DIR. With that folder full
(a 1.6 GB Hyprland log), Hyprland wrote the pixels into pages that couldn't exist
and died with SIGBUS in CScreenshareFrame::copyShm. Now the buffer is a memfd with
every page reserved up front; with no room the screenshot fails in AnyBot instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A turn started from the phone (not a computer) kept 「don't know」 for its whole run: a message
typed on the Mac while it waited on a hand-off had its 「我的電腦」 moved to the bot's own computer.
Each step now asks which computer the person last typed on, and the brain keeps it beside its
data (person-computer.txt) across restarts. remote_e2e: a second computer, and a mid-turn
message from the other one (fails without the fix); 28 checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New icon: the bot's cloud body and slanted eyes (as in the app's avatars) on a dark tile —
the Mac/Linux app, browser tabs, iPhone home screen (PNG: it ignores SVG), Android and
notifications. The web had its own ring-and-dot.
- anybot --write-icon out.png size [file.svg] draws any SVG (for the web PNGs).
- Phone: lists that reach the bottom keep their last row above the home bar and Safari's
toolbar (it looked cut off); bots show their shape, colour and eyes instead of a letter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A desktop AnyBot left on the server (older build) took "anybot-box" for its own and paused it
every ~20 s as idle while the brain woke it: the bots' computer kept showing as asleep. The
brain now renames its box once (files are in the volume, kept by name) and uses that name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The brain looked the account's computers up once a minute: for up to a minute after the Mac
came on, 「我的電腦」 was unknown and steps went elsewhere. Now every 15 s, and at once when a
message comes from a computer it doesn't know yet.
- A step naming a computer that is off says so in words instead of running on the chosen one
with a note that the person hadn't asked.
- Snapshots: tar's 'file changed' (the open browser) is not a failure; browser caches are left
out; ten minutes for a large home folder.
- box_end_to_end clicks the terminal where it opened; restore only with ANYBOT_TEST_RESTORE=1.
- deploy/test/remote_e2e.py: account server + headless brain + hands + scripted model, 27 checks
(computers, redirect, agent-desktop on the hands, hand-off 3 ways, phone view, sync copy,
edits, hands offline/back, brain restart, clear). anybot --sample-state for such tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- hand_over_computer (the original's box hand-off card): instruction, live screen with 接管,
跳過 / 我完成了 / 接管; 我完成了 also in the viewer; a typed message answers it too. Header,
sidebar and notifications say 等你處理 / 「{name}：需要你」; the phone shows it with 接管.
- Mac: the content reaches the top (no grey title bar); the top strip drags the window.
- Panel 電腦 card: each line stops before the picture; the brain's name on its own line.
- Viewer: buttons first, the title ends in … when narrow; the picture is centred.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- No idle pause: an asleep box is woken; its files are saved once a day instead of before sleep.
- A headless brain offers no Local computer; attachments it receives go into the bot's own one.
- 「你的電腦」 kept by older versions means the bot's own computer; only an explicit choice moves it.
- The composer chip moved to the panel's 電腦 card (做事的電腦).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every untrusted step popped macOS's prompt and opened Settings; a remote brain's steps covered
the screen. Now once per pane per ten minutes.
- Ad hoc signing made each build a new app to macOS, so 輔助使用/螢幕錄製 silently stopped
working after updates (agent-desktop PERM_DENIED). deploy/mac/sign.sh signs with a local key
kept in its own keychain under AnyBot's folder.
- computer_use_open_app finds 「計算機」 by display name and says the app opened even when its
window can't be read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A bot's own computer is 「<bot> 的電腦」; this one and the account's others go by their names.
The server brain picked 你的電腦 by sorted host id, so a desktop left on the server won.
- 「我的電腦」 is the computer the person typed on (the relay names it); unknown means ask.
- A step moved back to the chosen computer tells the bot so.
- Composer chip and bot settings choose the computer by name; a screen asks the brain.
- A screen's 電腦 card and viewer show the brain's computer, not this one's box.
- A screen's chat copy that falls short asks for everything again instead of appending;
清除聊天記錄 on a screen clears the brain's chat.
- Model requests to a private address go through the person's computer first.
- Phone typing on the box sends the text the tool reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A brain that can't reach a model at a private address (e.g. Qwen at 10.x) sends the request
through its hands; the hands only forward to addresses on their own network, and only for
their brain (uses the existing device channel, so the account server is unchanged).
- The computer chosen for a bot is binding: a step naming another computer goes to the chosen
one unless the person's message names that one (「在我的 Mac 上」).
- Browser steps follow the computer: Bot 的電腦 drives the Chromium on its own desktop (through
docker exec, no new port), 你的電腦 its browser; before, they always used this computer's.
Checked: on this Mac, Bot 的電腦 browsing opened example.com in the box's Chromium and an
unasked 你的電腦 command ran in the box; 「在我的 Mac 上」 ran on the Mac. Linux brain in Docker
with 10.0.0.0/8 blocked + this Mac as hands: model calls went through the Mac, shell on
你的電腦 answered from the Mac, browser defaulted to the Mac's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- 語言 繁體中文: a reply's Simplified characters are turned into Traditional with Taiwan
wording (zhconv), since some models (Qwen) slip despite the instruction.
- A brain on a server: browser steps go to the person's computer (its hands) unless the
server's own browser is named; 你的電腦 is listed first.
- update_bot says the default computer it set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- On a server brain whose hands are the person's computer, 你的電腦 is that computer;
the server itself is 大腦伺服器 (its browser too), so 「我的 Mac」 means the Mac.
- A brain with its own hands no longer lists the account's other brains as computers.
- 查看電腦 lists the computers the bot really has, with their real names.
- Phone page: with several brains on, it opens the one the account's computers use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A new install (desktop or anybot --serve) starts with no bots; the main window shows
「歡迎使用 AnyBot」: connect a model, make the first bot, or sign in.
- Old data whose five sample bots were never used starts empty too.
- Signing in syncs right away: an online brain of the account is used and this computer's
bots are moved there; otherwise the backup's missing bots are taken in and backed up again.
- The five sample bots remain only for screenshots and tests (State::samples).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A model that can't see pictures got only 「截圖已拍好，但看不到」 for computer_use_screenshot.
Now the newest screenshot is described by a model of the person's that can (windows, text,
buttons and roughly where they are); older screenshots stay a short note.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
A brain on a server has no AnyBot browser, and browser_open/read were sent to it even when
the bot named a 手腳 computer. Now a named computer wins, and with none named the hands'
browser is used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
A service set up without a screen (over SSH, or on Wayland) used to pick up the desktop
session's DISPLAY/WAYLAND_DISPLAY from systemd, so screenshots waited for a 「分享螢幕」
answer nobody could give. Now it starts its own virtual screen; `anybot setup --real-screen`
keeps the desktop. `anybot setup --refresh-service` rewrites the start script after an update
(install.sh runs it), so joined computers pick up such changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
computer_use_* name the computer whenever the screen is another computer's, and with no
computer named the step goes to a screen that is really there (the bot's desktop, this
computer's, then a 手腳 computer) instead of a brain's missing one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- The brain's bots get computer_use_* for 手腳 (devices), as the prompt already said; before,
they only had shell there and fell back to scrot, which can't see a Wayland desktop
- After saying yes in the 「使用電腦工具」 pop-up (it shows the step), that same step isn't asked
again (sending messages still asks every time)
- A certificate failure says so instead of 「網路連線出了問題」
- install.sh restarts the background AnyBot after an update
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- write_page / read_page / list_pages: Markdown pages with versions (last 10 kept), per bot;
the reply gets a card (title, fading preview, page icon, 頁面 · 更新時間, 最近更新), and
[標題](page:ID) links open the page
- The page opens large: summary, what changed, the version picker, 複製 / 存成檔案 / 刪除;
listed under 資料庫 in the right panel
- Page blocks for richer pages, in the chat's Markdown too: ```stats tiles, ```steps progress,
> [!NOTE]/[!TIP]/[!WARNING]/[!IMPORTANT] call-outs (with tables and Mermaid charts)
- Phone: the card in chats, page links, a page view with the same blocks; charts drawn by the
AnyBot as SVG (GET /api/pages/<id>[/chart/<n>]); members may read pages
- Bots are told to make a page for things to track and to update it from a routine only when
there is real news
- Screenshot views: ANYBOT_SHOT_VIEW=pagecard / page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- install.sh (curl | sh): app, hands, server and deps modes; installs system packages
(right names per release, e.g. libasound2t64 on Ubuntu 24.04), Rust via rustup (1.95+),
Docker for the server; limits parallel compiles on small machines
- Server Docker image: add the ALSA build/runtime packages (the build failed), cache the
cargo registry and target between rebuilds, JOBS build arg
- 加一台電腦 now gives a line that installs AnyBot on a blank computer before joining it
- Makefile: make deps installs Rust too, make doctor, clear message when Rust is missing/old
- README rewritten: one line per use, getting started, update/remove, troubleshooting,
deployment options; docs/install.md and multi-device.md follow it
- Tested on fresh ubuntu:24.04 (app) and debian:12 (curl | sh hands) containers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- 看圖小幫手: when a bot's model can't take pictures, a model of the
person's that can (Grok, Claude, GPT, Gemini… whichever has a key)
describes each picture the person sent, text and numbers included, and
the bot gets that description. Remembered per picture; helpers that
fail are skipped for ten minutes.
- On the Mac, commands still can't read AnyBot's data folder (the keys),
but may read the chat's attachments in it (not write them).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- Ticking 建立新帳號 for an account that already exists signs in with it;
when that fails too, the message says the server has an account and
how to reset a forgotten password.
- anybot-relay --data DIR --reset-password NAME (only while the server is
stopped; asks twice, not shown; signs out that account's phones).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- 加一台電腦 (phone menu and the app's 大腦 card) gives a ten-minute,
one-time code and the line to run: anybot setup <address> <code>. The
new computer joins as hands for that brain with no password and nothing
to answer, starts with the computer and checks the brain sees it.
- anybot setup without arguments is one screen: address and code (a pasted
line fills both), or the account and password instead. No role, name or
confirm questions; a brain is asked only when there are several.
- Hands computers are no longer offered as brains.
- The server setup asks only the domain, account and password.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- anybot setup: a full-screen terminal guide (role, name, account server,
account, which brain) that saves the settings, installs a systemd user
service (with a virtual screen when there is none), and checks that the
account server and the brain see the computer. Flags for scripts.
- Hands can be limited to one brain; the account server says which AnyBot
asks. Choosing a brain in the app does the same.
- deploy/server/setup.sh: domain, account, sign-ups and keys step by step;
writes .env and Caddyfile, starts everything and checks HTTPS. The
server's brain creates the account on first start.
- deploy/linux/hands.sh installs what's needed and opens anybot setup
(no password kept in a file any more).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- Each AnyBot keeps one WebSocket to the account server (pinged, reopened
within a second when cut). Requests come down it at once; calls to other
AnyBots of the account (brain sync, hands) go up it. Frames carry raw
bytes, compressed when that saves space. HTTP long-poll stays as fallback.
- The account server knows at once when an AnyBot goes away.
- Edits to lists and maps go item by item, so two computers editing one
list keep both changes.
- Screens fetch chat files from the brain, get model lists through the
brain's keys, and show the brain's computer as the brain's.
- Moving this computer's bots skips starter bots the brain has and bots
already moved.
- The phone goes to the brain instead of a computer that shows it.
- deploy/linux/hands.sh: a Linux computer (with or without a screen) as
hands, kept running by systemd.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- One worker sends edits, messages and actions to the brain in order.
- Edits carry a number; the brain says which it has, and a screen replays
the rest on top of newer settings so typing isn't undone.
- A message or edit that didn't arrive makes the screen fetch everything again.
- An older copy of a routine can't reset when it last/next runs.
- Network and server failures read as words (no HTTP codes), naming what
can't be reached; the account server notices a closed AnyBot in 8s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- Brain answers /api/sync (per-screen deltas of settings and messages),
/api/sync/edit (small edit operations) and /api/sync/act (chat actions).
- Relay forwards host-to-host sync requests within one account (role mirror).
- Desktop: choose a brain in 手機與帳號; its own bots are kept aside; offline
shows the last copy read-only and reconnects by itself.
- Keys and tokens stay on the brain (redacted and restored on edits).
- Browser tools can run on another computer of the account (computer arg).
- anybot --serve: ANYBOT_DEVICE_LINK=1 and ANYBOT_BRAIN=名稱.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
Read apps as numbered element lists (computer_use_read_app), act by ref
(click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0.
Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
王性驊