Permissions and reach
- a file leaving a computer passes the same questions as reading it;
"looking" shell commands that do more than look ask first
- a symlink out of the work/sealed folder counts as outside it
(real paths, not just lexical ones); the search walk skips links
- web_fetch and skill_install read the public web only: localhost, the
LAN and cloud metadata are refused, at every redirect
- preview shows a file without running it; browser_upload may ask
- vault: logins no longer match across shared suffixes (pages.dev,
com.tw, ...); registrable_domain knows public suffixes
- axwrite: found elements are retained (use-after-free)
Relay
- replacing or turning off 2FA needs the current code (rate-limited);
new recovery codes only count once enrollment is verified
- a removed computer loses its socket and its `call` access
- a site name taken by another account answers 409
- RELAY_TRUST_PROXY: the login lock counts the caller's address, not
the proxy's (compose files and README)
- httpd: half a body is not a request
Desktop
- sign-in, join and restore work with 2FA on (code field; --serve reads
ANYBOT_ACCOUNT_CODE; the setup wizard points to a pairing code)
- Windows: text was typed twice; modifiers are held through click, drag
and scroll; horizontal scroll; screen control is no longer refused
- a recording that hits the limit is transcribed, not thrown away
- bot's computer: a new home volume restores the snapshot; a snapshot
is read through before the home is wiped; a failed volume removal
stops reset/restore
- events of a stopped turn no longer land in the next one; each channel
message gets a turn of its own
- saving a bot's settings keeps what it got meanwhile (three-way merge);
a save file that cannot be read is kept
- pages in Big5/GBK/Shift_JIS are read in their own encoding
(encoding_rs); osascript output past 64 KB no longer stalls
- pick_tab keeps a job's own non-http tab; pictures larger than the
texture limit are fitted; the auto-update time field takes typing
- skills live in <data>/skills; memories are unnumbered in the prompt;
the phone may skillify and skill_install
CI
- the test step fails when the tests fail (it piped into tail)
Tests: anybot 268 passed, 27 ignored; relay 16 passed; clippy unchanged.
Not run: Windows on a real machine (win.rs type-checked alone), the
Docker paths of the bot's computer, the live accessibility writes.
Co-authored-by: Cursor <cursoragent@cursor.com>
- src/win.rs: the same mac:: API the harness drives, on Win32 —
SendInput mouse (absolute, multi-click, drag, wheel), keyboard
(macOS key codes mapped to VK, unicode text, modifiers), GDI BitBlt
screenshot, WindowFromPoint/GetForegroundWindow for window_at and
front_app; compiles clean against the windows crate for
x86_64-pc-windows-msvc (API-checked, no hardware here)
- the control tree was already speaking agent-desktop-windows
- intents answer honestly on Windows ("捷徑目前是 macOS/Linux 的")
and don't advertise themselves from the facts line
- CI: a report-only Windows compile job for wherever a Windows runner lives
- needs a real machine: multi-monitor coordinates, high-DPI, UIA-number
clicks, Chinese input
- login with 2FA on returns a 5-minute challenge, not a session; the session
comes from /relay/2fa/verify with an authenticator code — or one of eight
recovery codes, each good once (hash slot emptied)
- /relay/2fa/enroll hands out the secret (QR) + codes once; enabling completes
only when a real code proves the app has the secret; disable proves the code
- /relay/sessions/logout kicks every device of the account from one page
- password hashes are argon2id (memory-hard, OWASP parameters); stored PBKDF2
hashes still sign in — no forced reset, upgrade on next password change
- TOTP is RFC 6238 (vectors pinned in tests); mobile page prompts for the
second step; protocol.md documents the endpoints
- fixes a double-lock in /relay/me found by the flow test (handlers hold one
data guard: read through it)
Firefox dropped Chrome's DevTools protocol, so bidi.rs speaks WebDriver BiDi and does each
DevTools command the browser tools use the BiDi way (real mouse and keys, typing, screenshots,
PDF, dialogs, uploads, downloads, tabs). The 18 tools themselves are unchanged and pass the same
dashboard test on Firefox as on Chrome.
- A computer with only Firefox runs AnyBot's own profiles in it (under the Snap/Flatpak corner
of the home folder when Firefox is packaged that way).
- The person's own Firefox profiles are choices too, by the names Firefox shows (profiles.ini
and the newer profile switcher's SQLite list): 「阿金（Firefox）」, or just 「阿金」. Firefox
can't be taken over once open, so a profile open the everyday way is refused with "close it
first"; AnyBot then opens it itself, logins intact.
- Settings lists them without a remove button; the messages name Firefox as an option.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
setup.sh offers 「區網 IP」 (alone, or next to Tailscale): Caddy serves https://<LAN IP> with its
own small certificate authority (tls internal, default_sni for clients that send no name) and
hands out the root at http://<LAN IP>/anybot-ca.crt for each device to trust once.
docker-compose.lan.yml binds Caddy to that address only, so it sits beside tailscale serve.
The app now accepts, for the account server, the certificates this computer trusts as well as
the built-in public ones (HTTP and the WebSocket alike), so it connects once the root is
trusted; before that it says what to install instead of "the certificate is wrong".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
The core: the bot works the computer; a person watches it like a colleague, and takes over
when it needs them. VNC is made for that (only what changed, every mouse move and key, touch on
phones), so the pictures sent one by one go, and with them their own input path.
- vnc.rs: a VNC proxy to the box's x11vnc through one docker exec (a Python bridge that ends
with either side). It signs in to the box with the box's own password (its VNC_PASSWORD) and
offers the viewer no password, so the password never leaves the brain. The person's clicks
and keys make the bot's steps wait until 8 s after they stop; text typed in the viewer's box
(any language) is pasted in the box.
- Account server: /screen (the viewer page) and /novnc.js (noVNC 1.6.0, MPL-2.0, bundled);
/relay/vnc/<host> takes the browser's WebSocket (account session, or a 10-minute ticket the
app asks for with POST /relay/screen-ticket; owners and admins) and carries it over that
AnyBot's lasting connection (vopen / vdata / vtext / vclose). Both ends of that connection
wait 2 ms instead of 40 ms to read while things move: a request and its answer took up to
160 ms more than needed.
- Phone page: 看 Bot 的電腦 and 接管 open the viewer page (also on the local network: /api/vnc on
the AnyBot's own phone server). This Mac's own screen moved to the ⋯ menu.
- Mac app: 看畫面 opens the viewer in the browser (a screen of a brain: the brain's computer by
ticket; else this computer's box, noVNC straight from it). The in-app picture viewer, the
live JPEG stream, the person's click/move/scroll/key endpoints and the input shell are gone.
- Opening a Mac app uses LaunchServices (`open -a`): agent-desktop's launch waited on the window
through accessibility and failed without 輔助使用 now and then.
Measured on this Mac (made-up account, local relay + headless brain + the real box): viewer
signed in in 0.4–0.7 s, a request and its answer through relay + brain + box 25–58 ms median;
headless Chrome shows the desktop live in /screen. remote_e2e 28/28 ×2, 149 unit tests, relay 11.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Acting by an app's number (read_app's e7) is now a real click at that control's spot, the
same path as screenshot coordinates. agent-desktop only reads; its own actions (a second set
with its own key names and failures: no typing in terminals, "/" and "`" failing off the main
thread) are gone, with computer_use_select, background clicks and press_key's app.
- Every action comes back with the screen after it (or, with no picture allowed, the front
app's numbered controls), so the bot no longer spends a step on a screenshot to check.
- Coordinates map without remembering a screenshot: the picture's size depends only on the
screen. The phone's taps go straight to the screen as the person's, in the pixels of the
picture it was sent; they used to borrow the bot's last screenshot scale and landed 1.25x off
on the bots' computer, or failed with "screenshot first".
- Mac typing pastes (⌘V, clipboard put back): with 注音 on, text sent as key events vanished
while return arrived, so commands typed into a terminal ran empty. Bare printing keys
("slash", "r") are typed the same way; keys like "slash" and "grave" have names.
- The bots' Docker computer: one docker exec per step (action, settle, picture), never
`mousemove --sync` (it waits forever when the pointer is already there: a second click on
the same spot hung two minutes), open_app (chromium = the browser browser_* drive, xterm).
Tests: box_screen_is_exact (real Docker: bot and phone coordinates exact, same spot twice in
under 2 s, ASCII + Chinese + named key typed), Calculator by numbers and Terminal typing under
注音 through the signed app, remote_e2e 28/28, Linux build + 151 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- 語言 繁體中文: a reply's Simplified characters are turned into Traditional with Taiwan
wording (zhconv), since some models (Qwen) slip despite the instruction.
- A brain on a server: browser steps go to the person's computer (its hands) unless the
server's own browser is named; 你的電腦 is listed first.
- update_bot says the default computer it set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- install.sh (curl | sh): app, hands, server and deps modes; installs system packages
(right names per release, e.g. libasound2t64 on Ubuntu 24.04), Rust via rustup (1.95+),
Docker for the server; limits parallel compiles on small machines
- Server Docker image: add the ALSA build/runtime packages (the build failed), cache the
cargo registry and target between rebuilds, JOBS build arg
- 加一台電腦 now gives a line that installs AnyBot on a blank computer before joining it
- Makefile: make deps installs Rust too, make doctor, clear message when Rust is missing/old
- README rewritten: one line per use, getting started, update/remove, troubleshooting,
deployment options; docs/install.md and multi-device.md follow it
- Tested on fresh ubuntu:24.04 (app) and debian:12 (curl | sh hands) containers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- anybot setup: a full-screen terminal guide (role, name, account server,
account, which brain) that saves the settings, installs a systemd user
service (with a virtual screen when there is none), and checks that the
account server and the brain see the computer. Flags for scripts.
- Hands can be limited to one brain; the account server says which AnyBot
asks. Choosing a brain in the app does the same.
- deploy/server/setup.sh: domain, account, sign-ups and keys step by step;
writes .env and Caddyfile, starts everything and checks HTTPS. The
server's brain creates the account on first start.
- deploy/linux/hands.sh installs what's needed and opens anybot setup
(no password kept in a file any more).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
- Each AnyBot keeps one WebSocket to the account server (pinged, reopened
within a second when cut). Requests come down it at once; calls to other
AnyBots of the account (brain sync, hands) go up it. Frames carry raw
bytes, compressed when that saves space. HTTP long-poll stays as fallback.
- The account server knows at once when an AnyBot goes away.
- Edits to lists and maps go item by item, so two computers editing one
list keep both changes.
- Screens fetch chat files from the brain, get model lists through the
brain's keys, and show the brain's computer as the brain's.
- Moving this computer's bots skips starter bots the brain has and bots
already moved.
- The phone goes to the brain instead of a computer that shows it.
- deploy/linux/hands.sh: a Linux computer (with or without a screen) as
hands, kept running by systemd.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
Read apps as numbered element lists (computer_use_read_app), act by ref
(click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0.
Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
王性驊
AnyBot Dev