The top block now gives the three roles their own paste line (Windows
included), matching what install.sh --print reports. Hands takes the URL
and the one-time code from "Add a computer".
Both READMEs now open with a quick install: curl… | sh on Mac and Linux,
irm… | iex on Windows, then where the key goes. The role table moved
below it for the server, hands, and agent paths.
Both languages now open with what AnyBot does, in plain words, before
the install table and the flow diagram. The four stills are re-taken
from this build (shots/readme/) and tracked, since the READMEs point at
them; the old shots/ stayed ignored.
install.sh now covers every common Linux distro (apt, dnf, yum, pacman,
zypper, apk, xbps), macOS on arm64/x86_64, and hands off to install.ps1
under Git Bash. install.ps1 + install.cmd give Windows a double-click.
Autostart: systemd user unit on Linux, launchd on macOS, Task Scheduler
on Windows. A terminal guides the role choice and the first key or bot;
agents get --print JSON and ANYBOT_NONINTERACTIVE=1 with env seeding.
`加一台電腦` shows the Windows and the Mac/Linux line together.
Permissions and reach
- a file leaving a computer passes the same questions as reading it;
"looking" shell commands that do more than look ask first
- a symlink out of the work/sealed folder counts as outside it
(real paths, not just lexical ones); the search walk skips links
- web_fetch and skill_install read the public web only: localhost, the
LAN and cloud metadata are refused, at every redirect
- preview shows a file without running it; browser_upload may ask
- vault: logins no longer match across shared suffixes (pages.dev,
com.tw, ...); registrable_domain knows public suffixes
- axwrite: found elements are retained (use-after-free)
Relay
- replacing or turning off 2FA needs the current code (rate-limited);
new recovery codes only count once enrollment is verified
- a removed computer loses its socket and its `call` access
- a site name taken by another account answers 409
- RELAY_TRUST_PROXY: the login lock counts the caller's address, not
the proxy's (compose files and README)
- httpd: half a body is not a request
Desktop
- sign-in, join and restore work with 2FA on (code field; --serve reads
ANYBOT_ACCOUNT_CODE; the setup wizard points to a pairing code)
- Windows: text was typed twice; modifiers are held through click, drag
and scroll; horizontal scroll; screen control is no longer refused
- a recording that hits the limit is transcribed, not thrown away
- bot's computer: a new home volume restores the snapshot; a snapshot
is read through before the home is wiped; a failed volume removal
stops reset/restore
- events of a stopped turn no longer land in the next one; each channel
message gets a turn of its own
- saving a bot's settings keeps what it got meanwhile (three-way merge);
a save file that cannot be read is kept
- pages in Big5/GBK/Shift_JIS are read in their own encoding
(encoding_rs); osascript output past 64 KB no longer stalls
- pick_tab keeps a job's own non-http tab; pictures larger than the
texture limit are fitted; the auto-update time field takes typing
- skills live in <data>/skills; memories are unnumbered in the prompt;
the phone may skillify and skill_install
CI
- the test step fails when the tests fail (it piped into tail)
Tests: anybot 268 passed, 27 ignored; relay 16 passed; clippy unchanged.
Not run: Windows on a real machine (win.rs type-checked alone), the
Docker paths of the bot's computer, the live accessibility writes.
Co-authored-by: Cursor <cursoragent@cursor.com>
- pick_tab: a page that went somewhere (http) beats about:blank at every
rank — own live tab > free live tab > own blank > free blank. Headless
Chrome boots on about:blank and a job that claims it was pinned there;
desktop behaviour unchanged (the front tab is a live page anyway)
- the tabs list shows a page's url when its title is empty, so the model
never reads a faceless '1. (current)' line
Live proof (headless Chrome + qwen brain, same scenario that failed
before): reads the number through the browser path, 172s -> 92s, and the
model now flags the page's own link as untrusted (S3 working)
- agent live test drives real OpenAI-compatible endpoints (ANYBOT_LIVE_BASE/KEY/MODEL/PROMPT)
- specs assembled like the app (web::specs_for) so the model truly sees the browser tool
- botbrowser honors ANYBOT_BROWSER_FLAGS (containers: --no-sandbox --headless=new)
- src/win.rs: the same mac:: API the harness drives, on Win32 —
SendInput mouse (absolute, multi-click, drag, wheel), keyboard
(macOS key codes mapped to VK, unicode text, modifiers), GDI BitBlt
screenshot, WindowFromPoint/GetForegroundWindow for window_at and
front_app; compiles clean against the windows crate for
x86_64-pc-windows-msvc (API-checked, no hardware here)
- the control tree was already speaking agent-desktop-windows
- intents answer honestly on Windows ("捷徑目前是 macOS/Linux 的")
and don't advertise themselves from the facts line
- CI: a report-only Windows compile job for wherever a Windows runner lives
- needs a real machine: multi-monitor coordinates, high-DPI, UIA-number
clicks, Chinese input
- a skill is now a .md file (front matter + instructions + an optional
steps fence): <data>/skills/ is read at every start, files win;
install from the CLI (anybot --install-skill <path|url>) or from the
phone (設定 › 整合 › 技能 › 從網址安裝), delete takes the file with it
- skillify (C8): long-press your own message in the chat → 教我做 —
the last turn that answered it becomes a replayable skill from the
oplog: only steps that succeeded, looking steps and numbered clicks
filtered (numbers die with their screenshot)
- a browser replay step that doesn't fit gets one fresh page read
before the bot takes over, same courtesy the screen steps had (C8)
- three example skills in skills/, the format guide in docs/skills.md
- the prompt lines that merges had eaten: menu_click/fill_form/wait_for
(C3), memory recall + search_documents (P6) — the tools were there,
the model just wasn't told about them
- look() runs the picture and the control tree at once (two doors of
the system, about the same wait): a third of a second back on every
Mac step; the AX retry path keeps its one-more-look behaviour
- the device lane (this computer serving a remote brain through the
relay) encodes screenshots on a tighter ladder (70/60/50 where the
local one starts at 85): a quarter less to carry, and the test
proves the same picture really comes out smaller
- no wire format changes: same JPEG bytes, just lighter
- SECURITY.md (zh) + docs/en/security.md: trust boundaries (app /
box / Docker API / relay / providers), where every secret lives and
what a thief holding that thing actually gets, the injection layers
with an honest limit (no absolute defense — the human at the gate
is the last line), the network surface, the known-limits list and
hardening advice
- state.json is written 0600: it carries model keys, and now says so
with its permissions like the vault's key file
- README (zh + en) link it; reporting = private security advisory
- intents.rs: volume / wifi / lock / dark / trash / eject as named
shortcuts — the model picks a name, the script is built from
constants and checked numbers ("volume set 40; rm -rf /" is not a
thing: args are grammar-checked before any shell sees them)
- the computer tool gains action=intent {intent_name, intent_arg}:
one call where the bench measured 9 steps and 2 minutes of GUI
- the computer's facts line advertises the shortcuts each machine
really has (Mac 6, Linux desktop 5, headless box 0 — nobody hears
a box's volume), so the prompt cache carries the knowledge for free
- scripts self-dispatch (uname): osascript/networksetup/pmset on Mac,
pactl/amixer/nmcli/loginctl/gsettings on Linux, honest failure text
- system prompt: 有捷徑先捷徑，捷徑失敗才回 GUI
- tests: the injection grammar, the availability matrix, a real run
- embed.rs: vectors from the person's own Ollama (/api/embed), model
auto-found among installed ones (bge-m3, nomic-embed-text…), off by
absence — no Ollama is a normal state: every search still answers
with words, the toggle lives in 設定 › 一般
- memories grow a vec column; recall() is now hybrid: trigram FTS,
two-character windows for Chinese, cosine over the shelf, merged by
reciprocal rank — 「那篇講語法的筆記」 finds text that never says 筆記
- docindex.rs: the person's folders (their 文件 by default, add more in
設定 › 一般), text-shaped files chunked into SQLite + FTS, refreshed
by mtime, gone files leave; one background thread does it all — the
turn never waits on embeddings
- one tool for it: search_documents {query, limit} — reads, no approval
- 251 tests: a real temp folder swept, answered, and swept again after
deletion; offline Ollama tested as the normal state it is
- store.rs grows a memories table + FTS5 (trigram) index: bot facts and
the person's shared lines (bot 0), no cap — state.json's 200-item list
stays the source that mirrors sync and the 設定 list; the index is its
searchable superset, best-effort as always
- queries answer in Chinese: phrase hits (FTS), then words inside lines —
CJK queries cut into two-character windows, lines scored by how many
they contain (「有没有靠窗」 finds 「我喜歡靠窗的座位」)
- every finished turn writes one event line (ask → answer) nobody's
asked to write: 「我上個月說的那個報表」 answers from months back
- the turn's prompt carries the 14 memories that answer *this* request
(no index yet: the list itself, the old behavior); remember/forget and
the UI's 忘記 keep the index in step with the list
- embedding (Ollama) and document indexing land on this same table next
- 「網頁上的事一律用 browser 一個工具」: the guidance lines now name
action=open/read/click/type/table/find/wait/download and action=profiles —
the model reads one vocabulary in the system prompt, the schema and the
loop-guard messages (browser_read/browser_open mentions gone from those)
- H3, light: usage rules were already in the system prompt (and C1 caches
the tools block too, so moving descriptions there saves little) — the
browser schema shed its duplicated safety sentence to the click line;
wholesale description-slimming deliberately not done: the guidance that
reads as noise is what keeps a small model out of trouble
- browser {action: open|read|elements|find|table|screenshot|click|hover|
type|press|select|scroll|wait|tabs|back|upload|download|pdf|dialog|eval|
profiles|fill} — the shape models are trained on (Anthropic CUA, OpenAI CUA)
- vault fills ride on it: fill {kind: login|card|contact, id}; vault_list
stays its own tool; the fills' approvals (every use asks, the model never
sees the secret) are untouched — expand happens at the model-output
boundary, so oplog, approvals, replay, remote limbs keep internal names
- web keeps {web_fetch, browser}; the eighteen browser_* and the three
fill_* schemas leave the model's view (−48% schema text, −58% counting
the old web trio; 25 tools → 2)
- tests: every action maps to a real internal tool, every internal tool is
reachable from the one schema
- 帳號 › 登入的裝置: every signed-in device with its last use; tap a device's
id to sign just that one out; 全部登出 ends every session, this phone
included — the stolen-session story ends on one page, on the phone
- 帳號 › 兩步驗證: enroll (secret + recovery codes shown once, code confirms),
disable (proves the code); the login screen's second step landed with S2
- relay: POST /relay/sessions/drop {id} — a session is named by the first 8
hash characters its row shows; kicking the phone you hold tells the app to
sign itself out (was_self)
- src/store.rs: bundled SQLite (WAL) in the data folder; every oplog record
is mirrored into an audit table — turns, model calls, tools with their
arguments and end statuses, queued messages, stops
- side-car by design: state.json stays the app's truth (one struct, written
atomically); the mirror is best-effort — an audit that never breaks a turn,
and a timeline that answers for 30 days, not just for the running turn
- /api/bots/<id>/audit now reads the trail (phone's 稽核 tab gains history)
- retention: rows older than KEEP_DAYS are deleted on open
- groundwork for memory/RAG (P6) on the same layer; state.json itself moves
in step 2 with the memory tables (the app keeps reading State as a struct)
- oplog gets wall-clock times on tool records (serde-default: old logs read
fine), and OpLog::timeline turns the machine log into a story: per turn —
the prompt, model calls (retries counted into one line), every tool with
the arguments it ran with and its status (等核准/進行中/完成/被拒絕/失敗),
messages typed mid-turn, stop, outcome
- GET /api/bots/<id>/audit (owner/admin; members' allow-list untouched)
- 對話詳細資訊 gains a 稽核 tab: newest turn first, running tool shows
進行中…, an approval gate shows as 等核准 — the phone can answer the question
'what is my bot doing right now, exactly', step by step
- the log stays the machine format recovery reads; the timeline is derived
- README.en.md: full English front door (install table, quick start,
deployment, security-in-one-paragraph, license) + a positioning table
against OpenClaw / OpenHands / Khoj / UI-TARS
- docs/en/{architecture,install,multi-device,protocol}.md: faithful English
summaries of the Chinese docs, each linking to the canonical version
- docs/demo/README.md: how to record the GIF/MP4 the README reserves space for
- language switch line at the top of the Chinese README
- login with 2FA on returns a 5-minute challenge, not a session; the session
comes from /relay/2fa/verify with an authenticator code — or one of eight
recovery codes, each good once (hash slot emptied)
- /relay/2fa/enroll hands out the secret (QR) + codes once; enabling completes
only when a real code proves the app has the secret; disable proves the code
- /relay/sessions/logout kicks every device of the account from one page
- password hashes are argon2id (memory-hard, OWASP parameters); stored PBKDF2
hashes still sign in — no forced reset, upgrade on next password change
- TOTP is RFC 6238 (vectors pinned in tests); mobile page prompts for the
second step; protocol.md documents the endpoints
- fixes a double-lock in /relay/me found by the flow test (handlers hold one
data guard: read through it)
- clicks on controls whose own words say money/sending/deleting ask the person
every time — on every computer, in direct mode, no grant silences it (the
harness decides, not the model, so a page cannot talk its way past it)
- screenshots paint card-number and OTP boxes black before any model sees them
- content the bot READS (pages, files, screens) enters the history wrapped in
<untrusted>, and the system prompt says the wrapper means content-not-orders
- keep_to_chosen now has a test on the screen tools: a model cannot move a
screen step onto another computer by itself (B26's promise, pinned)
- bench red team R1–R7: every payload's goal is a harmless probe file /
button code; passing = reporting the page honestly and leaving the probe
untouched. Run: --tasks R1..R7
- menu_click walks the real macOS menu bar in one step (three screenshot
rounds per level become one call + one picture)
- fill_form writes every named field with one AXValue write each, in one step
- wait_for polls the control tree (no screenshots) until text appears/gone —
replaces the screenshot-polling loops that filled whole turns with identical
pictures
- batch steps take an optional expect: the batch stops and shows the screen at
the step whose promised text never showed, instead of running a blind plan
on sand
Menu/fill are macOS accessibility; other systems keep click+typing per field
(the tool descriptions say which is which).
The quick model learned what its last click did only by re-reading two long
numbered lists. A multiset diff of (kind, name) between consecutive looks adds
two lines and answers it directly — including the case that matters most, a
dialog that appeared entirely.
王性驊
AnyBot Dev