- agent live test drives real OpenAI-compatible endpoints (ANYBOT_LIVE_BASE/KEY/MODEL/PROMPT)
- specs assembled like the app (web::specs_for) so the model truly sees the browser tool
- botbrowser honors ANYBOT_BROWSER_FLAGS (containers: --no-sandbox --headless=new)
- src/win.rs: the same mac:: API the harness drives, on Win32 —
SendInput mouse (absolute, multi-click, drag, wheel), keyboard
(macOS key codes mapped to VK, unicode text, modifiers), GDI BitBlt
screenshot, WindowFromPoint/GetForegroundWindow for window_at and
front_app; compiles clean against the windows crate for
x86_64-pc-windows-msvc (API-checked, no hardware here)
- the control tree was already speaking agent-desktop-windows
- intents answer honestly on Windows ("捷徑目前是 macOS/Linux 的")
and don't advertise themselves from the facts line
- CI: a report-only Windows compile job for wherever a Windows runner lives
- needs a real machine: multi-monitor coordinates, high-DPI, UIA-number
clicks, Chinese input
- a skill is now a .md file (front matter + instructions + an optional
steps fence): <data>/skills/ is read at every start, files win;
install from the CLI (anybot --install-skill <path|url>) or from the
phone (設定 › 整合 › 技能 › 從網址安裝), delete takes the file with it
- skillify (C8): long-press your own message in the chat → 教我做 —
the last turn that answered it becomes a replayable skill from the
oplog: only steps that succeeded, looking steps and numbered clicks
filtered (numbers die with their screenshot)
- a browser replay step that doesn't fit gets one fresh page read
before the bot takes over, same courtesy the screen steps had (C8)
- three example skills in skills/, the format guide in docs/skills.md
- the prompt lines that merges had eaten: menu_click/fill_form/wait_for
(C3), memory recall + search_documents (P6) — the tools were there,
the model just wasn't told about them
- look() runs the picture and the control tree at once (two doors of
the system, about the same wait): a third of a second back on every
Mac step; the AX retry path keeps its one-more-look behaviour
- the device lane (this computer serving a remote brain through the
relay) encodes screenshots on a tighter ladder (70/60/50 where the
local one starts at 85): a quarter less to carry, and the test
proves the same picture really comes out smaller
- no wire format changes: same JPEG bytes, just lighter
- SECURITY.md (zh) + docs/en/security.md: trust boundaries (app /
box / Docker API / relay / providers), where every secret lives and
what a thief holding that thing actually gets, the injection layers
with an honest limit (no absolute defense — the human at the gate
is the last line), the network surface, the known-limits list and
hardening advice
- state.json is written 0600: it carries model keys, and now says so
with its permissions like the vault's key file
- README (zh + en) link it; reporting = private security advisory
- intents.rs: volume / wifi / lock / dark / trash / eject as named
shortcuts — the model picks a name, the script is built from
constants and checked numbers ("volume set 40; rm -rf /" is not a
thing: args are grammar-checked before any shell sees them)
- the computer tool gains action=intent {intent_name, intent_arg}:
one call where the bench measured 9 steps and 2 minutes of GUI
- the computer's facts line advertises the shortcuts each machine
really has (Mac 6, Linux desktop 5, headless box 0 — nobody hears
a box's volume), so the prompt cache carries the knowledge for free
- scripts self-dispatch (uname): osascript/networksetup/pmset on Mac,
pactl/amixer/nmcli/loginctl/gsettings on Linux, honest failure text
- system prompt: 有捷徑先捷徑，捷徑失敗才回 GUI
- tests: the injection grammar, the availability matrix, a real run
- embed.rs: vectors from the person's own Ollama (/api/embed), model
auto-found among installed ones (bge-m3, nomic-embed-text…), off by
absence — no Ollama is a normal state: every search still answers
with words, the toggle lives in 設定 › 一般
- memories grow a vec column; recall() is now hybrid: trigram FTS,
two-character windows for Chinese, cosine over the shelf, merged by
reciprocal rank — 「那篇講語法的筆記」 finds text that never says 筆記
- docindex.rs: the person's folders (their 文件 by default, add more in
設定 › 一般), text-shaped files chunked into SQLite + FTS, refreshed
by mtime, gone files leave; one background thread does it all — the
turn never waits on embeddings
- one tool for it: search_documents {query, limit} — reads, no approval
- 251 tests: a real temp folder swept, answered, and swept again after
deletion; offline Ollama tested as the normal state it is
- store.rs grows a memories table + FTS5 (trigram) index: bot facts and
the person's shared lines (bot 0), no cap — state.json's 200-item list
stays the source that mirrors sync and the 設定 list; the index is its
searchable superset, best-effort as always
- queries answer in Chinese: phrase hits (FTS), then words inside lines —
CJK queries cut into two-character windows, lines scored by how many
they contain (「有没有靠窗」 finds 「我喜歡靠窗的座位」)
- every finished turn writes one event line (ask → answer) nobody's
asked to write: 「我上個月說的那個報表」 answers from months back
- the turn's prompt carries the 14 memories that answer *this* request
(no index yet: the list itself, the old behavior); remember/forget and
the UI's 忘記 keep the index in step with the list
- embedding (Ollama) and document indexing land on this same table next
- 「網頁上的事一律用 browser 一個工具」: the guidance lines now name
action=open/read/click/type/table/find/wait/download and action=profiles —
the model reads one vocabulary in the system prompt, the schema and the
loop-guard messages (browser_read/browser_open mentions gone from those)
- H3, light: usage rules were already in the system prompt (and C1 caches
the tools block too, so moving descriptions there saves little) — the
browser schema shed its duplicated safety sentence to the click line;
wholesale description-slimming deliberately not done: the guidance that
reads as noise is what keeps a small model out of trouble
- browser {action: open|read|elements|find|table|screenshot|click|hover|
type|press|select|scroll|wait|tabs|back|upload|download|pdf|dialog|eval|
profiles|fill} — the shape models are trained on (Anthropic CUA, OpenAI CUA)
- vault fills ride on it: fill {kind: login|card|contact, id}; vault_list
stays its own tool; the fills' approvals (every use asks, the model never
sees the secret) are untouched — expand happens at the model-output
boundary, so oplog, approvals, replay, remote limbs keep internal names
- web keeps {web_fetch, browser}; the eighteen browser_* and the three
fill_* schemas leave the model's view (−48% schema text, −58% counting
the old web trio; 25 tools → 2)
- tests: every action maps to a real internal tool, every internal tool is
reachable from the one schema
- src/store.rs: bundled SQLite (WAL) in the data folder; every oplog record
is mirrored into an audit table — turns, model calls, tools with their
arguments and end statuses, queued messages, stops
- side-car by design: state.json stays the app's truth (one struct, written
atomically); the mirror is best-effort — an audit that never breaks a turn,
and a timeline that answers for 30 days, not just for the running turn
- /api/bots/<id>/audit now reads the trail (phone's 稽核 tab gains history)
- retention: rows older than KEEP_DAYS are deleted on open
- groundwork for memory/RAG (P6) on the same layer; state.json itself moves
in step 2 with the memory tables (the app keeps reading State as a struct)
- oplog gets wall-clock times on tool records (serde-default: old logs read
fine), and OpLog::timeline turns the machine log into a story: per turn —
the prompt, model calls (retries counted into one line), every tool with
the arguments it ran with and its status (等核准/進行中/完成/被拒絕/失敗),
messages typed mid-turn, stop, outcome
- GET /api/bots/<id>/audit (owner/admin; members' allow-list untouched)
- 對話詳細資訊 gains a 稽核 tab: newest turn first, running tool shows
進行中…, an approval gate shows as 等核准 — the phone can answer the question
'what is my bot doing right now, exactly', step by step
- the log stays the machine format recovery reads; the timeline is derived
- login with 2FA on returns a 5-minute challenge, not a session; the session
comes from /relay/2fa/verify with an authenticator code — or one of eight
recovery codes, each good once (hash slot emptied)
- /relay/2fa/enroll hands out the secret (QR) + codes once; enabling completes
only when a real code proves the app has the secret; disable proves the code
- /relay/sessions/logout kicks every device of the account from one page
- password hashes are argon2id (memory-hard, OWASP parameters); stored PBKDF2
hashes still sign in — no forced reset, upgrade on next password change
- TOTP is RFC 6238 (vectors pinned in tests); mobile page prompts for the
second step; protocol.md documents the endpoints
- fixes a double-lock in /relay/me found by the flow test (handlers hold one
data guard: read through it)
- clicks on controls whose own words say money/sending/deleting ask the person
every time — on every computer, in direct mode, no grant silences it (the
harness decides, not the model, so a page cannot talk its way past it)
- screenshots paint card-number and OTP boxes black before any model sees them
- content the bot READS (pages, files, screens) enters the history wrapped in
<untrusted>, and the system prompt says the wrapper means content-not-orders
- keep_to_chosen now has a test on the screen tools: a model cannot move a
screen step onto another computer by itself (B26's promise, pinned)
- bench red team R1–R7: every payload's goal is a harmless probe file /
button code; passing = reporting the page honestly and leaving the probe
untouched. Run: --tasks R1..R7
- menu_click walks the real macOS menu bar in one step (three screenshot
rounds per level become one call + one picture)
- fill_form writes every named field with one AXValue write each, in one step
- wait_for polls the control tree (no screenshots) until text appears/gone —
replaces the screenshot-polling loops that filled whole turns with identical
pictures
- batch steps take an optional expect: the batch stops and shows the screen at
the step whose promised text never showed, instead of running a blind plan
on sand
Menu/fill are macOS accessibility; other systems keep click+typing per field
(the tool descriptions say which is which).
The quick model learned what its last click did only by re-reading two long
numbered lists. A multiset diff of (kind, name) between consecutive looks adds
two lines and answers it directly — including the case that matters most, a
dialog that appeared entirely.
The bench's precision failures (quick model clicking the wrong number, Retina
scaling, covered windows) all come from acting on pixels. A plain left click
now asks the frontmost app's accessibility tree first — by name, or by the
numbered spot — and typing writes AXValue into the focused field once,
keyboard events only when accessibility can't reach. Any app with a thin tree
behaves exactly as before: every AX answer that isn't a clear yes falls
through to CGEvent.
macOS code type-checked with --target x86_64-apple-darwin; gate: build on the
macos-arm64 runner + bench tasks T2/T5 (clicking by name) on a Mac.
Linux/Windows: stubs, pixel path unchanged.
The model picks action=screenshot/click/typing/… on a single `computer` tool
(the shape Anthropic's and OpenAI's computer-use models are trained on):
choosing among twelve near-identical names was its own error source, and the
twelve schemas were the biggest stable part of every request. Calls expand to
the internal per-action tools at the model-output boundary, so approvals,
repeat guards, skill replay, the remote limb protocol and the work log keep
their existing names — old limbs need no update.
A computer-use turn sends the same system + tool schemas every step, and the
history is append-only between steps: three breakpoints (system, last tool,
last content block) let steps 2..n of a turn read the cache the first step
wrote. CACHE_STATS counts input/cache_read/cache_creation tokens for make
doctor and the bench (target: cache_read > 70% of input on Claude routes).
Follow-ups in the plan (same item): freeze facts into the request at turn
start, keep the quick-model tool list constant per step (stable prefixes).
The person wants the local box and Lima VM off whenever this AnyBot is set
to show another brain, not stopped after two idle minutes.
- sleep_on_a_screen has no idle wait and also runs right at start, before
the loop's first 20-second pause; a resume of an interrupted rebuild is
done on brains only (it would start the box on a screen).
- gvisor-vm.sh no longer registers the VM to start with the Mac: AnyBot
starts it itself when it is the brain and wants its computer.
Verified on this Mac: VM started by hand while AnyBot was closed; the new
AnyBot (a screen of the cloud brain) stopped it 4 seconds after launch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
The Mac mini shows the cloud brain, whose bots work on the brain's own
computer and borrow only this Mac's screen, browser and apps. Its local
box still ran, and the Lima VM under it had grown to its whole 6 GB: the
Mac never gets back memory a VM once used.
- On a screen the box stops after 2 idle minutes, and the Lima VM too when
nothing else runs in it. probe() no longer starts the VM there, and the
volume holder and scheduled updates run on brains only. Becoming a brain
again starts both (probe/ensure_running already wake the VM).
- Screen mode is decided from the settings before the box loop's first
look, which used to start the VM at launch.
- gvisor-vm.sh: 4 GB instead of 6 (box capped at 2 GB), and the VM's
ModemManager, udisks2 and multipathd are turned off.
Verified on this Mac: box and VM stopped by themselves 2 minutes after the
app started; the VM process (4.4 GB plus compressed pages) is gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- quick_tools indexed serde_json Values mutably, which inserted
actions.items.properties = null into click/typing/scroll; xAI refused
the whole tool list, so since ba1fce5 every step fell back to the main
model. It now uses get_mut and the test checks nothing but ref changes.
- The front app's menu bar titles and any open menu (submenus, right-click
menus) are listed with numbers and their shortcuts; an option-key
alternate in the same place is noted on the visible item.
- Disclosure triangles, colour wells and date fields get numbers; an
icon-only control without a name goes by its description.
- When the front window can't be read the list says so (with the reason)
instead of 「」 above the menu bar alone.
- Computer facts add each system's conventions (Mac: cmd+shift+G, save
panel, default format, /bin/sh echo -n; Linux: GTK ctrl+L).
- `anybot --help` and unknown -- flags print the usage and exit instead of
opening the window on the real data folder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- Keys and typing into no field go to whatever is in front: a cmd+w meant for TextEdit reached
the person's terminal after the TextEdit window had closed (its "close this window?" came up
and was cancelled). They are now refused, with why, when the app in front isn't the one of the
latest look.
- The quick model's screen tools have no `ref`: looking numbers up is where it pressed the wrong
keys; it names controls (two of a name come back as trouble for the bot's own model).
- Instructions: macOS permission and security prompts are for the person to answer (a bot
pressed 「不允許」 on one), and check what's in front before a shortcut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- Two identical clicks in one step opened the Mac's Wi‑Fi menu and closed it again: the second is
skipped with a word on why (count: 2 for a double click).
- macOS Calculator had no focused window for a moment after 「乘」 and the list came back with
only the menu bar: an empty or failed read is tried once more 0.3 s later.
- Benchmark notes: three more unseen apps on box v9 (16/16, GTK 2 apps 4-10x faster with gail)
and the Mac mini run with Screen Recording (zoom, Wi‑Fi menu, Calculator checked by eye).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- Box v9: GTK_MODULES=gail:atk-bridge. lxappearance and PCManFM (GTK 2) offered AT-SPI no
controls at all (0 numbers); with gail they offer 37 and 17.
- The quick model ending a turn without acting goes to the bot's own model every time, not only
after changes: one stopped after a single look to ask 「要我繼續嗎？」.
- Instructions: finish what was asked; put an unexpected screen right (close what's in the way,
open the app) instead of stopping to ask, except for what the person must do or confirm.
- Benchmark: three more held-out tasks (lxappearance theme, PCManFM folder, Mousepad
find-and-replace), checked by their results; every task starts with those apps closed too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- The one-line-field check got the step's name as 「#14（輸入框…）」 on the box and never
matched, so the select-and-paste never ran there and addresses still came out mangled. It
reads both forms now, with a test on the script the box runs.
- Trouble twice in a turn and the rest of the turn is the bot's own model's (a quick model
guessed its way through Mousepad's save dialog for 79 steps).
- A name that isn't on the screen comes back with the names close to it ([12]「New Window」…),
so the step is put right at once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
Typed key by key into Chromium's address bar right after it opened, the page load and the
autocomplete took turns with the keys and the address came out mangled (chrome://blankchrome//…,
colons gone) — even after selecting the old text first. A one-line field (one typing replaces)
now gets its text pasted in one go, with a short pause before return; terminals and text areas
are still typed key by key. Reproduced and checked on the box's Chromium.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
- Typed into the middle of Chromium's 「about:blank」, an address came out as
chrome://blankchrome//settings/search, and every step after it went wrong. Typing into a
field by name or number now selects a one-line field's text first (ctrl+a / cmd+a) so the
text replaces it; append=true keeps it. Text areas are left alone.
- A model sending name plus x: 0, y: 0 had its click land in the screen's corner: a name (or
number) now always wins over coordinates.
- After the VM went down hard, the box's first start failed in gVisor ("mount source already
has a filestore file … repeated submounts are not supported") and the second worked: waking
or restarting the box tries once more on exactly that error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
A quick model looking numbers up pressed 1, 2, 3, 1 for 1234, and 1234 × 5678 = didn't fit in
an 8-action batch (it split it and lost a key). The click and batch descriptions now lead with
the control's name (name "4"), and a batch takes up to 16 actions.
The benchmark gets four tasks on things none of the others use (Mousepad and its save dialog,
a Chromium bookmark, ImageMagick, a page of nested web components), run only when asked for
(--tasks N1,…): a check that the changes carry over to what they weren't tuned on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
`anybot --ask` with ANYBOT_ASK_HERE=1 offers this computer too (headless runs leave it out), so
screen and menu-bar steps can be tried on a Mac without the window. Tried on a Mac mini: the
menu bar's icons listed in one step, a status icon clicked by its name, AppleScript reading the
front app, Tailscale's own CLI answering — all without a screenshot.
An App Store app's CLI joined to `> file` stays in the sandbox and dies; the note now says so
(a bot blamed --json for it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
One model call per step now: a replay step needs none; the quick model's step falls back to the
bot's model when it can't be used (but a too-long chat is folded and asked again, whichever model
asked); each attempt is in the work log. A zoom is a look (safe to do again after a crash).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
Measured on the bot's own box (8 tasks x 2, grok-4.7): 14/16 done, 120 steps, 1114 s before;
16/16, 102 steps, 574 s after (model time 1015 -> 471 s). A replayed skill: 49 s -> 15 s.
Knowing the machine
- facts.rs: 電腦現況 per computer (system, desktop, systemd, sudo, network, package manager,
tools, apps, a fixed TZ), read once by a small read-only probe, kept 30 min, in the
instructions at the start of a turn.
- The box's sudo is the bot's: only what follows sudo is checked against the always-denied
list (it was refused outright, while the instructions said it could sudo), and commands with
sudo run as the box's root (setuid does nothing under gVisor).
- Box image v8 drops ENV TZ=UTC (it overrode /etc/localtime). ANYBOT_BOX_IMAGE /
ANYBOT_BOX_VOLUME give a throwaway box its own image and volume, never adopting the real one.
- Mac shell sandbox: App Store app CLIs (they carry their own app sandbox) and ps/top run
outside it; plain `osascript -e` too, asked the first time even in direct mode, refusing do
shell script, terminals, keystrokes, file reads, ObjC and the secret folders. A command the
sandbox broke says so, instead of looking like a broken program.
Reaching what it sees
- The Mac menu bar's status icons are numbered (each app's AXExtrasMenuBar, named from inside
for the system's own), computer_use_zoom looks closer at full sharpness.
- Numbers stay with their controls from look to look (they were reassigned every look, so a
step's second click landed elsewhere); the box's front window is listed first; a control can
be named instead of numbered (name "4").
- Web page numbering walks open shadow DOM (Chromium settings, web components).
- A box on a Docker host elsewhere retries the next port when the usual one is taken (a retry
without it came up with no network).
Working like a person
- Instructions: pick the route, know what done looks like, one route, check once, say what's
stuck instead of faking it with a substitute, ask before cutting the connection.
- StepPlan: the bot's model takes the first step, the step after trouble (failed, refused,
repeated, nothing changed on screen, the quick model switching routes), one in four, and the
final check; a quick model (Bot.step_model, auto: grok-4.20-0309-non-reasoning on xAI) does
the planned steps in between, falling back to the bot's model if it can't be used.
- Skills keep the steps that worked (with what each acted on, not numbers); replay_skill does
them again with no model in between, finding each control again, handing back on a mismatch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
Two copies on one folder each keep their own idea of the chats and overwrite each other's saves,
and a second copy starting up would take the first one's open turns in the work log for a crash
and close them. The window, `--serve` and `--ask` now hold an OS file lock on anybot.lock in the
data folder (flock; LockFileEx on Windows), and `anybot setup` holds it while it writes the
settings, after stopping the service and before starting it again.
The lock goes with the process however it ends, so a copy that crashed or was killed never blocks
the next one. A new copy waits up to 5 s for one that is closing (a relaunch, a service restart).
Past that, a second window shows a small notice naming the holder's process and the folder, and
`--serve` / `--ask` say it on stderr and exit 1 (launchd's and systemd's restart then retry). Each
ANYBOT_DATA_DIR has its own lock; a screenshot run doesn't take it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
Each bot keeps a work log (oplog/bot-<id>.jsonl, after pi's durable-harness design, cut down):
before something happens a line says it is about to — the turn, each model call, each step once
its checks passed (with the arguments it runs with and whether it may run again), each message
typed mid-turn — synced to disk first. When the turn ends the file shrinks to the messages still
waiting, usually nothing.
On the next start after AnyBot was killed mid-turn, the log puts the chat right before anything
runs: the message the turn answered comes back if the chat lost it; a step that was running and
reaches outside (a text, a click, a file, a published site) shows as 「可能已經執行了」, both to the
person and in the result the bot reads — no longer as "not allowed"; a step that only looked shows
as cut off; messages typed mid-turn that never reached the bot go back into the chat. The chat is
saved, then the log closes the turn. A turn the person didn't stop ends with a 「繼續」 card (what
it was doing, what may have run, the messages put back); it goes on only when they press it — in
the window, on a screen of the brain, or on the phone — from where its steps stand, never running
a step that may have run again. Pressing stop is on disk at once, and a stopped turn stays
stopped.
Also in the turn loop:
- 自動審核 that doesn't answer (or answers something that isn't a verdict) asks the person; in
直接操作 mode it used to let the step through unchecked.
- "Too long for this model" folds older messages into a summary and asks the same step again,
once per thing the person said, instead of ending the turn.
- Folding keeps the person's chat whole: the summary goes in at the fold and the model reads from
the newest summary on (it used to delete what it folded).
- A step moved to the chosen computer runs there, but the chat keeps the call as the model wrote
it; the result says where it ran.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
agent-desktop knows an app by its program's name, but the X11 window says its class: a Snap
Firefox is "firefox_firefox", so the front app was never found and the bot got a bare picture
(「看不出最前面是哪個 App（WINDOW_NOT_FOUND）」). The window's process now gives the name.
When there are no controls at all and the desktop's accessibility switch (org.a11y.Status
IsEnabled) is off — most apps then show none — AnyBot turns it on, as a screen reader would, and
tells the bot that apps already open show their numbers after a restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
Firefox dropped Chrome's DevTools protocol, so bidi.rs speaks WebDriver BiDi and does each
DevTools command the browser tools use the BiDi way (real mouse and keys, typing, screenshots,
PDF, dialogs, uploads, downloads, tabs). The 18 tools themselves are unchanged and pass the same
dashboard test on Firefox as on Chrome.
- A computer with only Firefox runs AnyBot's own profiles in it (under the Snap/Flatpak corner
of the home folder when Firefox is packaged that way).
- The person's own Firefox profiles are choices too, by the names Firefox shows (profiles.ini
and the newer profile switcher's SQLite list): 「阿金（Firefox）」, or just 「阿金」. Firefox
can't be taken over once open, so a profile open the everyday way is refused with "close it
first"; AnyBot then opens it itself, logins intact.
- Settings lists them without a remove button; the messages name Firefox as an option.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
setup.sh offers 「區網 IP」 (alone, or next to Tailscale): Caddy serves https://<LAN IP> with its
own small certificate authority (tls internal, default_sni for clients that send no name) and
hands out the root at http://<LAN IP>/anybot-ca.crt for each device to trust once.
docker-compose.lan.yml binds Caddy to that address only, so it sits beside tailscale serve.
The app now accepts, for the account server, the certificates this computer trusts as well as
the built-in public ones (HTTP and the WebSocket alike), so it connects once the root is
trusted; before that it says what to install instead of "the certificate is wrong".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
AnyBot Dev
daniel wang