Commit Graph

4 Commits

Author SHA1 Message Date
王性驊 4506cc149b Clippy is clean on macOS and Linux, for the app and the relay
Every warning `cargo clippy --all-targets` gave (50 on macOS, 8 more only on
Linux, 1 in the relay), fixed without changing what anything does:
- long types get a name (Peer, Members, Imported, Joining, Mention, Stream…);
- the two big enum variants are boxed (the vault's login plan, joining an
  account);
- a test now waits for the copy it relaunched (it left a zombie behind);
- a hint's format had a dead `if` whose two branches were both "";
- the rest is what clippy suggests: sort_by_key, as_chunks, is_none_or,
  is_multiple_of, while let, from_ref, next_back, struct init, no casts to
  the same type.

Checked: cargo test (177 macOS, 181 Linux), relay tests (11), remote_e2e
28/28, and the live Linux test (X11; Wayland clicks, keys, Chinese typing,
scroll and screenshots).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-07 10:10:28 +08:00
王性驊 5656d355af Bots click by number: every control on the screen is boxed and numbered, on every computer
A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates
from a picture, and the web tools' description told it they didn't work on its own computer.

- marks.rs: the picture a bot gets has every button, link, field and menu item boxed and
  numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click /
  typing / scroll / mouse_move take the number (ref) and land on the control's centre.
- Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's
  computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's
  box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets
  no number (topmost window at its centre must be the app's; the Dock's invisible full-screen
  window doesn't count). open_app waits until the app is really in front.
- No going round in circles: a step that changed nothing says so; the same step (clicks within
  16 px) a third time in a turn comes with a warning, from the fourth time it isn't run.
- The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry
  at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP,
  renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look
  (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a
  minute (files and logins stay); a new box is never made from an older image.
- Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the
  per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt
  says web work goes through browser_* by number, and never to read browser cookie or history
  files.

Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new
image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed
into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on
TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28,
153 unit tests; Linux build + 157 tests + x11_live.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 00:47:01 +08:00
王性驊 a8265409a1 Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking 2026-11-05 15:48:22 +08:00
王性驊 d7ee3f5db9 AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 16:51:51 +08:00