AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
[package]
|
|
|
|
|
name = "anybot"
|
|
|
|
|
version = "0.1.0"
|
|
|
|
|
edition = "2021"
|
One-step install: install.sh, fixed Linux and server builds, clearer README
- install.sh (curl | sh): app, hands, server and deps modes; installs system packages
(right names per release, e.g. libasound2t64 on Ubuntu 24.04), Rust via rustup (1.95+),
Docker for the server; limits parallel compiles on small machines
- Server Docker image: add the ALSA build/runtime packages (the build failed), cache the
cargo registry and target between rebuilds, JOBS build arg
- 加一台電腦 now gives a line that installs AnyBot on a blank computer before joining it
- Makefile: make deps installs Rust too, make doctor, clear message when Rust is missing/old
- README rewritten: one line per use, getting started, update/remove, troubleshooting,
deployment options; docs/install.md and multi-device.md follow it
- Tested on fresh ubuntu:24.04 (app) and debian:12 (curl | sh hands) containers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 16:52:10 +00:00
|
|
|
# Older Rust stops with a clear message instead of odd errors deep in a dependency.
|
|
|
|
|
rust-version = "1.95"
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
|
|
|
|
[[bin]]
|
|
|
|
|
name = "anybot"
|
|
|
|
|
path = "src/main.rs"
|
|
|
|
|
|
|
|
|
|
[dependencies]
|
2026-11-05 06:46:42 +00:00
|
|
|
zhconv = "0.4"
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
eframe = "0.37"
|
|
|
|
|
ureq = { version = "3", features = ["json"] }
|
2026-11-04 11:30:32 +00:00
|
|
|
# 長連線 to the account server (WebSocket over the same TLS ureq uses), and compressing what goes over it.
|
|
|
|
|
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
|
|
|
|
|
webpki-roots = "1"
|
|
|
|
|
flate2 = "1"
|
2026-11-04 12:38:45 +00:00
|
|
|
# `anybot setup`: the guided setup in a terminal (servers and Linux computers without a window).
|
|
|
|
|
crossterm = "0.29"
|
|
|
|
|
unicode-width = "0.2"
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
serde = { version = "1", features = ["derive"] }
|
|
|
|
|
serde_json = "1"
|
|
|
|
|
dirs = "7"
|
|
|
|
|
image = { version = "0.25", default-features = false, features = ["png", "jpeg", "gif", "webp", "bmp", "tiff", "ico"] }
|
|
|
|
|
base64 = "0.22"
|
People watch and help on the bots' computer through VNC (noVNC), carried by the account server
The core: the bot works the computer; a person watches it like a colleague, and takes over
when it needs them. VNC is made for that (only what changed, every mouse move and key, touch on
phones), so the pictures sent one by one go, and with them their own input path.
- vnc.rs: a VNC proxy to the box's x11vnc through one docker exec (a Python bridge that ends
with either side). It signs in to the box with the box's own password (its VNC_PASSWORD) and
offers the viewer no password, so the password never leaves the brain. The person's clicks
and keys make the bot's steps wait until 8 s after they stop; text typed in the viewer's box
(any language) is pasted in the box.
- Account server: /screen (the viewer page) and /novnc.js (noVNC 1.6.0, MPL-2.0, bundled);
/relay/vnc/<host> takes the browser's WebSocket (account session, or a 10-minute ticket the
app asks for with POST /relay/screen-ticket; owners and admins) and carries it over that
AnyBot's lasting connection (vopen / vdata / vtext / vclose). Both ends of that connection
wait 2 ms instead of 40 ms to read while things move: a request and its answer took up to
160 ms more than needed.
- Phone page: 看 Bot 的電腦 and 接管 open the viewer page (also on the local network: /api/vnc on
the AnyBot's own phone server). This Mac's own screen moved to the ⋯ menu.
- Mac app: 看畫面 opens the viewer in the browser (a screen of a brain: the brain's computer by
ticket; else this computer's box, noVNC straight from it). The in-app picture viewer, the
live JPEG stream, the person's click/move/scroll/key endpoints and the input shell are gone.
- Opening a Mac app uses LaunchServices (`open -a`): agent-desktop's launch waited on the window
through accessibility and failed without 輔助使用 now and then.
Measured on this Mac (made-up account, local relay + headless brain + the real box): viewer
signed in in 0.4–0.7 s, a request and its answer through relay + brain + box 25–58 ms median;
headless Chrome shows the desktop live in /screen. remote_e2e 28/28 ×2, 149 unit tests, relay 11.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:05:44 +00:00
|
|
|
# VNC sign-in (the 16-byte challenge) for people watching the bots' computer.
|
|
|
|
|
des = "0.8"
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
chrono = "0.4"
|
|
|
|
|
chrono-tz = "0.10"
|
|
|
|
|
iana-time-zone = "0.1"
|
|
|
|
|
ring = "0.17"
|
|
|
|
|
tungstenite = { version = "0.30", default-features = false, features = ["handshake"] }
|
|
|
|
|
calamine = "0.38.0"
|
|
|
|
|
resvg = "0.49.0"
|
|
|
|
|
hayro = "0.9"
|
|
|
|
|
zip = { version = "9", default-features = false, features = ["deflate"] }
|
|
|
|
|
rodio = { version = "0.23", default-features = false, features = ["playback", "flac", "mp3", "mp4", "vorbis", "wav", "symphonia-aiff", "symphonia-alac", "symphonia-caf"] }
|
|
|
|
|
pdf-extract = "0.12"
|
|
|
|
|
unicode-normalization = "0.1"
|
|
|
|
|
cpal = "0.18"
|
|
|
|
|
mermaid-rs-renderer = { version = "0.3", default-features = false }
|
|
|
|
|
pulldown-cmark = { version = "0.13", default-features = false }
|
2026-11-04 09:19:59 +00:00
|
|
|
# Seeing and using other apps through the OS accessibility APIs (AX on macOS, AT-SPI on Linux,
|
|
|
|
|
# UI Automation on Windows): numbered UI elements a bot can act on without screenshots.
|
|
|
|
|
# vercel-labs/agent-desktop, Apache-2.0, pinned to a release.
|
|
|
|
|
agent-desktop-core = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
|
|
|
|
|
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
[profile.release]
|
|
|
|
|
opt-level = 2
|
|
|
|
|
|
|
|
|
|
[target.'cfg(not(target_os = "macos"))'.dependencies]
|
|
|
|
|
|
|
|
|
|
[target.'cfg(target_os = "linux")'.dependencies]
|
2026-11-04 09:19:59 +00:00
|
|
|
agent-desktop-linux = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
x11rb = { version = "0.13", features = ["xtest", "screensaver", "record"] }
|
|
|
|
|
# Wayland: wlroots desktops (sway, Hyprland…) through their own protocols; GNOME and KDE through
|
|
|
|
|
# the desktop portals (one "allow" dialog, remembered).
|
|
|
|
|
wayland-client = "0.31"
|
2026-11-05 07:48:22 +00:00
|
|
|
# (wl.rs: SO_RCVTIMEO on the compositor socket, so a frozen compositor can't freeze the app)
|
|
|
|
|
libc = "0.2"
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
wayland-protocols = { version = "0.32", features = ["client", "staging", "unstable"] }
|
|
|
|
|
wayland-protocols-wlr = { version = "0.3", features = ["client"] }
|
|
|
|
|
wayland-protocols-misc = { version = "0.3", features = ["client"] }
|
|
|
|
|
memmap2 = "0.9"
|
|
|
|
|
ashpd = { version = "0.13", default-features = false, features = ["async-io", "remote_desktop", "screencast", "screenshot"] }
|
|
|
|
|
zbus = { version = "5", default-features = false, features = ["async-io", "blocking-api"] }
|
|
|
|
|
futures-lite = "2"
|
|
|
|
|
async-io = "2"
|
2026-11-04 09:19:59 +00:00
|
|
|
|
|
|
|
|
[target.'cfg(target_os = "macos")'.dependencies]
|
|
|
|
|
agent-desktop-macos = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
|
One way to act on a screen: a real click or keystroke, wherever the bot looked
- Acting by an app's number (read_app's e7) is now a real click at that control's spot, the
same path as screenshot coordinates. agent-desktop only reads; its own actions (a second set
with its own key names and failures: no typing in terminals, "/" and "`" failing off the main
thread) are gone, with computer_use_select, background clicks and press_key's app.
- Every action comes back with the screen after it (or, with no picture allowed, the front
app's numbered controls), so the bot no longer spends a step on a screenshot to check.
- Coordinates map without remembering a screenshot: the picture's size depends only on the
screen. The phone's taps go straight to the screen as the person's, in the pixels of the
picture it was sent; they used to borrow the bot's last screenshot scale and landed 1.25x off
on the bots' computer, or failed with "screenshot first".
- Mac typing pastes (⌘V, clipboard put back): with 注音 on, text sent as key events vanished
while return arrived, so commands typed into a terminal ran empty. Bare printing keys
("slash", "r") are typed the same way; keys like "slash" and "grave" have names.
- The bots' Docker computer: one docker exec per step (action, settle, picture), never
`mousemove --sync` (it waits forever when the pointer is already there: a second click on
the same spot hung two minutes), open_app (chromium = the browser browser_* drive, xterm).
Tests: box_screen_is_exact (real Docker: bot and phone coordinates exact, same spot twice in
under 2 s, ASCII + Chinese + named key typed), Calculator by numbers and Terminal typing under
注音 through the signed app, remote_e2e 28/28, Linux build + 151 tests + x11_live.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 13:51:19 +00:00
|
|
|
# Typing by pasting (any input method, any language); already in the build through egui.
|
|
|
|
|
arboard = { version = "3.7", default-features = false }
|
2026-11-04 09:19:59 +00:00
|
|
|
|
|
|
|
|
[target.'cfg(target_os = "windows")'.dependencies]
|
|
|
|
|
agent-desktop-windows = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
|