daniel.w
/
AnyBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AnyBot/Cargo.toml

83 lines
3.7 KiB
TOML
Raw Normal View History

AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
[package]
name = "anybot"
version = "0.1.0"
edition = "2021"
One-step install: install.sh, fixed Linux and server builds, clearer README - install.sh (curl | sh): app, hands, server and deps modes; installs system packages (right names per release, e.g. libasound2t64 on Ubuntu 24.04), Rust via rustup (1.95+), Docker for the server; limits parallel compiles on small machines - Server Docker image: add the ALSA build/runtime packages (the build failed), cache the cargo registry and target between rebuilds, JOBS build arg - 加一台電腦 now gives a line that installs AnyBot on a blank computer before joining it - Makefile: make deps installs Rust too, make doctor, clear message when Rust is missing/old - README rewritten: one line per use, getting started, update/remove, troubleshooting, deployment options; docs/install.md and multi-device.md follow it - Tested on fresh ubuntu:24.04 (app) and debian:12 (curl | sh hands) containers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 16:52:10 +00:00
# Older Rust stops with a clear message instead of odd errors deep in a dependency.
rust-version = "1.95"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
[[bin]]
name = "anybot"
path = "src/main.rs"
[dependencies]
Replies in Traditional Chinese when that is the setting; a cloud brain's browser is the person's - 語言 繁體中文: a reply's Simplified characters are turned into Traditional with Taiwan wording (zhconv), since some models (Qwen) slip despite the instruction. - A brain on a server: browser steps go to the person's computer (its hands) unless the server's own browser is named; 你的電腦 is listed first. - update_bot says the default computer it set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 06:46:42 +00:00
zhconv = "0.4"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
eframe = "0.37"
ureq = { version = "3", features = ["json"] }
Lasting connection to the account server, per-item sync, files and models via the brain - Each AnyBot keeps one WebSocket to the account server (pinged, reopened within a second when cut). Requests come down it at once; calls to other AnyBots of the account (brain sync, hands) go up it. Frames carry raw bytes, compressed when that saves space. HTTP long-poll stays as fallback. - The account server knows at once when an AnyBot goes away. - Edits to lists and maps go item by item, so two computers editing one list keep both changes. - Screens fetch chat files from the brain, get model lists through the brain's keys, and show the brain's computer as the brain's. - Moving this computer's bots skips starter bots the brain has and bots already moved. - The phone goes to the brain instead of a computer that shows it. - deploy/linux/hands.sh: a Linux computer (with or without a screen) as hands, kept running by systemd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 11:30:32 +00:00
# 長連線 to the account server (WebSocket over the same TLS ureq uses), and compressing what goes over it.
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
HTTPS on the local network's own IP, with a certificate the server signs itself setup.sh offers 「區網 IP」 (alone, or next to Tailscale): Caddy serves https://<LAN IP> with its own small certificate authority (tls internal, default_sni for clients that send no name) and hands out the root at http://<LAN IP>/anybot-ca.crt for each device to trust once. docker-compose.lan.yml binds Caddy to that address only, so it sits beside tailscale serve. The app now accepts, for the account server, the certificates this computer trusts as well as the built-in public ones (HTTP and the WebSocket alike), so it connects once the root is trusted; before that it says what to install instead of "the certificate is wrong". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-07 13:27:07 +00:00
# Which certificates the account server may show: the public ones (built in), plus the ones this
# computer trusts itself, so a server on the local network that signs its own (https://10.0.0.25)
# works once this computer trusts it, the same as for the browser.
webpki-root-certs = "1"
rustls-native-certs = "0.8"
Lasting connection to the account server, per-item sync, files and models via the brain - Each AnyBot keeps one WebSocket to the account server (pinged, reopened within a second when cut). Requests come down it at once; calls to other AnyBots of the account (brain sync, hands) go up it. Frames carry raw bytes, compressed when that saves space. HTTP long-poll stays as fallback. - The account server knows at once when an AnyBot goes away. - Edits to lists and maps go item by item, so two computers editing one list keep both changes. - Screens fetch chat files from the brain, get model lists through the brain's keys, and show the brain's computer as the brain's. - Moving this computer's bots skips starter bots the brain has and bots already moved. - The phone goes to the brain instead of a computer that shows it. - deploy/linux/hands.sh: a Linux computer (with or without a screen) as hands, kept running by systemd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 11:30:32 +00:00
flate2 = "1"
anybot setup and the server setup: guided setup with a brain picker - anybot setup: a full-screen terminal guide (role, name, account server, account, which brain) that saves the settings, installs a systemd user service (with a virtual screen when there is none), and checks that the account server and the brain see the computer. Flags for scripts. - Hands can be limited to one brain; the account server says which AnyBot asks. Choosing a brain in the app does the same. - deploy/server/setup.sh: domain, account, sign-ups and keys step by step; writes .env and Caddyfile, starts everything and checks HTTPS. The server's brain creates the account on first start. - deploy/linux/hands.sh installs what's needed and opens anybot setup (no password kept in a file any more). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 12:38:45 +00:00
# `anybot setup`: the guided setup in a terminal (servers and Linux computers without a window).
crossterm = "0.29"
unicode-width = "0.2"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
serde = { version = "1", features = ["derive"] }
serde_json = "1"
dirs = "7"
image = { version = "0.25", default-features = false, features = ["png", "jpeg", "gif", "webp", "bmp", "tiff", "ico"] }
base64 = "0.22"
People watch and help on the bots' computer through VNC (noVNC), carried by the account server The core: the bot works the computer; a person watches it like a colleague, and takes over when it needs them. VNC is made for that (only what changed, every mouse move and key, touch on phones), so the pictures sent one by one go, and with them their own input path. - vnc.rs: a VNC proxy to the box's x11vnc through one docker exec (a Python bridge that ends with either side). It signs in to the box with the box's own password (its VNC_PASSWORD) and offers the viewer no password, so the password never leaves the brain. The person's clicks and keys make the bot's steps wait until 8 s after they stop; text typed in the viewer's box (any language) is pasted in the box. - Account server: /screen (the viewer page) and /novnc.js (noVNC 1.6.0, MPL-2.0, bundled); /relay/vnc/<host> takes the browser's WebSocket (account session, or a 10-minute ticket the app asks for with POST /relay/screen-ticket; owners and admins) and carries it over that AnyBot's lasting connection (vopen / vdata / vtext / vclose). Both ends of that connection wait 2 ms instead of 40 ms to read while things move: a request and its answer took up to 160 ms more than needed. - Phone page: 看 Bot 的電腦 and 接管 open the viewer page (also on the local network: /api/vnc on the AnyBot's own phone server). This Mac's own screen moved to the ⋯ menu. - Mac app: 看畫面 opens the viewer in the browser (a screen of a brain: the brain's computer by ticket; else this computer's box, noVNC straight from it). The in-app picture viewer, the live JPEG stream, the person's click/move/scroll/key endpoints and the input shell are gone. - Opening a Mac app uses LaunchServices (`open -a`): agent-desktop's launch waited on the window through accessibility and failed without 輔助使用 now and then. Measured on this Mac (made-up account, local relay + headless brain + the real box): viewer signed in in 0.4–0.7 s, a request and its answer through relay + brain + box 25–58 ms median; headless Chrome shows the desktop live in /screen. remote_e2e 28/28 ×2, 149 unit tests, relay 11. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:05:44 +00:00
# VNC sign-in (the 16-byte challenge) for people watching the bots' computer.
des = "0.8"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
chrono = "0.4"
chrono-tz = "0.10"
iana-time-zone = "0.1"
ring = "0.17"
tungstenite = { version = "0.30", default-features = false, features = ["handshake"] }
calamine = "0.38.0"
resvg = "0.49.0"
hayro = "0.9"
zip = { version = "9", default-features = false, features = ["deflate"] }
rodio = { version = "0.23", default-features = false, features = ["playback", "flac", "mp3", "mp4", "vorbis", "wav", "symphonia-aiff", "symphonia-alac", "symphonia-caf"] }
pdf-extract = "0.12"
unicode-normalization = "0.1"
cpal = "0.18"
mermaid-rs-renderer = { version = "0.3", default-features = false }
pulldown-cmark = { version = "0.13", default-features = false }
Add agent-desktop text mode for computer use Read apps as numbered element lists (computer_use_read_app), act by ref (click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0. Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 09:19:59 +00:00
# Seeing and using other apps through the OS accessibility APIs (AX on macOS, AT-SPI on Linux,
# UI Automation on Windows): numbered UI elements a bot can act on without screenshots.
# vercel-labs/agent-desktop, Apache-2.0, pinned to a release.
agent-desktop-core = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
[profile.release]
opt-level = 2
[target.'cfg(not(target_os = "macos"))'.dependencies]
[target.'cfg(target_os = "linux")'.dependencies]
Add agent-desktop text mode for computer use Read apps as numbered element lists (computer_use_read_app), act by ref (click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0. Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 09:19:59 +00:00
agent-desktop-linux = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
x11rb = { version = "0.13", features = ["xtest", "screensaver", "record"] }
# Wayland: wlroots desktops (sway, Hyprland…) through their own protocols; GNOME and KDE through
# the desktop portals (one "allow" dialog, remembered).
wayland-client = "0.31"
Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking
2026-11-05 07:48:22 +00:00
# (wl.rs: SO_RCVTIMEO on the compositor socket, so a frozen compositor can't freeze the app)
libc = "0.2"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
wayland-protocols = { version = "0.32", features = ["client", "staging", "unstable"] }
wayland-protocols-wlr = { version = "0.3", features = ["client"] }
wayland-protocols-misc = { version = "0.3", features = ["client"] }
memmap2 = "0.9"
ashpd = { version = "0.13", default-features = false, features = ["async-io", "remote_desktop", "screencast", "screenshot"] }
zbus = { version = "5", default-features = false, features = ["async-io", "blocking-api"] }
futures-lite = "2"
async-io = "2"
Add agent-desktop text mode for computer use Read apps as numbered element lists (computer_use_read_app), act by ref (click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0. Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 09:19:59 +00:00
[target.'cfg(target_os = "macos")'.dependencies]
agent-desktop-macos = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }
One way to act on a screen: a real click or keystroke, wherever the bot looked - Acting by an app's number (read_app's e7) is now a real click at that control's spot, the same path as screenshot coordinates. agent-desktop only reads; its own actions (a second set with its own key names and failures: no typing in terminals, "/" and "`" failing off the main thread) are gone, with computer_use_select, background clicks and press_key's app. - Every action comes back with the screen after it (or, with no picture allowed, the front app's numbered controls), so the bot no longer spends a step on a screenshot to check. - Coordinates map without remembering a screenshot: the picture's size depends only on the screen. The phone's taps go straight to the screen as the person's, in the pixels of the picture it was sent; they used to borrow the bot's last screenshot scale and landed 1.25x off on the bots' computer, or failed with "screenshot first". - Mac typing pastes (⌘V, clipboard put back): with 注音 on, text sent as key events vanished while return arrived, so commands typed into a terminal ran empty. Bare printing keys ("slash", "r") are typed the same way; keys like "slash" and "grave" have names. - The bots' Docker computer: one docker exec per step (action, settle, picture), never `mousemove --sync` (it waits forever when the pointer is already there: a second click on the same spot hung two minutes), open_app (chromium = the browser browser_* drive, xterm). Tests: box_screen_is_exact (real Docker: bot and phone coordinates exact, same spot twice in under 2 s, ASCII + Chinese + named key typed), Calculator by numbers and Terminal typing under 注音 through the signed app, remote_e2e 28/28, Linux build + 151 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 13:51:19 +00:00
# Typing by pasting (any input method, any language); already in the build through egui.
arboard = { version = "3.7", default-features = false }
Add agent-desktop text mode for computer use Read apps as numbered element lists (computer_use_read_app), act by ref (click/typing/select/scroll/press_key/open_app) via agent-desktop v1.1.0. Verified on macOS (Calculator, Finder) and headless Linux (Xvfb+openbox+AT-SPI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 09:19:59 +00:00
[target.'cfg(target_os = "windows")'.dependencies]
agent-desktop-windows = { git = "https://github.com/vercel-labs/agent-desktop", tag = "v1.1.0" }