AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
// AnyBot's own browser window, like Grok Bot's own browser: a separate Chrome-family profile that
|
|
|
|
|
|
// AnyBot starts and talks to directly (Chrome's DevTools protocol on a local port), so reading
|
|
|
|
|
|
// and opening pages needs no browser setting ("允許來自 Apple 事件的 JavaScript") and no macOS
|
|
|
|
|
|
// "control this app" prompt. The person logs in to sites in this window once; the profile keeps it.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Profiles (設定檔): the person's own Chrome profiles can't be used (macOS refuses other apps
|
|
|
|
|
|
// Chrome's data folder, and Chrome refuses remote control of its normal folder), so AnyBot's
|
|
|
|
|
|
// browser keeps its own: 「預設」 plus any the person or the bot adds (工作, 私人…). Each is its own
|
|
|
|
|
|
// browser folder and window with its own logins; switching picks which one the bot uses.
|
|
|
|
|
|
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
use serde_json::Value;
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
use std::path::PathBuf;
|
|
|
|
|
|
use std::process::{Command, Stdio};
|
|
|
|
|
|
use std::time::{Duration, Instant};
|
|
|
|
|
|
|
|
|
|
|
|
/// Local port for talking to the bot's browser (estimated; any free local port works).
|
|
|
|
|
|
pub const PORT: u16 = 9333;
|
|
|
|
|
|
/// How long to wait for the browser to come up (estimated).
|
|
|
|
|
|
const START_TIMEOUT_SECS: u64 = 20;
|
|
|
|
|
|
/// The first profile: the browser folder made before profiles existed (its logins stay).
|
|
|
|
|
|
pub const DEFAULT_PROFILE: &str = "預設";
|
|
|
|
|
|
/// The person's own browser (read through accessibility, like Grok Bot's Computer Use helper):
|
|
|
|
|
|
/// whichever Chrome profile they have open there is the one read.
|
|
|
|
|
|
pub const PERSON_PROFILE: &str = "你的瀏覽器";
|
|
|
|
|
|
/// Other ways people name their own browser.
|
|
|
|
|
|
const PERSON_ALIASES: [&str; 6] = ["你的 chrome", "我的 chrome", "我的瀏覽器", "你的chrome", "我的chrome", "my chrome"];
|
|
|
|
|
|
/// Longest profile name (estimated).
|
|
|
|
|
|
pub const PROFILE_NAME_MAX_CHARS: usize = 30;
|
|
|
|
|
|
/// File Chrome writes into its folder with the port it chose (`--remote-debugging-port=0`).
|
|
|
|
|
|
const ACTIVE_PORT_FILE: &str = "DevToolsActivePort";
|
|
|
|
|
|
|
|
|
|
|
|
/// Chrome-family browsers that can be started as AnyBot's own window.
|
|
|
|
|
|
#[cfg(target_os = "macos")]
|
|
|
|
|
|
const CANDIDATES: &[(&str, &str)] = &[
|
|
|
|
|
|
("Google Chrome", "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"),
|
|
|
|
|
|
("Brave Browser", "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"),
|
|
|
|
|
|
("Microsoft Edge", "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"),
|
|
|
|
|
|
("Chromium", "/Applications/Chromium.app/Contents/MacOS/Chromium"),
|
|
|
|
|
|
];
|
|
|
|
|
|
/// Linux: the usual install places (distribution packages, the vendors' .deb/.rpm, Snap, Flatpak).
|
|
|
|
|
|
#[cfg(not(target_os = "macos"))]
|
|
|
|
|
|
const CANDIDATES: &[(&str, &str)] = &[
|
|
|
|
|
|
("Google Chrome", "/usr/bin/google-chrome-stable"),
|
|
|
|
|
|
("Google Chrome", "/usr/bin/google-chrome"),
|
|
|
|
|
|
("Google Chrome", "/opt/google/chrome/chrome"),
|
|
|
|
|
|
("Brave Browser", "/usr/bin/brave-browser"),
|
|
|
|
|
|
("Brave Browser", "/opt/brave.com/brave/brave"),
|
|
|
|
|
|
("Microsoft Edge", "/usr/bin/microsoft-edge"),
|
|
|
|
|
|
("Chromium", "/usr/bin/chromium"),
|
|
|
|
|
|
("Chromium", "/usr/bin/chromium-browser"),
|
|
|
|
|
|
("Chromium", "/snap/bin/chromium"),
|
|
|
|
|
|
("Chromium", "/var/lib/flatpak/exports/bin/org.chromium.Chromium"),
|
|
|
|
|
|
];
|
|
|
|
|
|
|
|
|
|
|
|
pub fn engine() -> Option<(&'static str, &'static str)> {
|
|
|
|
|
|
CANDIDATES.iter().copied().find(|(_, p)| std::path::Path::new(p).exists())
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
pub fn available() -> bool {
|
2026-11-05 07:00:31 +00:00
|
|
|
|
if using_box() {
|
|
|
|
|
|
return true;
|
|
|
|
|
|
}
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
engine().is_some()
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn data_dir() -> PathBuf {
|
|
|
|
|
|
crate::model::State::path().parent().map(PathBuf::from).unwrap_or_else(std::env::temp_dir)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// The 「預設」 profile lives with AnyBot's data (logins stay between runs).
|
|
|
|
|
|
pub fn profile_dir() -> PathBuf {
|
|
|
|
|
|
data_dir().join("browser")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn profiles_root() -> PathBuf {
|
|
|
|
|
|
data_dir().join("browser-profiles")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn dir_for(name: &str) -> PathBuf {
|
|
|
|
|
|
if name == DEFAULT_PROFILE { profile_dir() } else { profiles_root().join(name) }
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// All profiles: 「預設」 first, then the added ones by name.
|
|
|
|
|
|
pub fn profiles() -> Vec<String> {
|
|
|
|
|
|
let mut extra: Vec<String> = std::fs::read_dir(profiles_root())
|
|
|
|
|
|
.map(|it| it.flatten().filter(|e| e.path().is_dir()).filter_map(|e| e.file_name().to_str().map(str::to_string)).collect())
|
|
|
|
|
|
.unwrap_or_default();
|
|
|
|
|
|
extra.sort();
|
|
|
|
|
|
let mut all = vec![DEFAULT_PROFILE.to_string()];
|
|
|
|
|
|
all.extend(extra.into_iter().filter(|n| n != DEFAULT_PROFILE));
|
|
|
|
|
|
all
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn current_file() -> PathBuf {
|
|
|
|
|
|
data_dir().join("browser-profile.txt")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Every choice: AnyBot's own profiles, then the person's own browser.
|
|
|
|
|
|
pub fn choices() -> Vec<String> {
|
|
|
|
|
|
let mut all = profiles();
|
|
|
|
|
|
// Reading the person's own browser goes through macOS accessibility; not on Linux.
|
|
|
|
|
|
if cfg!(target_os = "macos") {
|
|
|
|
|
|
all.push(PERSON_PROFILE.to_string());
|
|
|
|
|
|
}
|
|
|
|
|
|
all
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// The profile the bot uses when it isn't told which.
|
|
|
|
|
|
pub fn current() -> String {
|
|
|
|
|
|
let want = std::fs::read_to_string(current_file()).unwrap_or_default().trim().to_string();
|
|
|
|
|
|
match_profile(&choices(), &want).unwrap_or_else(|| DEFAULT_PROFILE.to_string())
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
pub fn set_current(name: &str) -> Result<String, String> {
|
|
|
|
|
|
let name = resolve(Some(name))?;
|
|
|
|
|
|
std::fs::write(current_file(), &name).map_err(|e| format!("存不了設定檔選擇:{e}"))?;
|
|
|
|
|
|
Ok(name)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Find a profile by name, ignoring case and spaces at the ends.
|
|
|
|
|
|
pub fn match_profile(list: &[String], want: &str) -> Option<String> {
|
|
|
|
|
|
let w = want.trim().to_lowercase();
|
|
|
|
|
|
if w.is_empty() {
|
|
|
|
|
|
return None;
|
|
|
|
|
|
}
|
|
|
|
|
|
if PERSON_ALIASES.contains(&w.as_str()) && list.iter().any(|n| n == PERSON_PROFILE) {
|
|
|
|
|
|
return Some(PERSON_PROFILE.to_string());
|
|
|
|
|
|
}
|
|
|
|
|
|
list.iter().find(|n| n.to_lowercase() == w).cloned()
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Which profile a tool call means: the one named, or the current one.
|
|
|
|
|
|
pub fn resolve(requested: Option<&str>) -> Result<String, String> {
|
2026-11-05 07:00:31 +00:00
|
|
|
|
if using_box() {
|
|
|
|
|
|
return Ok(DEFAULT_PROFILE.to_string());
|
|
|
|
|
|
}
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
match requested.map(str::trim).filter(|s| !s.is_empty()) {
|
|
|
|
|
|
None => Ok(current()),
|
|
|
|
|
|
Some(r) => {
|
|
|
|
|
|
let all = choices();
|
|
|
|
|
|
match_profile(&all, r).ok_or_else(|| format!("AnyBot 瀏覽器沒有叫「{r}」的設定檔。現有:{}。要新的可以用 browser_profiles 新增(action: add)。", all.join("、")))
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// A usable profile name: not empty, not too long, no slashes or dots-only names.
|
|
|
|
|
|
pub fn check_name(name: &str) -> Result<String, String> {
|
|
|
|
|
|
let n = name.trim();
|
|
|
|
|
|
if n.is_empty() {
|
|
|
|
|
|
return Err("請給設定檔名稱".into());
|
|
|
|
|
|
}
|
|
|
|
|
|
if n.chars().count() > PROFILE_NAME_MAX_CHARS {
|
|
|
|
|
|
return Err(format!("設定檔名稱最多 {PROFILE_NAME_MAX_CHARS} 個字"));
|
|
|
|
|
|
}
|
|
|
|
|
|
if n == PERSON_PROFILE || PERSON_ALIASES.contains(&n.to_lowercase().as_str()) {
|
|
|
|
|
|
return Err(format!("「{n}」是留給你自己瀏覽器的名字,請換一個"));
|
|
|
|
|
|
}
|
|
|
|
|
|
if n.contains('/') || n.contains(':') || n.starts_with('.') || n.chars().any(char::is_control) {
|
|
|
|
|
|
return Err("設定檔名稱不能有 / 或 : 也不能用 . 開頭".into());
|
|
|
|
|
|
}
|
|
|
|
|
|
Ok(n.to_string())
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Make a new empty profile (its own folder, so its own logins).
|
|
|
|
|
|
pub fn add(name: &str) -> Result<String, String> {
|
|
|
|
|
|
let n = check_name(name)?;
|
|
|
|
|
|
if let Some(have) = match_profile(&profiles(), &n) {
|
|
|
|
|
|
return Ok(have);
|
|
|
|
|
|
}
|
|
|
|
|
|
std::fs::create_dir_all(profiles_root().join(&n)).map_err(|e| format!("建不了設定檔:{e}"))?;
|
|
|
|
|
|
Ok(n)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Remove an added profile and its logins (the person's choice, from Settings). 「預設」 and the
|
|
|
|
|
|
/// person's own browser can't be removed, and an open window must be closed first.
|
|
|
|
|
|
pub fn remove(name: &str) -> Result<(), String> {
|
|
|
|
|
|
let all = profiles();
|
|
|
|
|
|
let n = match_profile(&all, name).ok_or_else(|| format!("沒有叫「{name}」的設定檔"))?;
|
|
|
|
|
|
if n == DEFAULT_PROFILE || n == PERSON_PROFILE {
|
|
|
|
|
|
return Err(format!("「{n}」不能移除"));
|
|
|
|
|
|
}
|
|
|
|
|
|
if live_port(&n).is_some() {
|
|
|
|
|
|
return Err(format!("「{n}」的視窗還開著,請先關掉它再移除。"));
|
|
|
|
|
|
}
|
|
|
|
|
|
let dir = profiles_root().join(&n);
|
|
|
|
|
|
// Only ever a folder directly inside AnyBot's browser-profiles folder.
|
|
|
|
|
|
if dir.parent() != Some(profiles_root().as_path()) {
|
|
|
|
|
|
return Err("設定檔位置不對,沒有移除".into());
|
|
|
|
|
|
}
|
|
|
|
|
|
std::fs::remove_dir_all(&dir).map_err(|e| format!("移除不了「{n}」:{e}"))?;
|
|
|
|
|
|
if current() == n || !choices().contains(¤t()) {
|
|
|
|
|
|
let _ = std::fs::write(current_file(), DEFAULT_PROFILE);
|
|
|
|
|
|
}
|
|
|
|
|
|
Ok(())
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
pub(crate) fn local() -> ureq::Agent {
|
|
|
|
|
|
ureq::Agent::config_builder().timeout_global(Some(Duration::from_secs(5))).http_status_as_error(false).build().into()
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// The fixed port of 「預設」 (`ANYBOT_BROWSER_PORT` overrides it, for testing).
|
|
|
|
|
|
fn default_port() -> u16 {
|
|
|
|
|
|
std::env::var("ANYBOT_BROWSER_PORT").ok().and_then(|p| p.parse().ok()).unwrap_or(PORT)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn endpoint(port: u16, path: &str) -> String {
|
|
|
|
|
|
format!("http://127.0.0.1:{port}{path}")
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
fn alive(port: u16) -> bool {
|
|
|
|
|
|
port != 0 && local().get(&endpoint(port, "/json/version")).call().map(|r| r.status().as_u16() == 200).unwrap_or(false)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// First line of Chrome's DevToolsActivePort file.
|
|
|
|
|
|
pub fn parse_active_port(text: &str) -> Option<u16> {
|
|
|
|
|
|
text.lines().next()?.trim().parse().ok().filter(|p| *p != 0)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// The port a running window of this profile answers on, if any.
|
|
|
|
|
|
fn live_port(name: &str) -> Option<u16> {
|
|
|
|
|
|
let file = std::fs::read_to_string(dir_for(name).join(ACTIVE_PORT_FILE)).ok().and_then(|t| parse_active_port(&t));
|
|
|
|
|
|
let fixed = (name == DEFAULT_PROFILE).then(default_port);
|
|
|
|
|
|
[fixed, file].into_iter().flatten().find(|p| alive(*p))
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Percent-encode a URL so it can ride in `/json/new?<url>`.
|
|
|
|
|
|
pub fn encode_component(s: &str) -> String {
|
|
|
|
|
|
s.bytes()
|
|
|
|
|
|
.map(|b| match b {
|
|
|
|
|
|
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' | b':' | b'/' => (b as char).to_string(),
|
|
|
|
|
|
_ => format!("%{b:02X}"),
|
|
|
|
|
|
})
|
|
|
|
|
|
.collect()
|
|
|
|
|
|
}
|
|
|
|
|
|
|
2026-11-05 07:00:31 +00:00
|
|
|
|
thread_local! {
|
|
|
|
|
|
/// Browser steps on this thread go to Bot 的電腦's Chromium (its box), not this computer's.
|
|
|
|
|
|
static IN_BOX: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Run `f` with the browser steps on Bot 的電腦's own Chromium.
|
|
|
|
|
|
pub fn in_box<T>(f: impl FnOnce() -> T) -> T {
|
|
|
|
|
|
IN_BOX.with(|b| b.set(true));
|
|
|
|
|
|
let r = f();
|
|
|
|
|
|
IN_BOX.with(|b| b.set(false));
|
|
|
|
|
|
r
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
pub fn using_box() -> bool {
|
|
|
|
|
|
IN_BOX.with(|b| b.get())
|
|
|
|
|
|
}
|
|
|
|
|
|
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
/// Start this profile's window if it isn't running. Returns (browser name, port, just started).
|
|
|
|
|
|
pub fn ensure(profile: &str, first_url: Option<&str>) -> Result<(&'static str, u16, bool), String> {
|
2026-11-05 07:00:31 +00:00
|
|
|
|
if using_box() {
|
|
|
|
|
|
let fresh = !crate::cloudbox::browser_running();
|
|
|
|
|
|
let port = crate::cloudbox::browser_port()?;
|
|
|
|
|
|
if fresh {
|
|
|
|
|
|
if let Some(u) = first_url {
|
|
|
|
|
|
let _ = local().put(&endpoint(port, &format!("/json/new?{}", encode_component(u)))).send_empty();
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
2026-11-05 08:47:28 +00:00
|
|
|
|
return Ok(("它桌面上的 Chromium", port, fresh && first_url.is_some()));
|
2026-11-05 07:00:31 +00:00
|
|
|
|
}
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let Some((name, bin)) = engine() else { return Err("這台電腦沒有 Chrome、Brave、Edge 或 Chromium,沒辦法開 AnyBot 自己的瀏覽器。".into()) };
|
|
|
|
|
|
if profile == PERSON_PROFILE {
|
|
|
|
|
|
return Err("「你的瀏覽器」是你自己的視窗,AnyBot 不會另外開。".into());
|
|
|
|
|
|
}
|
|
|
|
|
|
if let Some(p) = live_port(profile) {
|
|
|
|
|
|
return Ok((name, p, false));
|
|
|
|
|
|
}
|
|
|
|
|
|
let dir = dir_for(profile);
|
|
|
|
|
|
let _ = std::fs::create_dir_all(&dir);
|
|
|
|
|
|
let _ = std::fs::remove_file(dir.join(ACTIVE_PORT_FILE));
|
|
|
|
|
|
// 「預設」 keeps its fixed port; other profiles let Chrome pick a free one and read it back.
|
|
|
|
|
|
let launch_port = if profile == DEFAULT_PROFILE { default_port() } else { 0 };
|
2026-11-05 07:48:22 +00:00
|
|
|
|
// Defense in depth: this becomes a Chrome launch argument. Every caller normalizes to a
|
|
|
|
|
|
// full URL first; a bare "-…" (a Chrome flag) must never get here from a tool argument.
|
|
|
|
|
|
let url = match first_url {
|
|
|
|
|
|
Some(u) if u.starts_with("http://") || u.starts_with("https://") => u,
|
|
|
|
|
|
Some(_) => return Err("第一個網址必須是 http(s) 網址".into()),
|
|
|
|
|
|
None => "about:blank",
|
|
|
|
|
|
};
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let mut c = Command::new(bin);
|
|
|
|
|
|
c.arg(format!("--user-data-dir={}", dir.display()))
|
|
|
|
|
|
.arg(format!("--remote-debugging-port={launch_port}"))
|
|
|
|
|
|
.arg("--remote-allow-origins=http://127.0.0.1")
|
|
|
|
|
|
.arg("--no-first-run")
|
|
|
|
|
|
.arg("--no-default-browser-check")
|
2026-11-05 07:48:22 +00:00
|
|
|
|
.arg(url)
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
.stdin(Stdio::null())
|
|
|
|
|
|
.stdout(Stdio::null())
|
|
|
|
|
|
.stderr(Stdio::null());
|
|
|
|
|
|
#[cfg(unix)]
|
|
|
|
|
|
{
|
|
|
|
|
|
use std::os::unix::process::CommandExt;
|
|
|
|
|
|
c.process_group(0);
|
|
|
|
|
|
}
|
|
|
|
|
|
c.spawn().map_err(|e| format!("打不開 {name}:{e}"))?;
|
|
|
|
|
|
let start = Instant::now();
|
|
|
|
|
|
while start.elapsed() < Duration::from_secs(START_TIMEOUT_SECS) {
|
|
|
|
|
|
if let Some(p) = live_port(profile) {
|
|
|
|
|
|
return Ok((name, p, true));
|
|
|
|
|
|
}
|
|
|
|
|
|
std::thread::sleep(Duration::from_millis(250));
|
|
|
|
|
|
}
|
|
|
|
|
|
Err(format!("{name} 沒有及時打開(AnyBot 瀏覽器,設定檔「{profile}」)。請再試一次。"))
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Open a link in a new tab of the bot's browser (in the named profile, which becomes current).
|
|
|
|
|
|
pub fn open(url: &str, profile: Option<&str>) -> Result<String, String> {
|
|
|
|
|
|
let prof = resolve(profile)?;
|
|
|
|
|
|
let _ = set_current(&prof);
|
|
|
|
|
|
let (name, port, fresh) = ensure(&prof, Some(url))?;
|
|
|
|
|
|
if !fresh {
|
|
|
|
|
|
let r = local().put(&endpoint(port, &format!("/json/new?{}", encode_component(url)))).send_empty().map_err(|e| e.to_string())?;
|
|
|
|
|
|
if r.status().as_u16() >= 400 {
|
|
|
|
|
|
return Err(format!("{name} 沒辦法開新分頁({})", r.status()));
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
// Back when the page has loaded (not after a fixed 3 s: a slow page was still blank then),
|
|
|
|
|
|
// with what it shows — the same as browser_read, one step fewer for every page.
|
|
|
|
|
|
let loaded = crate::browser::wait_loaded(Some(&prof), OPEN_WAIT_MS);
|
|
|
|
|
|
let page = read(PAGE_CHARS_ON_OPEN, Some(&prof)).unwrap_or_default();
|
|
|
|
|
|
let head = format!("已在 AnyBot 瀏覽器({name},設定檔「{prof}」)打開 {url}{}", if loaded { "。" } else { ",還在載入(可以用 browser_wait 等它,再 browser_read)。" });
|
|
|
|
|
|
Ok(if page.is_empty() { head } else { format!("{head}\n\n{page}") })
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
/// Longest wait for a page browser_open opened (estimated).
|
|
|
|
|
|
const OPEN_WAIT_MS: u64 = 12_000;
|
|
|
|
|
|
/// Text of the page handed back by browser_open (browser_read gives more when asked).
|
|
|
|
|
|
const PAGE_CHARS_ON_OPEN: usize = 12_000;
|
|
|
|
|
|
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
/// Bring a profile's window up (Settings button): start it, or bring its front tab forward.
|
|
|
|
|
|
pub fn show(profile: &str) -> Result<(), String> {
|
|
|
|
|
|
let (_, port, fresh) = ensure(profile, None)?;
|
|
|
|
|
|
if fresh {
|
|
|
|
|
|
return Ok(());
|
|
|
|
|
|
}
|
|
|
|
|
|
let list: Value = local().get(&endpoint(port, "/json/list")).call().map_err(|e| e.to_string())?.into_body().read_json().map_err(|e| e.to_string())?;
|
|
|
|
|
|
let id = list.as_array().and_then(|a| a.iter().find(|t| t["type"] == "page")).and_then(|t| t["id"].as_str()).map(str::to_string);
|
|
|
|
|
|
match id {
|
|
|
|
|
|
Some(id) => local().get(&endpoint(port, &format!("/json/activate/{id}"))).call().map(|_| ()).map_err(|e| e.to_string()),
|
|
|
|
|
|
None => local().put(&endpoint(port, "/json/new?about:blank")).send_empty().map(|_| ()).map_err(|e| e.to_string()),
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// The browser_profiles tool: list, add or switch profiles.
|
|
|
|
|
|
pub fn profiles_tool(action: &str, name: &str) -> Result<String, String> {
|
|
|
|
|
|
let list_text = || {
|
|
|
|
|
|
let cur = current();
|
|
|
|
|
|
choices().iter().map(|p| if *p == cur { format!("{p}(使用中)") } else { p.clone() }).collect::<Vec<_>>().join("、")
|
|
|
|
|
|
};
|
|
|
|
|
|
match action.trim() {
|
|
|
|
|
|
"" | "list" => Ok(format!("AnyBot 瀏覽器的設定檔:{}。每個設定檔有自己的登入。「{PERSON_PROFILE}」是使用者平常的瀏覽器:使用者在那裡切到哪個 Chrome 設定檔,就讀哪個。", list_text())),
|
|
|
|
|
|
"add" | "create" => {
|
|
|
|
|
|
let n = add(name)?;
|
|
|
|
|
|
set_current(&n)?;
|
|
|
|
|
|
Ok(format!("已新增並切換到設定檔「{n}」(全新的,還沒登入任何網站)。用 browser_open 打開網站後,請使用者在那個視窗登入一次。現在:{}", list_text()))
|
|
|
|
|
|
}
|
|
|
|
|
|
"use" | "switch" => {
|
|
|
|
|
|
let n = set_current(name)?;
|
|
|
|
|
|
let note = if n == PERSON_PROFILE { "(使用者平常的瀏覽器:請使用者在那裡切到想要的 Chrome 設定檔)" } else { "" };
|
|
|
|
|
|
Ok(format!("已切換到「{n}」{note},之後的 browser_open / browser_read 都用它。現在:{}", list_text()))
|
|
|
|
|
|
}
|
|
|
|
|
|
other => Err(format!("不懂「{other}」,action 請用 list、add 或 use")),
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/// Run JavaScript in the current tab of the bot's browser. Returns (address, result).
|
|
|
|
|
|
pub fn eval_current(profile: Option<&str>, expression: &str) -> Result<(String, Value), String> {
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
let mut tab = crate::browser::Tab::current(profile)?;
|
|
|
|
|
|
let url = tab.eval("location.href")?.as_str().unwrap_or("").to_string();
|
|
|
|
|
|
let v = tab.eval(expression)?;
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
Ok((url, v))
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
const READ_EXPR: &str = "JSON.stringify({t:document.title,u:location.href,x:document.body?document.body.innerText:''})";
|
|
|
|
|
|
|
|
|
|
|
|
/// Title, address and visible text of the current tab of the bot's browser.
|
|
|
|
|
|
pub fn read(max_chars: usize, profile: Option<&str>) -> Result<String, String> {
|
|
|
|
|
|
let prof = resolve(profile)?;
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
let (name, _, _) = ensure(&prof, None)?;
|
|
|
|
|
|
// One connection to the tab for the text and the numbered elements.
|
|
|
|
|
|
let mut tab = crate::browser::Tab::current(Some(&prof))?;
|
|
|
|
|
|
let v = tab.eval(READ_EXPR)?;
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let page: Value = v.as_str().and_then(|s| serde_json::from_str(s).ok()).unwrap_or(Value::Null);
|
|
|
|
|
|
let text: String = page["x"].as_str().unwrap_or("").lines().map(str::trim).filter(|l| !l.is_empty()).collect::<Vec<_>>().join("\n");
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
let title = page["t"].as_str().unwrap_or("").to_string();
|
|
|
|
|
|
let url = page["u"].as_str().unwrap_or("").to_string();
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let mut clipped: String = text.chars().take(max_chars).collect();
|
|
|
|
|
|
if text.chars().count() > max_chars {
|
|
|
|
|
|
clipped.push_str("\n…(後面還有,已截斷)");
|
|
|
|
|
|
}
|
|
|
|
|
|
let login_hint = {
|
|
|
|
|
|
let l = clipped.to_lowercase();
|
|
|
|
|
|
(l.contains("log in") || l.contains("sign in") || clipped.contains("登入")) && clipped.chars().count() < 1500
|
|
|
|
|
|
};
|
|
|
|
|
|
let mut out = format!("瀏覽器:AnyBot 瀏覽器({name},設定檔「{prof}」)\n標題:{title}\n網址:{url}\n\n{clipped}");
|
|
|
|
|
|
// The links, buttons and fields, numbered for browser_click / browser_type / browser_select.
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
if let Ok((_, _, list, _)) = crate::browser::elements(&mut tab) {
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
if !list.is_empty() {
|
|
|
|
|
|
out.push_str("\n\n可以操作的元素(用 browser_click、browser_type、browser_select 等工具的 ref;連結的網址在 → 後面,不要自己猜網址;看不懂時用 browser_screenshot):\n");
|
|
|
|
|
|
out.push_str(&list.join("\n"));
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
if login_hint {
|
|
|
|
|
|
out.push_str("\n\n(這頁看起來要登入。請使用者在 AnyBot 瀏覽器視窗登入一次,之後就會記住。)");
|
|
|
|
|
|
}
|
Real-use fixes: the person sees the screens the bot shows, files reach the phone, no blank-tab loops
From real runs on the cloud brain with the person's Mac (Threads patrol, a post draft, "show me my
screen"):
- computer_use_show: a plain picture of any computer's screen goes into the chat (the bot sees it
too). The bot used to describe a screenshot in words and fail to send it.
- The phone shows the bot's attachments: pictures in place, other files download on a tap (api/file,
only from the chats' file folder). It showed nothing for the bot and names only for the person.
- send_file works from the person's other computers (fetch_file over the relay, 20 MB, secrets stay).
- The same call with the same answer again and again is warned the 3rd time and stopped from the
4th (a blank tab was read 11 times, then the bot said Threads was logged out without opening it);
a blank tab says so; checking a login means opening the site.
- Screenshot lists: a nameless row around a named button isn't numbered, a nameless row takes the
words inside it, empty spacer rows aren't numbered (System Settings: 63 numbers -> 33, all named).
- Writing "in my voice" reads the person's own recent posts first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 11:37:45 +00:00
|
|
|
|
// A blank tab says nothing about a site (a bot read one eleven times and called the person
|
|
|
|
|
|
// logged out of Threads without ever opening it).
|
|
|
|
|
|
if url.is_empty() || url == "about:blank" || url.starts_with("chrome://newtab") {
|
|
|
|
|
|
out.push_str("\n\n(這個分頁是空白的,不代表任何網站的狀態。要看哪個網站(例如有沒有登入),就用 browser_open 打開它。)");
|
|
|
|
|
|
}
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
Ok(out)
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#[cfg(test)]
|
|
|
|
|
|
mod tests {
|
|
|
|
|
|
use super::*;
|
|
|
|
|
|
|
|
|
|
|
|
/// Live check against a running Chrome on ANYBOT_BROWSER_PORT (run by hand with --ignored).
|
|
|
|
|
|
#[test]
|
|
|
|
|
|
#[ignore]
|
|
|
|
|
|
fn reads_a_real_page() {
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
// Opening hands back the loaded page itself (no browser_read needed).
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let r = open("https://example.com", None).unwrap();
|
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice
Web
- web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus.
- browser_open waits for the page to load and returns it like browser_read (one step fewer per page);
actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab.
The bot's computer (box v6)
- Numbers only on what can be seen: clipped to the window and the page's visible area, none under
another app's window (a link scrolled under the toolbar was numbered on Back).
- press_key with a typed character no longer swallows the keys after it (xdotool type --args 1).
- The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys).
- "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a
time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs.
- One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone).
The person's computers
- Linux: the window system's front app when it has one, and the reason when there is none (was 「」);
hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist.
The loop
- Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows):
it was a plain shell on the brain itself, keys in reach, asked nothing.
- The bot's own computer is re-checked each step; unreachable computers are failures, not safety
blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys
aren't counted as going in circles; steps aren't moved to a computer that can't do them.
- One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the
impossible server branches removed; duplicated prompt instructions merged.
Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses
on its box; remote E2E 28/28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
|
|
|
|
assert!(r.contains("example.com") && r.contains("Example Domain") && r.contains("[1] 連結"), "{r}");
|
AnyBot: first commit of the working desktop app
Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming
providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama,
custom), tools and approvals, the bot's Docker computer, computer use on
macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based
toolset, the encrypted vault, pinned grid and sidebar sections, Markdown
with Mermaid diagrams, phone access through the relay, and the Linux
build and test scripts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
|
|
|
|
let page = read(2000, None).unwrap();
|
|
|
|
|
|
assert!(page.contains("Example Domain"), "{page}");
|
|
|
|
|
|
println!("{page}");
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
|
fn urls_are_encoded_for_new_tabs() {
|
|
|
|
|
|
assert_eq!(encode_component("https://www.threads.net/@me?x=1&y=2"), "https://www.threads.net/%40me%3Fx%3D1%26y%3D2");
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
|
fn profiles_are_found_by_name() {
|
|
|
|
|
|
let list = vec![DEFAULT_PROFILE.to_string(), "工作".to_string(), "Personal".to_string()];
|
|
|
|
|
|
assert_eq!(match_profile(&list, " personal ").as_deref(), Some("Personal"));
|
|
|
|
|
|
assert_eq!(match_profile(&list, "工作").as_deref(), Some("工作"));
|
|
|
|
|
|
assert_eq!(match_profile(&list, "私人"), None);
|
|
|
|
|
|
assert_eq!(match_profile(&list, ""), None);
|
|
|
|
|
|
let with_person = vec![DEFAULT_PROFILE.to_string(), PERSON_PROFILE.to_string()];
|
|
|
|
|
|
assert_eq!(match_profile(&with_person, "我的 Chrome").as_deref(), Some(PERSON_PROFILE));
|
|
|
|
|
|
assert_eq!(match_profile(&list, "我的 Chrome"), None);
|
|
|
|
|
|
assert!(check_name("我的 Chrome").is_err());
|
|
|
|
|
|
assert!(check_name(PERSON_PROFILE).is_err());
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
|
fn profile_names_are_checked() {
|
|
|
|
|
|
assert_eq!(check_name(" 工作 ").unwrap(), "工作");
|
|
|
|
|
|
assert!(check_name("").is_err());
|
|
|
|
|
|
assert!(check_name("a/b").is_err());
|
|
|
|
|
|
assert!(check_name("..").is_err());
|
|
|
|
|
|
assert!(check_name(&"字".repeat(PROFILE_NAME_MAX_CHARS + 1)).is_err());
|
|
|
|
|
|
assert!(check_name(&"字".repeat(PROFILE_NAME_MAX_CHARS)).is_ok());
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
|
|
fn chosen_port_is_read_back() {
|
|
|
|
|
|
assert_eq!(parse_active_port("53211\n/devtools/browser/abc\n"), Some(53211));
|
|
|
|
|
|
assert_eq!(parse_active_port("0\n"), None);
|
|
|
|
|
|
assert_eq!(parse_active_port(""), None);
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
}
|