daniel.w
/
AnyBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AnyBot/box/start.sh

59 lines
3.3 KiB
Bash
Raw Normal View History

AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
#!/bin/bash
# Boots the desktop: X server, window manager, VNC and the noVNC web viewer.
set -u
mkdir -p /home/bot/.fluxbox /home/bot/Desktop && cp -n /etc/anybot/fluxbox-init /home/bot/.fluxbox/init 2>/dev/null
cp -n /etc/anybot/overlay /home/bot/.fluxbox/overlay 2>/dev/null
cp -n /etc/anybot/Xresources /home/bot/.Xresources 2>/dev/null
cp /etc/anybot/menu /home/bot/.fluxbox/menu 2>/dev/null
chown -R bot:bot /home/bot
rm -f /tmp/.X1-lock /tmp/.X11-unix/X1
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
# One TigerVNC server is the screen and the VNC server (was Xvfb + x11vnc, which polls the
# screen for changes). VNC only on 127.0.0.1 inside the box, with the box's VNC password when the
# app gave one (VNC_PASSWORD), as before.
GEOM="${SCREEN_SIZE%x*}"
Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking
2026-11-05 07:48:22 +00:00
if [ -n "${VNC_PASSWORD:-}" ]; then
mkdir -p /home/bot/.vnc
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
printf '%s\n' "$VNC_PASSWORD" | vncpasswd -f > /home/bot/.vnc/passwd && chmod 600 /home/bot/.vnc/passwd
VNC_SEC="-SecurityTypes VncAuth -rfbauth /home/bot/.vnc/passwd"
Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking
2026-11-05 07:48:22 +00:00
else
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
VNC_SEC="-SecurityTypes None"
Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking
2026-11-05 07:48:22 +00:00
fi
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
Xtigervnc :1 -geometry "$GEOM" -depth 24 -rfbport 5900 -localhost $VNC_SEC -AlwaysShared -AcceptCutText -SendCutText \
-UseBlacklist 0 -IdleTimeout 0 -MaxIdleTime 0 -MaxDisconnectionTime 0 -MaxConnectionTime 0 -s 0 -dpms -desktop "Bot 的電腦" -ac -noreset \
>/tmp/xvnc.log 2>&1 &
XVFB_PID=$!
for i in $(seq 1 50); do xdpyinfo -display :1 >/dev/null 2>&1 && break; sleep 0.1; done
# The session bus and the accessibility registry; tools started later over `docker exec`
# (anybot-a11y, anybot-browser) join this bus through /tmp/anybot-dbus.env.
su bot -c 'cd /home/bot && dbus-launch --sh-syntax > /tmp/anybot-dbus.sh' && . /tmp/anybot-dbus.sh
printf 'DBUS_SESSION_BUS_ADDRESS=%s\n' "$DBUS_SESSION_BUS_ADDRESS" > /tmp/anybot-dbus.env && chmod 644 /tmp/anybot-dbus.env
export DBUS_SESSION_BUS_ADDRESS
for c in /usr/libexec/at-spi-bus-launcher /usr/lib/at-spi2-core/at-spi-bus-launcher; do
[ -x "$c" ] && { su bot -c "DISPLAY=:1 DBUS_SESSION_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS $c --launch-immediately >/tmp/atspi-bus.log 2>&1 &"; break; }
done
for c in /usr/libexec/at-spi2-registryd /usr/lib/at-spi2-core/at-spi2-registryd; do
[ -x "$c" ] && { su bot -c "DISPLAY=:1 DBUS_SESSION_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS $c >/tmp/atspi-registry.log 2>&1 &"; break; }
done
su bot -c "DBUS_SESSION_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS gsettings set org.gnome.desktop.interface toolkit-accessibility true" 2>/dev/null || true
su bot -c "DISPLAY=:1 DBUS_SESSION_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS xrdb -nocpp -merge /home/bot/.Xresources"
su bot -c "DISPLAY=:1 DBUS_SESSION_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS fluxbox >/tmp/fluxbox.log 2>&1 &"
(sleep 1.5; xsetroot -display :1 -solid '#2b2f36') &
xsetroot -display :1 -solid '#2b2f36'
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
websockify --web /usr/share/novnc 6080 localhost:5900 >/tmp/novnc.log 2>&1 &
Docker calls time out; the box has a VNC key, limits and sandboxed files; X11/Wayland eyes and hands stop leaking
2026-11-05 07:48:22 +00:00
WEBSOCKIFY_PID=$!
# Say ready only once the viewer's port listens: the app opens noVNC on this flag, and a too
# early file meant the first viewer load 404'd.
(
for i in $(seq 1 100); do
python3 -c "import socket; socket.create_connection(('127.0.0.1', 6080), 0.5)" 2>/dev/null && break
sleep 0.2
done
echo "anybot-box ready" > /tmp/anybot-box.ready
) &
# When the desktop's heart stops, the container ends too (it used to sit there as Running
# with a dead X server, and every screen action failed forever).
wait $XVFB_PID
kill $WEBSOCKIFY_PID 2>/dev/null
pkill -u bot fluxbox 2>/dev/null
exit 1