daniel.w
/
AnyBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AnyBot/box/Dockerfile

41 lines
2.8 KiB
Docker
Raw Normal View History

AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
# AnyBot's own computer ("Bot 的電腦"): a Linux desktop the bots share, like the original's box
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
# (an X11 desktop driven by xdotool, watched through noVNC on port 6080). One TigerVNC server is
# the screen and the VNC server; AT-SPI is on, so every app's controls (web pages included) can
# be read with their places and numbered on the bot's picture (box/a11y, after XBot's box).
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive LANG=C.UTF-8 TZ=UTC
RUN apt-get update && apt-get install -y --no-install-recommends \
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
tigervnc-standalone-server tigervnc-tools xdotool x11-utils x11-xserver-utils imagemagick fluxbox xterm \
at-spi2-core libatk-adaptor python3-gi gir1.2-atspi-2.0 libglib2.0-bin gsettings-desktop-schemas dconf-gsettings-backend \
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
novnc python3-websockify chromium fonts-noto-cjk fonts-noto-color-emoji \
ca-certificates curl wget git jq unzip zip less nano vim-tiny procps psmisc \
Whole control flow checked against real chats: what the bot sees is what is there, every step lands, nothing done twice Web - web_fetch kept pages with a <header> only up to it (taken for <head>): articles came back as their menus. - browser_open waits for the page to load and returns it like browser_read (one step fewer per page); actions wait for the page to hold still instead of fixed 1-2.5 s sleeps; one way to reach the tab. The bot's computer (box v6) - Numbers only on what can be seen: clipped to the window and the page's visible area, none under another app's window (a link scrolled under the toolbar was numbered on Back). - press_key with a typed character no longer swallows the keys after it (xdotool type --args 1). - The person's input through noVNC/TigerVNC is noticed again (fence, continuous updates, QEMU keys). - "Nothing changed" only after clicks, scrolls and drags; long text is pasted; one screen step at a time; one docker inspect per step when the box is known ready; poppler-utils for long PDFs. - One typing path and fewer round trips (stdin variant, unused picture path, extra batch execs gone). The person's computers - Linux: the window system's front app when it has one, and the reason when there is none (was 「」); hidden and off-screen controls get no number; wording no longer names a tool that doesn't exist. The loop - Without Docker, the bot's own computer runs no commands where there is no sandbox (Linux, Windows): it was a plain shell on the brain itself, keys in reach, asked nothing. - The bot's own computer is re-checked each step; unreachable computers are failures, not safety blocks; no direct-mode note on every result; an empty reply is asked again; scrolling and keys aren't counted as going in circles; steps aren't moved to a computer that can't do them. - One finish() for every step's result, keep_grant(), dead appctl list_computers/SERVING and the impossible server branches removed; duplicated prompt instructions merged. Tests: 162 (Mac) / 166 (Linux) unit; real box drills 6/6 on box v6; a brain without Chrome browses on its box; remote E2E 28/28. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 09:44:59 +00:00
python3 python3-pip python3-venv nodejs npm sudo tini dbus-x11 xclip poppler-utils \
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
&& useradd -m -s /bin/bash bot && echo 'bot ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/bot \
&& ln -sf /usr/share/novnc/vnc.html /usr/share/novnc/index.html
COPY start.sh /usr/local/bin/anybot-box-start
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
COPY a11y /usr/local/bin/anybot-a11y
COPY browser /usr/local/bin/anybot-browser
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
COPY fluxbox-init /etc/anybot/fluxbox-init
COPY overlay /etc/anybot/overlay
COPY Xresources /etc/anybot/Xresources
COPY menu /etc/anybot/menu
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
RUN chmod +x /usr/local/bin/anybot-box-start /usr/local/bin/anybot-a11y /usr/local/bin/anybot-browser \
Bot 的電腦 can run under gVisor: its own kernel, Chromium's sandbox back on 容器隔離 in 設定 › 電腦: standard (runc, the container is the only wall, Chromium runs --no-sandbox) or hardened (runsc: a virtualized kernel, all capabilities dropped but the few the desktop needs, ANYBOT_BOX_SANDBOX=1 turns Chromium's own sandbox back on). Box image v7. deploy/mac/gvisor-vm.sh makes a Lima VM with Docker + gVisor for a Mac (started at login), gvisor-migrate.sh moves the box's files and logins there. Checked and finished before committing: - the hardened choice is offered only on a Docker host that has runsc; a box set to it on a host without it opens the standard way and says so (it used to not start at all) - the settings row shows what the box really runs on, and when 更新 is needed - no swap only under gVisor (a standard box keeps Docker's default) - the migrate script sees the running Mac app ("AnyBot", not "anybot") - the VM script starts an existing VM by name, checks runsc and the kernel from the Mac - the version test says 7 Real run in the Lima VM: desktop, VNC, numbered controls, xdotool and screenshots work; no Chromium process has --no-sandbox, renderers run with seccomp and their own namespaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 16:03:25 +00:00
# Chromium has no sandbox inside a plain container (no user namespaces); the box itself is
# the sandbox. Under gVisor (容器隔離=runsc) the box has its own virtualized kernel and
# Chromium's sandbox works: AnyBot then starts the box with ANYBOT_BOX_SANDBOX=1 and the
# sandbox is switched back on (one more wall between a hostile web page and the machine).
# Debian's chromium wrapper reads these flags for every launch (menu, bots, xdg-open).
&& printf '%s\n' \
'if [ "${ANYBOT_BOX_SANDBOX:-0}" = 1 ]; then' \
' export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --disable-dev-shm-usage --no-first-run --no-default-browser-check --disable-gpu --password-store=basic"' \
'else' \
' export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --no-sandbox --test-type --disable-dev-shm-usage --no-first-run --no-default-browser-check --disable-gpu --password-store=basic"' \
'fi' > /etc/chromium.d/anybot
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
# Accessibility on for every app, including those started over `docker exec`.
ENV DISPLAY=:1 SCREEN_SIZE=1280x800x24 GTK_MODULES=atk-bridge GTK_A11Y=atspi GNOME_ACCESSIBILITY=1 NO_AT_BRIDGE=0
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
EXPOSE 6080
Bot 的電腦 can run under gVisor: its own kernel, Chromium's sandbox back on 容器隔離 in 設定 › 電腦: standard (runc, the container is the only wall, Chromium runs --no-sandbox) or hardened (runsc: a virtualized kernel, all capabilities dropped but the few the desktop needs, ANYBOT_BOX_SANDBOX=1 turns Chromium's own sandbox back on). Box image v7. deploy/mac/gvisor-vm.sh makes a Lima VM with Docker + gVisor for a Mac (started at login), gvisor-migrate.sh moves the box's files and logins there. Checked and finished before committing: - the hardened choice is offered only on a Docker host that has runsc; a box set to it on a host without it opens the standard way and says so (it used to not start at all) - the settings row shows what the box really runs on, and when 更新 is needed - no swap only under gVisor (a standard box keeps Docker's default) - the migrate script sees the running Mac app ("AnyBot", not "anybot") - the VM script starts an existing VM by name, checks runsc and the kernel from the Mac - the version test says 7 Real run in the Lima VM: desktop, VNC, numbered controls, xdotool and screenshots work; no Chromium process has --no-sandbox, renderers run with seccomp and their own namespaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha2RVfSYmrhqNnqx9Z4
2026-11-06 16:03:25 +00:00
LABEL anybot.box.version="7"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["/usr/local/bin/anybot-box-start"]