daniel.w
/
AnyBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AnyBot/box/Dockerfile

33 lines
2.3 KiB
Docker
Raw Normal View History

AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
# AnyBot's own computer ("Bot 的電腦"): a Linux desktop the bots share, like the original's box
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
# (an X11 desktop driven by xdotool, watched through noVNC on port 6080). One TigerVNC server is
# the screen and the VNC server; AT-SPI is on, so every app's controls (web pages included) can
# be read with their places and numbered on the bot's picture (box/a11y, after XBot's box).
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive LANG=C.UTF-8 TZ=UTC
RUN apt-get update && apt-get install -y --no-install-recommends \
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
tigervnc-standalone-server tigervnc-tools xdotool x11-utils x11-xserver-utils imagemagick fluxbox xterm \
at-spi2-core libatk-adaptor python3-gi gir1.2-atspi-2.0 libglib2.0-bin gsettings-desktop-schemas dconf-gsettings-backend \
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
novnc python3-websockify chromium fonts-noto-cjk fonts-noto-color-emoji \
ca-certificates curl wget git jq unzip zip less nano vim-tiny procps psmisc \
python3 python3-pip python3-venv nodejs npm sudo tini dbus-x11 xclip \
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
&& useradd -m -s /bin/bash bot && echo 'bot ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/bot \
&& ln -sf /usr/share/novnc/vnc.html /usr/share/novnc/index.html
COPY start.sh /usr/local/bin/anybot-box-start
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
COPY a11y /usr/local/bin/anybot-a11y
COPY browser /usr/local/bin/anybot-browser
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
COPY fluxbox-init /etc/anybot/fluxbox-init
COPY overlay /etc/anybot/overlay
COPY Xresources /etc/anybot/Xresources
COPY menu /etc/anybot/menu
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
RUN chmod +x /usr/local/bin/anybot-box-start /usr/local/bin/anybot-a11y /usr/local/bin/anybot-browser \
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
# Chromium has no sandbox inside a container (no user namespaces); the box itself is the
# sandbox. Debian's chromium wrapper reads these flags for every launch (menu, bots, xdg-open).
&& printf '%s\n' 'export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --no-sandbox --disable-dev-shm-usage --no-first-run --no-default-browser-check --disable-gpu --password-store=basic --test-type"' > /etc/chromium.d/anybot
Bots click by number: every control on the screen is boxed and numbered, on every computer A bot asked to open a Threads profile clicked an empty strip ten times: it guessed coordinates from a picture, and the web tools' description told it they didn't work on its own computer. - marks.rs: the picture a bot gets has every button, link, field and menu item boxed and numbered (a tiny built-in digit font, pure Rust), with the same numbers in a list. click / typing / scroll / mouse_move take the number (ref) and land on the control's centre. - Where the controls come from: macOS AX and Linux AT-SPI through agent-desktop on a person's computer (desktop::elements); AT-SPI in the bots' own computer (box/a11y, after XBot's box-a11y), web pages included. On a Mac a control whose spot another app's window covers gets no number (topmost window at its centre must be the app's; the Dock's invisible full-screen window doesn't count). open_app waits until the app is really in front. - No going round in circles: a step that changed nothing says so; the same step (clicks within 16 px) a third time in a turn comes with a warning, from the fourth time it isn't run. - The box (version 4): one TigerVNC server instead of Xvfb + x11vnc, the AT-SPI bus and registry at boot, one browser launcher (box/browser, after XBot's box-chrome: same profile and CDP, renderer accessibility, a launch lock, stale profile locks cleared). One docker exec per look (list + picture, ~0.4 s). A brain updates an older box by itself when it has been idle a minute (files and logins stay); a new box is never made from an older image. - Gone: computer_use_read_app (the list is in every picture), computer_use_mouse_button, the per-call picture for the phone (it gets a plain one), the stale browser_read note; the prompt says web work goes through browser_* by number, and never to read browser cookie or history files. Tests: marks (numbering, drawing, fingerprints), box_numbers_are_exact on a box from the new image (a Threads-like page: the icon links get numbers, Profile clicked by number, Chinese typed into the numbered field, a click on nothing called out), box_screen_is_exact, VNC e2e 11/11 on TigerVNC, Calculator 7×8=56 by numbers through the signed app on this Mac, remote_e2e 28/28, 153 unit tests; Linux build + 157 tests + x11_live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:47:01 +00:00
# Accessibility on for every app, including those started over `docker exec`.
ENV DISPLAY=:1 SCREEN_SIZE=1280x800x24 GTK_MODULES=atk-bridge GTK_A11Y=atspi GNOME_ACCESSIBILITY=1 NO_AT_BRIDGE=0
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
EXPOSE 6080
Box 5: one browser profile, controls read even when Chromium calls its window hidden, the launcher never waits on itself Found on the server's box after the update to 4: - Chromium called its whole window "not showing" (it took itself for minimized) while its page was on the screen; the reader (like XBot's) skipped anything not showing, so a page had no numbers. Now: visible with a place on the screen counts, containers are walked through, and only a page that isn't showing (a background tab) is left out. - Two browser profiles: Chromium's own (~/.config/chromium, where a person logged in by hand) and AnyBot's (~/.config/anybot-browser); the launcher used AnyBot's and Threads showed as logged out. One profile now (anybot-profile → Chromium's own; a box with only AnyBot's keeps that); both folders stay as they were. - The launcher's lock fd went to Chromium, which then held it for its whole life: the next start waited 30 s and failed. And its flags line had been lost in an edit. box_numbers_are_exact passes twice on a box from this image; the launcher answers in 0.09 s when the browser runs; box_screen_is_exact passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:03:59 +00:00
LABEL anybot.box.version="5"
AnyBot: first commit of the working desktop app Rust/egui rebuild of Grok Bot as of 2026-11-04: chat with streaming providers (xAI, OpenAI, Claude, Gemini, OpenRouter, OpenCode Go, Ollama, custom), tools and approvals, the bot's Docker computer, computer use on macOS / Linux X11 / Wayland, AnyBot's browser with a full DevTools-based toolset, the encrypted vault, pinned grid and sidebar sections, Markdown with Mermaid diagrams, phone access through the relay, and the Linux build and test scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uirha3oEtWeHh58zA7ErHa
2026-11-04 08:51:51 +00:00
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["/usr/local/bin/anybot-box-start"]