feat/flow #10
|
|
@ -14,7 +14,7 @@ A self-hosted AI agent workspace. Assign tasks in text or voice, watch the deskt
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
LazyBoy gives each agent its own Linux desktop in Docker — browser, terminal, and files. You can run several agents, put them in a group, turn a demonstration into a skill, and schedule it to run again.
|
LazyBoy gives each agent its own Linux desktop in Docker — browser, terminal, and files. You can run several agents, put them in a group, turn a demonstration into a skill, and schedule it to run again.
|
||||||
|
|
||||||
|
|
@ -24,7 +24,9 @@ This is an early `0.1.0` release with desktop and phone browser UIs. You bring y
|
||||||
|
|
||||||
- **A lasting workspace**: each agent has its own chats, run history, and optional long-term memory.
|
- **A lasting workspace**: each agent has its own chats, run history, and optional long-term memory.
|
||||||
- **A real computer**: open pages, use the terminal, organize files, drive the GUI — and watch it live.
|
- **A real computer**: open pages, use the terminal, organize files, drive the GUI — and watch it live.
|
||||||
|
- **Attachments in chat**: send files or images along with the message; the agent can open them on its own desktop, and inbox copies expire on their own.
|
||||||
- **Take over any time**: sign in, pass a check, or nudge things by hand on the same desktop, then hand it back.
|
- **Take over any time**: sign in, pass a check, or nudge things by hand on the same desktop, then hand it back.
|
||||||
|
- **Saved logins**: keep site credentials in the agent's encrypted vault, so it can fill them in at a login wall without the password ever passing through the model.
|
||||||
- **Several agents and groups**: shared Team computers or private dedicated desktops; `@name` decides who answers, so a message wakes the one agent it is for instead of all of them.
|
- **Several agents and groups**: shared Team computers or private dedicated desktops; `@name` decides who answers, so a message wakes the one agent it is for instead of all of them.
|
||||||
- **Teach by demo, then schedule**: turn a walkthrough into a skill; use cron for repeat work.
|
- **Teach by demo, then schedule**: turn a walkthrough into a skill; use cron for repeat work.
|
||||||
- **Your models and tools**: xAI, OpenCode Go, OpenAI-compatible endpoints, MCP, and file skills.
|
- **Your models and tools**: xAI, OpenCode Go, OpenAI-compatible endpoints, MCP, and file skills.
|
||||||
|
|
@ -69,7 +71,7 @@ For resource limits, environment variables, HTTPS, and in-container sudo, see [O
|
||||||
|
|
||||||
## On a phone
|
## On a phone
|
||||||
|
|
||||||
Desktop and phone share the same web UI. After you deploy on a host the phone can reach, open that URL in the phone browser. `127.0.0.1` is the phone itself — it will not reach another machine.
|
Desktop and phone share the same web UI. The API listens on `127.0.0.1` by default, so set `LAZYBOY_BIND_IP=0.0.0.0` (or one network card's address) in `.env` and recreate the api container before a phone on your network can reach it; off-loopback the login token has to be at least 32 characters. Then open that address in the phone browser — `127.0.0.1` there is the phone itself and will not reach another machine.
|
||||||
|
|
||||||
Tap outside the chat sidebar to collapse it. On the remote desktop you can switch between tap-to-click and trackpad, and use the toolbar for keyboard, right-click, or drag. Put the service behind HTTPS before you expose it; see [Operations](./docs/operations.md#安全模型).
|
Tap outside the chat sidebar to collapse it. On the remote desktop you can switch between tap-to-click and trackpad, and use the toolbar for keyboard, right-click, or drag. Put the service behind HTTPS before you expose it; see [Operations](./docs/operations.md#安全模型).
|
||||||
|
|
||||||
|
|
@ -79,7 +81,7 @@ Tap outside the chat sidebar to collapse it. On the remote desktop you can switc
|
||||||
- **Backend**: Rust 2024, Axum, Tokio
|
- **Backend**: Rust 2024, Axum, Tokio
|
||||||
- **Data**: PostgreSQL, pgvector, SQLx
|
- **Data**: PostgreSQL, pgvector, SQLx
|
||||||
- **Desktop**: Docker, Debian, XFCE, Chromium, Xvfb
|
- **Desktop**: Docker, Debian, XFCE, Chromium, Xvfb
|
||||||
- **Computer control**: CDP, AT-SPI, X11
|
- **Computer control**: Cua Driver over X11, AT-SPI, and Chromium
|
||||||
- **Extensions**: MCP, file skills, demonstration playbooks
|
- **Extensions**: MCP, file skills, demonstration playbooks
|
||||||
|
|
||||||
Flow diagrams, handoff, component roles, and the computer lifecycle live in **[Architecture](./docs/architecture.md)**. The older **[interactive diagram](./docs/workflow.html)** is still there — download it and open it in a browser.
|
Flow diagrams, handoff, component roles, and the computer lifecycle live in **[Architecture](./docs/architecture.md)**. The older **[interactive diagram](./docs/workflow.html)** is still there — download it and open it in a browser.
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,7 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
LazyBoy 讓 Agent 在 Docker 裡使用自己的 Linux 桌面，操作瀏覽器、終端與檔案。你可以建立不同的 Agent、在群組中協作，把示範整理成技能，再安排定時執行。
|
LazyBoy 讓 Agent 在 Docker 裡使用自己的 Linux 桌面，操作瀏覽器、終端與檔案。你可以建立不同的 Agent、在群組中協作，把示範整理成技能，再安排定時執行。
|
||||||
|
|
||||||
|
|
@ -24,7 +24,9 @@ LazyBoy 讓 Agent 在 Docker 裡使用自己的 Linux 桌面，操作瀏覽器
|
||||||
|
|
||||||
- **持續的工作空間**：每個 Agent 有自己的對話、工作紀錄與可設定的長期記憶。
|
- **持續的工作空間**：每個 Agent 有自己的對話、工作紀錄與可設定的長期記憶。
|
||||||
- **真的能操作電腦**：開網頁、使用終端、整理檔案、操作圖形介面，過程可即時觀看。
|
- **真的能操作電腦**：開網頁、使用終端、整理檔案、操作圖形介面，過程可即時觀看。
|
||||||
|
- **附件直接丟進對話**：訊息可帶檔案或圖片，Agent 能在自己的桌面開啟，放在收件匣的複本會自動到期清除。
|
||||||
- **隨時人工接管**：在同一個桌面完成登入、驗證或手動調整，再交回 Agent。
|
- **隨時人工接管**：在同一個桌面完成登入、驗證或手動調整，再交回 Agent。
|
||||||
|
- **登入帳密進保險庫**：帳密加密存在 Agent 自己的保險庫，遇到登入頁由它填寫，密碼不會經過模型。
|
||||||
- **多 Agent 與群組**：支援 Team 共用電腦與 Private 獨立電腦模式；群組裡 @誰就由誰回，沒點名時只叫醒工作內容相關的那個，不會全部出動。
|
- **多 Agent 與群組**：支援 Team 共用電腦與 Private 獨立電腦模式；群組裡 @誰就由誰回，沒點名時只叫醒工作內容相關的那個，不會全部出動。
|
||||||
- **示範教學與排程**：把操作示範整理成技能，使用 cron 安排重複工作。
|
- **示範教學與排程**：把操作示範整理成技能，使用 cron 安排重複工作。
|
||||||
- **自選模型與工具**：支援 xAI、OpenCode Go、OpenAI 相容端點，以及 MCP 與檔案技能。
|
- **自選模型與工具**：支援 xAI、OpenCode Go、OpenAI 相容端點，以及 MCP 與檔案技能。
|
||||||
|
|
@ -69,7 +71,7 @@ make down # 停止服務，保留 PostgreSQL 資料
|
||||||
|
|
||||||
## 在手機上使用
|
## 在手機上使用
|
||||||
|
|
||||||
手機與桌面使用同一個 Web 介面。將服務部署在可連線的主機後，用手機瀏覽器開啟該服務的網址即可；`127.0.0.1` 只代表手機本身，不能拿來連另一台電腦。
|
手機與桌面使用同一個 Web 介面。API 預設只綁 `127.0.0.1`，請在 `.env` 設定 `LAZYBOY_BIND_IP=0.0.0.0`（或指定網卡位址）並重建 api 容器，區網裡的手機才連得到；綁非 loopback 時登入 token 需至少 32 字元。接著用手機瀏覽器開啟該位址——手機上的 `127.0.0.1` 只代表手機本身，不能拿來連另一台電腦。
|
||||||
|
|
||||||
聊天側欄可點外側空白處收合。操作遠端桌面時，可切換直接點選與觸控板模式，使用工具列叫出鍵盤、按右鍵或拖曳。對外提供服務時請設定 HTTPS，詳見 [部署指南](./docs/operations.md#安全模型)。
|
聊天側欄可點外側空白處收合。操作遠端桌面時，可切換直接點選與觸控板模式，使用工具列叫出鍵盤、按右鍵或拖曳。對外提供服務時請設定 HTTPS，詳見 [部署指南](./docs/operations.md#安全模型)。
|
||||||
|
|
||||||
|
|
@ -79,7 +81,7 @@ make down # 停止服務，保留 PostgreSQL 資料
|
||||||
- **後端**：Rust 2024、Axum、Tokio
|
- **後端**：Rust 2024、Axum、Tokio
|
||||||
- **資料**：PostgreSQL、pgvector、SQLx
|
- **資料**：PostgreSQL、pgvector、SQLx
|
||||||
- **桌面**：Docker、Debian、XFCE、Chromium、Xvfb
|
- **桌面**：Docker、Debian、XFCE、Chromium、Xvfb
|
||||||
- **電腦控制**：CDP、AT-SPI、X11
|
- **電腦控制**：Cua Driver（X11、AT-SPI、Chromium）
|
||||||
- **擴充**：MCP、檔案技能、示範 playbook
|
- **擴充**：MCP、檔案技能、示範 playbook
|
||||||
|
|
||||||
流程圖、控制權交接、元件職責與電腦生命週期狀態機，集中在 **[架構與流程](./docs/architecture.md)**。原有的 **[互動流程圖](./docs/workflow.html)** 也保留；下載後用瀏覽器開啟即可操作。
|
流程圖、控制權交接、元件職責與電腦生命週期狀態機，集中在 **[架構與流程](./docs/architecture.md)**。原有的 **[互動流程圖](./docs/workflow.html)** 也保留；下載後用瀏覽器開啟即可操作。
|
||||||
|
|
@ -112,6 +114,7 @@ npm run dev
|
||||||
| [互動流程圖](./docs/workflow.html) | 可縮放、搜尋的 HTML 圖表；下載後開啟 |
|
| [互動流程圖](./docs/workflow.html) | 可縮放、搜尋的 HTML 圖表；下載後開啟 |
|
||||||
| [部署與操作](./docs/operations.md) | 資源、環境變數、安全設定、網站驗證、sudo |
|
| [部署與操作](./docs/operations.md) | 資源、環境變數、安全設定、網站驗證、sudo |
|
||||||
| [AI 使用體驗](./docs/agent-experience.md) | 輪次政策、持久終端機、聊天即時推送 |
|
| [AI 使用體驗](./docs/agent-experience.md) | 輪次政策、持久終端機、聊天即時推送 |
|
||||||
|
| [hermes-agent 比較](./docs/hermes-agent-cua-review.md) | Cua 操作流暢度：與 hermes-agent 對照 |
|
||||||
| [開發指南](./docs/development.md) | 本機開發、檢查與測試、目錄結構 |
|
| [開發指南](./docs/development.md) | 本機開發、檢查與測試、目錄結構 |
|
||||||
| [設定範例](./.env.example) | 環境變數與預設值 |
|
| [設定範例](./.env.example) | 環境變數與預設值 |
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,17 +0,0 @@
|
||||||
# Cua adapter timings (2026-09-07)
|
|
||||||
|
|
||||||
Measured on Apple Silicon, `lazyboy/computer:local`, Cua Driver 0.23.2, inside `make cua-smoke` (`--repeat 10`). These are LazyBoy `controld` HTTP calls, not raw `cua-driver` CLI.
|
|
||||||
|
|
||||||
| Call | n | median ms | max ms |
|
|
||||||
| --- | ---: | ---: | ---: |
|
|
||||||
| `GET /controller/health` | 1 | 11 | 11 |
|
|
||||||
| `POST /observe` | 30 | 101 | 125 |
|
|
||||||
| `POST /act` | 40 | 403 | 744 |
|
|
||||||
| `POST /browser` snapshot | 10 | 48 | 51 |
|
|
||||||
| `POST /browser` click | 20 | 332 | 337 |
|
|
||||||
| `POST /browser` type | 10 | 391 | 399 |
|
|
||||||
| `POST /browser` navigate | 10 | 927 | 941 |
|
|
||||||
|
|
||||||
`/act` includes native click, batched setvalue+click, focus, and one Cua `drag`. Navigate includes the 800 ms settle in the adapter.
|
|
||||||
|
|
||||||
There is no legacy control-plane comparison in this run. Do not treat these numbers as a ship gate against CDP/AT-SPI.
|
|
||||||
|
|
@ -1,159 +0,0 @@
|
||||||
# Cua Driver compatibility (LazyBoy desktop)
|
|
||||||
|
|
||||||
> 歷史紀錄：本文描述 2026-09-07 的雙後端驗證，已非現況。目前只保留 Cua；操作方式見 [operations.md](operations.md)，最新驗證狀態見 [cua-migration-progress.md](cua-migration-progress.md)。
|
|
||||||
|
|
||||||
This report answers one question, from a real `make cua-smoke` run on 2026-09-07:
|
|
||||||
|
|
||||||
> Can Cua Driver reliably control the existing LazyBoy XFCE + Xvfb desktop container?
|
|
||||||
|
|
||||||
**Yes, as an opt-in backend.** A disposable `lazyboy/computer:local` desktop on 2026-09-07 (linux/arm64, Cua Driver 0.23.2) passed raw Driver smoke 10/10, the LazyBoy adapter 10/10, dual-display isolation, and Chromium cookie persistence across `docker pause` and `docker restart`. Production still defaults to `legacy`. See [cua-review.md](cua-review.md) and [cua-benchmark.md](cua-benchmark.md).
|
|
||||||
|
|
||||||
## Environment
|
|
||||||
|
|
||||||
- Image: `lazyboy/computer:local` (`image/computer/Dockerfile`)
|
|
||||||
- Distro: Debian bookworm; Driver binary follows `TARGETARCH` (`linux-arm64` or `linux-x86_64`)
|
|
||||||
- Display: Xvfb `DISPLAY=:1` at 1280×800, XFCE (`xfwm4` compositor off, `xfce4-panel`, `xfdesktop`)
|
|
||||||
- Accessibility: AT-SPI 2 per screen (`at-spi-bus-launcher` + `at-spi2-registryd`)
|
|
||||||
- Browser: Debian `chromium` via `lazyboy-browser` (persistent profile, `--remote-debugging-port=9221+display`, `--force-renderer-accessibility`, `--lang=zh-TW`)
|
|
||||||
- Cua Driver: **0.23.2** (`cua-driver-rs-v0.23.2`; linux-arm64 SHA256 `be22768a207796a4bc1de50c52f32f9ef680b5e86e58c059e02eec2caba2e7bb`, linux-x86_64 SHA256 `01bf8339ec129cc00f4b4b2c6056ef1a7c5b52df39ff83ad17c9b16818aec500`)
|
|
||||||
- Install path: `/usr/local/lib/cua-driver` + `/usr/local/bin/cua-driver` (not under the persisted `/home/lazyboy` bind)
|
|
||||||
- Daemon: `cua-driver serve --grant existing-profile --socket /tmp/lazyboy/cua.sock --no-overlay` on the primary display only
|
|
||||||
- Telemetry: disabled
|
|
||||||
- How to reproduce: `make cua-smoke` (artifacts in `/tmp/lazyboy-cua-smoke-last/`)
|
|
||||||
|
|
||||||
## Doctor
|
|
||||||
|
|
||||||
`cua-driver doctor --json` exit 0, `ok: true`.
|
|
||||||
|
|
||||||
| Probe | Status | Note |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| binary | ok | `cua-driver 0.23.2 (x86_64-linux)` |
|
|
||||||
| install dir | ok | `/usr/local/lib/cua-driver/cua-driver` |
|
|
||||||
| telemetry | ok | disabled |
|
|
||||||
| display server | ok | X11 `DISPLAY=:1` |
|
|
||||||
| X11 connection | ok | connected, visible top-level windows |
|
|
||||||
| AT-SPI | **warn** | CLI `doctor` (docker exec) does not always see the XFCE session bus. The **daemon** started from `lazyboy-screen` does: native `get_window_state` + AT-SPI click/type worked 10/10. |
|
|
||||||
|
|
||||||
`cua-driver status --socket /tmp/lazyboy/cua.sock`: daemon running, permission mode `standard`. Unix socket rejects uid 0; smoke and future controld calls must run as uid 1000 (`lazyboy`).
|
|
||||||
|
|
||||||
## Results (10 consecutive iterations)
|
|
||||||
|
|
||||||
Independent application state, not Cua `"ok"`:
|
|
||||||
|
|
||||||
| Check | Result |
|
|
||||||
| --- | --- |
|
|
||||||
| `cua-driver --version` | `cua-driver 0.23.2` |
|
|
||||||
| screenshot (`get_desktop_state`) | 10/10 PNG of the XFCE desktop |
|
|
||||||
| window / accessibility observation | 10/10 `list_windows` + GTK `get_window_state` |
|
|
||||||
| native click | 10/10 GTK `Smoke Click` wrote `/tmp/lazyboy/cua-smoke-clicked` |
|
|
||||||
| native type | 10/10 GTK entry + `Smoke Save` wrote `hello-cua` |
|
|
||||||
| Chromium attach (existing window/profile) | 10/10 `browser_prepare` `attached_existing_profile` |
|
|
||||||
| browser semantic click / type | 10/10 local `http://127.0.0.1:8765/cua-smoke.html`; DOM became `clicked-ok` then `typed:hello-cua` |
|
|
||||||
| noVNC `:6080` still up | 10/10 |
|
|
||||||
| leftover Cua processes | none (only `cua-driver serve`) |
|
|
||||||
|
|
||||||
Same Chromium **pid 334** / **window_id 29360131** across all ten iterations. `browser_prepare` side effects were all false: no isolated profile, no copy, no restart, no extra remote-debugging toggle (LazyBoy already exposes loopback CDP).
|
|
||||||
|
|
||||||
Element refs are snapshot-scoped (`p1:1`, `p4:1`, … `p28:1`). Reusing an old ref would be wrong; the smoke re-snapshots every action.
|
|
||||||
|
|
||||||
## Relevant tools (0.23.2 `list-tools`)
|
|
||||||
|
|
||||||
Observation / native input used by `controld`: `get_desktop_state`, `list_windows`,
|
|
||||||
`get_window_state`, `click`, `type_text`, `press_key`, `hotkey`, `scroll`, `drag`,
|
|
||||||
`move_cursor`, `set_value`, `bring_to_front`, `get_cursor_position`, `get_screen_size`.
|
|
||||||
|
|
||||||
Browser (attach only): `browser_prepare` (`strategy.kind=existing_profile`,
|
|
||||||
`allow_launch=false`), `get_browser_state` (`semantic_v2`), `browser_navigate`
|
|
||||||
(http/https/about only), `browser_click`, `browser_type`.
|
|
||||||
|
|
||||||
Lifecycle / diagnostics: `start_session`, `end_session`, `health_report`. Skill
|
|
||||||
teaching also uses `start_recording` / `stop_recording`.
|
|
||||||
|
|
||||||
Not used: `mouse_button_down`, `mouse_button_up`, `mouse_drag` (held-button
|
|
||||||
background X11 tools). LazyBoy's action DSL has no partial-pointer state, so
|
|
||||||
`Pointer{Down}` / `Pointer{Up}` translate to `ControlError::Unsupported` instead
|
|
||||||
of a half-pressed button nobody releases. Also unused: isolated `launch_app`
|
|
||||||
browsers, Wayland helpers, and the deprecated `get_session_state` /
|
|
||||||
`escalate_session` aliases. None of these names may be exposed to the LLM.
|
|
||||||
|
|
||||||
## Driver contract rules (enforced in `crates/control/src/cua`)
|
|
||||||
|
|
||||||
Each of these was confirmed against a real 0.23.2 daemon, and each one fails
|
|
||||||
silently (exit 0) if violated:
|
|
||||||
|
|
||||||
1. **Repeat the `session` label on every call.** `CuaClient::call` injects
|
|
||||||
`lazyboy-<display>` unless the caller already set one. Without it each CLI
|
|
||||||
process gets an ephemeral `cli-<uuid>` session, so trajectory turns,
|
|
||||||
snapshots, and browser binds never line up, and every call also emits a bogus
|
|
||||||
`end_session` turn.
|
|
||||||
2. **Never send `target: {kind: "desktop", display_id: "primary"}`.** The Linux
|
|
||||||
driver rejects it with `invalid_action_target` (exit 0). Omit `target` to use
|
|
||||||
the global input route.
|
|
||||||
3. **Desktop `scroll` needs a point.** With `scope: "desktop"`, `x`/`y` are
|
|
||||||
required (`missing field x`); `dispatch` aims at the pointer and falls back to
|
|
||||||
the screen centre. `amount` is clamped to the schema range `1..=50`.
|
|
||||||
4. **Only a JSON object proves the tool ran.** A refusal or prose banner that
|
|
||||||
arrives with exit 0 is a failure (`decode_stdout`), never an empty success.
|
|
||||||
5. **One escalation retry.** `background_unavailable` carries
|
|
||||||
`escalation.recommended`; the client retries once with that `delivery_mode`
|
|
||||||
and never loops.
|
|
||||||
6. **`get_window_state` can be degraded.** AT-SPI intermittently answers with
|
|
||||||
`degraded: true` and a root-only tree. Such windows are skipped and the
|
|
||||||
observation reports `native_observation_complete: false` rather than failing
|
|
||||||
the whole `observe`.
|
|
||||||
7. **Nothing is validated for you.** The Linux schemas declare
|
|
||||||
`additionalProperties: false` and numeric bounds, but 0.23.2 accepts unknown
|
|
||||||
keys and out-of-range values anyway (a bogus key on `list_windows` and
|
|
||||||
`scroll amount: 0` both return exit 0 with `effect: unverifiable`), so the
|
|
||||||
bounds in `docs/cua-schemas/0.23.2/` are enforced here, by the client.
|
|
||||||
`session` is accepted by every tool, including the ones whose own schema
|
|
||||||
omits it (`list_windows`, `bring_to_front`, `health_report`,
|
|
||||||
`start_recording`), which is what lets rule 1 be applied uniformly.
|
|
||||||
8. **Socket peer uid.** `/tmp/lazyboy/cua*.sock` rejects uid 0; `controld` runs as
|
|
||||||
the desktop user (uid 1000).
|
|
||||||
|
|
||||||
## Integration notes for the next PR
|
|
||||||
|
|
||||||
- Call Cua as uid 1000 via `cua-driver call --socket /tmp/lazyboy/cua.sock`. Root is rejected (`reject Unix peer uid 0 for runtime owned by uid 1000`).
|
|
||||||
- `get_browser_state` on a live LazyBoy Chromium first returns `browser_consent_required` / `consumer_profile_endpoint_requires_grant`. Then `browser_prepare` with `existing_profile` attaches. Serve must keep `--grant existing-profile`. Never `allow_launch`.
|
|
||||||
- Linux Chromium trusted CDP pointer is unavailable; smoke used `browser_click` `input_route=dom_event` and verified the DOM. Production adapter should prefer that route on this platform and treat `browser_input_trust_unavailable` as classified, not a silent xdotool fallback.
|
|
||||||
- Extra Team screens are extra Xvfb `DISPLAY`s. This POC only runs a daemon on `:1`. Later: one socket per slot.
|
|
||||||
- `browser_navigate` refuses `file://`; local fixtures need `http://127.0.0.1`.
|
|
||||||
- `get_window_state` on Linux is `additionalProperties: false` — do not send macOS-only fields such as `include_accessibility_tree`.
|
|
||||||
|
|
||||||
## Known limits
|
|
||||||
|
|
||||||
- Doctor AT-SPI warn from a non-desktop D-Bus is not a daemon failure.
|
|
||||||
- Overlay warnings (`X11 channel rejected command`) appeared in the daemon log with `--no-overlay`; they did not block actions.
|
|
||||||
- Debian Chromium + zh-TW UI: existing-profile attach worked because CDP was already open, so Cua did not need the English setup-checkbox path.
|
|
||||||
- Multi-screen, pause/resume, takeover, and skill recording were **not** in this POC; later PRs added controller routing, browser attach, takeover re-observe, and dual-source skill recording.
|
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
Cua Driver 0.23.2 **can** control the existing LazyBoy XFCE + Xvfb container: screenshot, window/AT-SPI observation, native click/type, and Chromium semantic click/type, 10/10, without replacing the browser profile or breaking noVNC.
|
|
||||||
|
|
||||||
`ComputerController` is in place. Production defaults to `legacy` pending the full acceptance suite and benchmark. Set `LAZYBOY_COMPUTER_DRIVER=cua` only for explicit testing. Set `LAZYBOY_COMPUTER_DRIVER=legacy` on the supervisor (passed into each desktop container) to roll back to CDP/AT-SPI/xdotool. Recreate desktop containers after changing the flag.
|
|
||||||
|
|
||||||
With `cua`: `POST /observe`, `POST /act`, and `POST /browser` go through Cua Driver. The Agent-facing `browser` schema is unchanged (`snapshot` / `click` / `type` / `press` / `navigate` / `wait`); Cua attaches with `existing_profile` and maps `semantic_v2` refs (`pN:M`) onto the existing element list. After human takeover ends, the run forces a fresh `computer_observe` and drops pre-handoff ids/refs. Skill teaching starts Cua `start_recording` (no video) plus the existing CDP DOM recorder so a human noVNC demo still yields semantic click/type/navigate events; Cua trajectory turns are ingested as extra evidence and password-labelled typing is masked. `use_saved_login` still fills via CDP stdin so passwords never appear on argv. `cdp.py` / AT-SPI remain for login fill, human browser recording, and the `legacy` rollback.
|
|
||||||
|
|
||||||
|
|
||||||
## Review of the current checkout (2026-09-07)
|
|
||||||
|
|
||||||
The locally tagged `lazyboy/computer:local` image now installs Cua Driver 0.23.2
|
|
||||||
for the build architecture (`cua-driver 0.23.2` on linux/arm64 in this run).
|
|
||||||
`make cua-smoke` is the acceptance entry: raw Driver smoke, adapter E2E,
|
|
||||||
isolation, and pause/restart persistence. Production defaults remain `legacy`.
|
|
||||||
See [the migration audit](cua-review.md).
|
|
||||||
|
|
||||||
One upstream caveat about that persistence claim: Chromium writes its cookie
|
|
||||||
database on a ~30 s timer and does not flush on `SIGTERM`. A profile survives
|
|
||||||
`docker pause` / `docker restart` once that write has landed; stopping a desktop
|
|
||||||
seconds after a login can still lose the cookie, and no LazyBoy code controls
|
|
||||||
the timer. `scripts/cua-smoke-test.sh` waits for the fixture cookie to reach the
|
|
||||||
profile before it restarts, so the check measures profile persistence instead of
|
|
||||||
the flush timer.
|
|
||||||
|
|
||||||
Image architectures are capped at `linux/amd64` and `linux/arm64` by upstream
|
|
||||||
binaries: the Cua Driver ships only `linux-x86_64` / `linux-arm64` and ONNX
|
|
||||||
Runtime only `linux-x64` / `linux-aarch64`. See
|
|
||||||
[development.md](development.md#映像與-cpu-架構).
|
|
||||||
|
|
@ -1,58 +0,0 @@
|
||||||
# Cua-only migration verification
|
|
||||||
|
|
||||||
Verified locally on 2026-09-08 (Linux arm64, Cua Driver 0.23.2). Source changes and local acceptance are complete. Production deployment is not part of this verification.
|
|
||||||
|
|
||||||
The subsequent named-cursor feature, Chinese badge font, and its separate image
|
|
||||||
verification are documented in [Agent cursor on the shared desktop](agent-cursor.md).
|
|
||||||
|
|
||||||
## Requirements and evidence
|
|
||||||
|
|
||||||
| Requirement | Implementation | Verification |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Remove the old control implementation | Cua is the only ComputerDriver. Deleted LegacyController, direct CDP/AT-SPI Python controllers, clipboard.py, process helpers and the tmux shell. Removed xdotool/xclip packages. | Source audit; old backend names rejected; final running image contains neither binary. |
|
|
||||||
| See what Cua is doing | Actions foreground the existing browser/native window on the bot's shared display. Shell/file tools use visible named terminals; clipboard operations use a visible GTK editor. | Native/browser/noVNC smoke; terminal and clipboard integration; actual VNC demonstration. |
|
|
||||||
| All computer actions through Cua | Browser, native pointer/key/ref actions, launch/focus, shell/file tools, clipboard and saved-login use the Cua controller. Removed blanket browser-pixel blocking so canvas/unsupported controls can use fresh screenshot coordinates through Cua. | Adapter, terminal, clipboard, login and dual-display tests; API tool-path audit found no hidden shell execution in agent computer/file tools. |
|
|
||||||
| Time on every conversation record | Every persisted message renders MessageTime, including attachment/chip messages. Date/time includes seconds, ISO datetime and full local-time tooltip. | Real user and assistant error messages inspected at desktop and 500px width; frontend tests/typecheck/build. |
|
|
||||||
| Faster/reliable connections | Reuse browser bindings; start VNC before Cua; record the daemon PID; do not inherit startup locks; skip repeated network attachment; reject missing networks before connect and fall back to host ports; recover expired Cua sessions for reads without replaying mutations. | Warm ensure reuses one daemon with an available lock; missing-network fallback leaves no stale attachment; session-expiry observation/browser recovery and mutation refusal pass. |
|
|
||||||
|
|
||||||
Infrastructure provisioning/storage/database calls and independent connected-service MCP facilities remain. They are not alternate desktop controllers. Computer, browser, terminal and agent workspace-file interactions use Cua.
|
|
||||||
|
|
||||||
## Acceptance results
|
|
||||||
|
|
||||||
Final desktop image `lazyboy/computer:cua-work`:
|
|
||||||
`sha256:4ad46205023694a26ff06f9e969d7c7e0ac43b98cae97a4483d2ba3379462117`
|
|
||||||
|
|
||||||
- `cargo test --workspace`: 204 passed; the environment-dependent login integration is ignored by default and separately passed.
|
|
||||||
- `cargo clippy --workspace --all-targets -- -D warnings`: passed.
|
|
||||||
- `cargo fmt --all --check` and `git diff --check`: passed.
|
|
||||||
- Frontend: 44 tests passed; TypeScript/Vite build passed.
|
|
||||||
- `scripts/cua-smoke-test.sh --docker --repeat 1 --image lazyboy/computer:cua-work`: passed on the final image. Covers native/browser/noVNC, terminal persistence/Unicode/interrupt/reset, clipboard Unicode/multiline/copy, two-display isolation, session expiry, and browser-cookie persistence across pause/restart.
|
|
||||||
- `COMPUTER_IMAGE=lazyboy/computer:cua-work scripts/cua-login-test.sh`: passed. Uses a trusted local HTTPS fixture in a disposable container and the production field-filling function, checks both exact values and verifies no submission.
|
|
||||||
- Manual VNC demonstration switched the target desktop from Chromium to its terminal. Teaching retained 2 window events and 3 screenshots, including the final frame. The target's recording was driven by human-style VNC input, not target-side Cua action calls.
|
|
||||||
|
|
||||||
Local measurements are samples, not production benchmarks: boot request 2.358s; warm screen URL request 0.100s; missing-network fallback 0.086s with unchanged Docker network attachments. A full container replacement/restart took 10.419s, including Docker shutdown.
|
|
||||||
|
|
||||||
## Driver compatibility fixes
|
|
||||||
|
|
||||||
- A zero CLI exit code is insufficient: `effect: refused` is treated as failure along with `status: refused`.
|
|
||||||
- Email input can refuse Cua browser typing. Its fallback resolves one uniquely labelled visible native web entry through Cua, clicks the fresh observed bounds, selects all and pastes through Cua. Duplicate labels, browser chrome and zero-size fields are rejected.
|
|
||||||
- Native type_text loses Unicode in terminals. Shell commands use ASCII Bash literals encoding UTF-8 bytes; general Unicode/multiline paste uses the Cua-operated clipboard editor. Zsh confirms multiline bracketed paste with another Enter.
|
|
||||||
- The GTK helper exposes exact clipboard text through its accessibility label because the pinned driver does not return GTK entry values.
|
|
||||||
- Expired driver sessions are revived for observations. Mutations rejected at expiry are not replayed. Expired browser bindings are classified as stale and re-bound for read requests.
|
|
||||||
- Browser semantic clicks use `dom_event`; callers still inspect results. Unsupported controls can be operated with Cua coordinates from a fresh screenshot.
|
|
||||||
|
|
||||||
## Teaching and environment limits
|
|
||||||
|
|
||||||
Cua trajectories record driver invocations, not raw human VNC clicks/keys. Teaching retains window changes and visual keyframes. The start message and model prompt describe that accurately and require review of missing/ambiguous steps. Model failure no longer claims the skill was learned. The local environment has no real model key, so model-generated playbook quality was not tested; recording persistence and missing-key handling were verified.
|
|
||||||
|
|
||||||
Local API: `http://127.0.0.1:3111`; supervisor7191. The test bot `1ae00840-ceaf-4197-957d-661df677b015` is running the final image with one Cua daemon. Generated test credentials are in the local .env; no real provider credentials were used. API login sessions are in-memory, so restarting the API requires signing in again (existing behavior).
|
|
||||||
|
|
||||||
The disposable Postgres test database uses tmpfs on15434; the pre-existing compose database volume was left intact. The separate old `lazyboy-cua-verify` container is a UI-test viewer, not the final target desktop. Temporary smoke/login containers are removed by their scripts.
|
|
||||||
|
|
||||||
## Local evidence files
|
|
||||||
|
|
||||||
- `/tmp/lazyboy-workspace-tests.log`, `/tmp/lazyboy-workspace-clippy.log`
|
|
||||||
- `/tmp/lazyboy-acceptance-smoke.log`, `/tmp/lazyboy-acceptance-login.log`
|
|
||||||
- `/tmp/lazyboy-network-fallback-test.log`, `/tmp/lazyboy-session-recovery-test.log`
|
|
||||||
- `/tmp/lazyboy-chat-time.png`, `/tmp/lazyboy-chat-mobile.png`, `/tmp/teach-vnc.png`
|
|
||||||
- `/tmp/lazyboy-web-build-final.log`, `/tmp/lazyboy-final-frontend-tests.log`
|
|
||||||
|
|
@ -1,55 +0,0 @@
|
||||||
# Cua 遷移檢查（2026-09-07）
|
|
||||||
|
|
||||||
> 歷史紀錄：本文描述 2026-09-07 的雙後端驗證，已非現況。目前只保留 Cua；操作方式見 [operations.md](operations.md)，最新驗證狀態見 [cua-migration-progress.md](cua-migration-progress.md)。
|
|
||||||
|
|
||||||
結論：`a.md` Phase 1 規格尚未全部勾完，但 **opt-in Cua 已可在現有 XFCE + Xvfb 桌面容器使用**。生產預設仍是 `legacy`。
|
|
||||||
|
|
||||||
本次在 Apple Silicon（linux/arm64）上以 `lazyboy/computer:local` + Cua Driver **0.23.2** 重跑隔離桌面驗收。
|
|
||||||
|
|
||||||
## 現況
|
|
||||||
|
|
||||||
| 規格 | 狀態 |
|
|
||||||
| --- | --- |
|
|
||||||
| 雙後端、Agent schema 不變 | 完成。`LAZYBOY_COMPUTER_DRIVER=legacy`（預設）或 `cua`。 |
|
|
||||||
| Docker 安裝 | 完成。Dockerfile 依 `TARGETARCH` 安裝 linux-arm64 / linux-x86_64，checksum 固定。 |
|
|
||||||
| `computer_observe` | 完成。截圖 + native AT-SPI 元素（`kind=a11y`、snapshot-scoped `cua:…` handle）+ 視窗列表。 |
|
|
||||||
| `computer_act` | 完成。pointer / type / key / scroll / focus / 單次 `drag`；native ref 走 Cua `click` / `set_value`。不支援的動作明確 `Unsupported`，不再偷偷回退 legacy。 |
|
|
||||||
| Browser | 完成。`existing_profile` attach、`semantic_v2` refs、navigate 限 http/https/about。 |
|
|
||||||
| 多螢幕隔離 | 完成。display `:1` 的 handle 送到 `:2` 會被拒絕；各自點擊只改自己的測試程式。 |
|
|
||||||
| pause / restart 後 Chromium cookie | 完成。adapter `--check-persistence` 在 `docker pause` 與 `docker restart` 後都通過。 |
|
|
||||||
| noVNC | smoke 確認 `:6080` 仍可連；完整 human takeover 端到端未另開測試。 |
|
|
||||||
| 示範錄製 | Cua `start_recording` + 既有 CDP recorder 仍在；未做真人 noVNC 示範驗收。 |
|
|
||||||
| Benchmark 文件 | 見 [cua-benchmark.md](cua-benchmark.md)。尚未對 legacy 做對照。 |
|
|
||||||
| 預設切到 Cua | **未做。** 完整 DoD（takeover、錄製、生產 metrics）未過前維持 legacy。 |
|
|
||||||
|
|
||||||
## 這輪修正
|
|
||||||
|
|
||||||
- Native 同一批 `computer_act` 共用一份觀察快照；先前每個動作都把 handle map 拿掉，導致 `wait` 後面的 ref 或連續兩個 ref 被當成過期。
|
|
||||||
- 拒絕的過期 ref 不再清掉該螢幕上其他仍有效的 handle。
|
|
||||||
- JSON `code != ok` 即使行程成功碼為 0 也當失敗（避免 drag 誤報成功）。
|
|
||||||
- CLI JSON 改走 stdin，避免輸入文字出現在 argv。
|
|
||||||
- 拖曳改打最小包含該點的視窗（避免點到覆蓋其上的 Chromium），focus 優先精確標題（避免 `LazyBoy Cua Smoke - Chromium` 搶走 GTK 視窗）。
|
|
||||||
- GTK 測資把 drawing area 座標轉成螢幕座標。
|
|
||||||
- Cua 選到但 binary 不在或 daemon 起不來會明確失敗。
|
|
||||||
|
|
||||||
## 驗證（本機 2026-09-07）
|
|
||||||
|
|
||||||
- `cargo test --locked -p lazyboy-control`：84 通過。
|
|
||||||
- `cargo check --locked -p lazyboy-api -p lazyboy-sandbox -p lazyboy-controld`：通過。
|
|
||||||
- `make cua-smoke`（`--repeat 10`）：
|
|
||||||
- 原始 Driver smoke **10/10**（截圖、視窗、native click/type、Chromium attach、noVNC）
|
|
||||||
- LazyBoy adapter **10/10**（GTK click / 中文 setvalue、過期 handle 拒絕、批次 native、drag、Chromium 表單）
|
|
||||||
- 雙 display 隔離通過
|
|
||||||
- pause/unpause 與 `docker restart` 後 cookie 仍在
|
|
||||||
|
|
||||||
## 如何啟用
|
|
||||||
|
|
||||||
預設不要改。要在本機明確跑 Cua：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make cua-smoke
|
|
||||||
# 或
|
|
||||||
docker compose -f docker-compose.yml -f docker-compose.cua.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
overlay 把 supervisor 的 `LAZYBOY_COMPUTER_DRIVER` 設成 `cua`，桌面映像標成 `lazyboy/computer:cua`，不會覆寫預設的 `lazyboy/computer:local` legacy 映像。改 flag 後必須重建桌面容器。
|
|
||||||
|
|
@ -38,7 +38,6 @@ python3 tests/log-rotation.test.py
|
||||||
|
|
||||||
# Cua Driver 能否控制現有 XFCE + Xvfb 桌面（會建 computer image）
|
# Cua Driver 能否控制現有 XFCE + Xvfb 桌面（會建 computer image）
|
||||||
make cua-smoke
|
make cua-smoke
|
||||||
# 結果摘要見 docs/cua-compatibility.md、docs/cua-review.md
|
|
||||||
# 生產路徑只使用 Cua；更新後請重建 supervisor 與桌面映像。
|
# 生產路徑只使用 Cua；更新後請重建 supervisor 與桌面映像。
|
||||||
|
|
||||||
# Python 整合測試用 docker compose exec 連進 Postgres，自己建一次性資料庫後清掉
|
# Python 整合測試用 docker compose exec 連進 Postgres，自己建一次性資料庫後清掉
|
||||||
|
|
@ -105,6 +104,31 @@ builder 並註冊 binfmt；主機本身的架構會跳過註冊，否則在特
|
||||||
不推送 registry 時輸出 OCI archive（`nerdctl load -i <檔>` 可載入），因為 Docker 的
|
不推送 registry 時輸出 OCI archive（`nerdctl load -i <檔>` 可載入），因為 Docker 的
|
||||||
`docker` driver 無法匯出 manifest list。
|
`docker` driver 無法匯出 manifest list。
|
||||||
|
|
||||||
|
### README 首頁圖
|
||||||
|
|
||||||
|
首頁圖不再手刻假介面：先用 `scripts/capture-hero.mjs` 拍**正在跑的真實介面**，再由
|
||||||
|
`docs/hero.html` 套上品牌框輸出 `docs/readme-hero.png`。兩段都只要有任一 Chromium
|
||||||
|
（`CHROME=` 指定，否則依序找 PATH 上的 chromium／Chrome 與 Playwright 下載的 chrome）。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 對運行中的 stack 擷取真實畫面（登入 token 讀 .env，可用 LB_TOKEN／LB_URL 覆寫）
|
||||||
|
# 群組視圖的右欄是機器人自己的瀏覽器，進圖前先用 --hide 遮掉，再補一張中立的上去；
|
||||||
|
# --anchor 會印出被遮區塊在圖上的位置，hero.html 的圖層座標就是從這裡來的。
|
||||||
|
node scripts/capture-hero.mjs --out docs/hero/agent.png
|
||||||
|
node scripts/capture-hero.mjs --out docs/hero/room.png --pick 測試聊天 \
|
||||||
|
--hide .side-card --anchor .side-card
|
||||||
|
node scripts/capture-hero.mjs --out docs/hero/desktop.png --pick 阿狗 --selector .side-card
|
||||||
|
|
||||||
|
# 2. 套框輸出 README 用的圖，SHOT= 決定用哪張（預設 room）
|
||||||
|
scripts/render-readme-hero.sh
|
||||||
|
SHOT=agent scripts/render-readme-hero.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
`capture-hero.mjs` 直接講 CDP，沒有 npm 相依。預設擷取 1480x650（2 倍圖），正好對應
|
||||||
|
`hero.html` 的 1184x520 視窗，不會裁到圖。noVNC 是即時串流，`--settle`（預設 6 秒）是留給
|
||||||
|
桌面畫出內容的時間。**發布前務必逐張檢查**：機器人瀏覽器裡留著的帳號、網址與貼文會一起進
|
||||||
|
圖，`--hide` 的 selector 沒命中時擷取會直接失敗，不要繞過這個檢查把圖放進 repo。
|
||||||
|
|
||||||
### 專案結構
|
### 專案結構
|
||||||
|
|
||||||
```text
|
```text
|
||||||
|
|
@ -122,7 +146,8 @@ LazyBoy/
|
||||||
├── migrations/ SQLx PostgreSQL migrations
|
├── migrations/ SQLx PostgreSQL migrations
|
||||||
├── tests/ Rust 以外的契約與回歸測試
|
├── tests/ Rust 以外的契約與回歸測試
|
||||||
├── scripts/ 環境初始化與執行工具
|
├── scripts/ 環境初始化與執行工具
|
||||||
├── docs/hero.html README 首頁視覺原稿
|
├── docs/hero.html README 首頁框架（品牌＋真實截圖）
|
||||||
|
├── docs/hero/ README 用的真實截圖原檔
|
||||||
├── docs/workflow.html 可互動產品流程圖
|
├── docs/workflow.html 可互動產品流程圖
|
||||||
├── docker-compose.yml 正式堆疊
|
├── docker-compose.yml 正式堆疊
|
||||||
├── clippy.toml Clippy 閾值（要在 repo 根目錄執行才讀得到）
|
├── clippy.toml Clippy 閾值（要在 repo 根目錄執行才讀得到）
|
||||||
|
|
|
||||||
427
docs/hero.html
427
docs/hero.html
|
|
@ -26,332 +26,49 @@
|
||||||
html, body {
|
html, body {
|
||||||
margin: 0;
|
margin: 0;
|
||||||
width: 1280px;
|
width: 1280px;
|
||||||
height: 640px;
|
height: 760px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
background: #f3f3f5;
|
background: #f3f3f5;
|
||||||
color: #161618;
|
color: #161618;
|
||||||
font-family: Huninn, "jf open 粉圓", "PingFang TC", sans-serif;
|
font-family: Huninn, "jf open 粉圓", "PingFang TC", sans-serif;
|
||||||
}
|
}
|
||||||
.page {
|
.page { width: 1280px; height: 760px; padding: 30px 48px 0; }
|
||||||
width: 1280px;
|
.top { display: flex; align-items: center; justify-content: space-between; height: 36px; }
|
||||||
height: 640px;
|
.brand { display: flex; align-items: center; gap: 10px; font-size: 20px; letter-spacing: .02em; }
|
||||||
padding: 28px 48px 0;
|
.brand svg { width: 26px; height: 26px; }
|
||||||
position: relative;
|
|
||||||
}
|
|
||||||
.top {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
height: 36px;
|
|
||||||
}
|
|
||||||
.brand {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 10px;
|
|
||||||
font-size: 20px;
|
|
||||||
letter-spacing: .02em;
|
|
||||||
}
|
|
||||||
.pill {
|
.pill {
|
||||||
display: inline-flex;
|
display: inline-flex; align-items: center; gap: 8px; height: 28px; padding: 0 12px;
|
||||||
align-items: center;
|
border: 1px solid #e4e4e8; border-radius: 999px; background: #fff; color: #5c5c64; font-size: 12px;
|
||||||
gap: 8px;
|
|
||||||
height: 28px;
|
|
||||||
padding: 0 12px;
|
|
||||||
border: 1px solid #e4e4e8;
|
|
||||||
border-radius: 999px;
|
|
||||||
background: #fff;
|
|
||||||
color: #5c5c64;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
.pill b {
|
|
||||||
color: #0f766e;
|
|
||||||
font-weight: 400;
|
|
||||||
}
|
|
||||||
.copy {
|
|
||||||
text-align: center;
|
|
||||||
margin: 22px auto 18px;
|
|
||||||
max-width: 760px;
|
|
||||||
}
|
|
||||||
h1 {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 42px;
|
|
||||||
font-weight: 400;
|
|
||||||
letter-spacing: .01em;
|
|
||||||
line-height: 1.15;
|
|
||||||
}
|
|
||||||
.lead {
|
|
||||||
margin: 10px 0 0;
|
|
||||||
color: #6b6b73;
|
|
||||||
font-size: 15px;
|
|
||||||
line-height: 1.55;
|
|
||||||
}
|
}
|
||||||
|
.pill b { color: #0f766e; font-weight: 400; }
|
||||||
|
.copy { text-align: center; margin: 20px auto 18px; max-width: 820px; }
|
||||||
|
h1 { margin: 0; font-size: 40px; font-weight: 400; letter-spacing: .01em; line-height: 1.15; }
|
||||||
|
.lead { margin: 10px 0 0; color: #6b6b73; font-size: 15px; line-height: 1.55; }
|
||||||
.window {
|
.window {
|
||||||
width: 1184px;
|
width: 1184px; height: 554px; margin: 0 auto;
|
||||||
height: 470px;
|
border-radius: 16px 16px 0 0; overflow: hidden; background: #0d0d0e;
|
||||||
margin: 0 auto;
|
|
||||||
border-radius: 16px 16px 0 0;
|
|
||||||
overflow: hidden;
|
|
||||||
background: #0d0d0e;
|
|
||||||
box-shadow: 0 24px 70px rgba(16, 16, 20, .28), 0 0 0 1px rgba(0,0,0,.08);
|
box-shadow: 0 24px 70px rgba(16, 16, 20, .28), 0 0 0 1px rgba(0,0,0,.08);
|
||||||
display: grid;
|
display: grid; grid-template-rows: 34px 1fr;
|
||||||
grid-template-rows: 36px 1fr;
|
|
||||||
}
|
}
|
||||||
.chrome {
|
.chrome {
|
||||||
display: flex;
|
display: flex; align-items: center; gap: 7px; padding: 0 14px;
|
||||||
align-items: center;
|
background: #161618; border-bottom: 1px solid #202023;
|
||||||
gap: 7px;
|
|
||||||
padding: 0 14px;
|
|
||||||
background: #161618;
|
|
||||||
border-bottom: 1px solid #202023;
|
|
||||||
}
|
}
|
||||||
.dot { width: 10px; height: 10px; border-radius: 50%; }
|
.dot { width: 10px; height: 10px; border-radius: 50%; }
|
||||||
.dot.r { background: #ff5f57; }
|
.dot.r { background: #ff5f57; }
|
||||||
.dot.y { background: #febc2e; }
|
.dot.y { background: #febc2e; }
|
||||||
.dot.g { background: #28c840; }
|
.dot.g { background: #28c840; }
|
||||||
.app {
|
/* Real pixels: docs/hero/*.png comes from scripts/capture-hero.mjs. */
|
||||||
display: grid;
|
/* 1184x520 matches the capture ratio (1480x650), so nothing is cropped. */
|
||||||
grid-template-columns: 228px 1fr 300px;
|
.shot { position: relative; width: 1184px; height: 520px; overflow: hidden; }
|
||||||
min-height: 0;
|
.shot img { position: absolute; display: block; }
|
||||||
color: #dfdfe2;
|
|
||||||
background: #050506;
|
|
||||||
}
|
|
||||||
.side, .chat, .desk {
|
|
||||||
min-width: 0;
|
|
||||||
min-height: 0;
|
|
||||||
}
|
|
||||||
.side {
|
|
||||||
background: #0b0b0c;
|
|
||||||
border-right: 1px solid #171719;
|
|
||||||
padding: 12px 10px;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 8px;
|
|
||||||
}
|
|
||||||
.ws {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 8px;
|
|
||||||
padding: 4px 6px 8px;
|
|
||||||
font-size: 14px;
|
|
||||||
}
|
|
||||||
.ws i {
|
|
||||||
width: 26px; height: 26px;
|
|
||||||
display: grid; place-items: center;
|
|
||||||
border-radius: 50%;
|
|
||||||
background: #151517;
|
|
||||||
color: #85858a;
|
|
||||||
font-size: 9px;
|
|
||||||
font-style: normal;
|
|
||||||
letter-spacing: .04em;
|
|
||||||
}
|
|
||||||
.search {
|
|
||||||
height: 34px;
|
|
||||||
border: 1px solid #202023;
|
|
||||||
border-radius: 12px;
|
|
||||||
background: #121214;
|
|
||||||
color: #626267;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
padding: 0 10px;
|
|
||||||
font-size: 12px;
|
|
||||||
gap: 8px;
|
|
||||||
}
|
|
||||||
.search svg { width: 13px; height: 13px; stroke: currentColor; fill: none; stroke-width: 1.7; }
|
|
||||||
.label {
|
|
||||||
padding: 8px 8px 2px;
|
|
||||||
color: #626267;
|
|
||||||
font-size: 10px;
|
|
||||||
letter-spacing: .06em;
|
|
||||||
}
|
|
||||||
.bot {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 8px;
|
|
||||||
padding: 8px;
|
|
||||||
border-radius: 12px;
|
|
||||||
}
|
|
||||||
.bot.on { background: #171719; }
|
|
||||||
.bot strong { display: block; font-size: 13px; font-weight: 400; }
|
|
||||||
.bot small { display: block; color: #85858a; font-size: 11px; margin-top: 1px; }
|
|
||||||
.bot .copy { min-width: 0; flex: 1; }
|
|
||||||
.bot .copy strong, .bot .copy small {
|
|
||||||
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
|
|
||||||
}
|
|
||||||
.time { color: #626267; font-size: 10px; align-self: start; }
|
|
||||||
.chat {
|
|
||||||
background: #0d0d0e;
|
|
||||||
display: grid;
|
|
||||||
grid-template-rows: 52px 1fr 118px;
|
|
||||||
}
|
|
||||||
.topbar {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 8px;
|
|
||||||
padding: 0 16px;
|
|
||||||
border-bottom: 1px solid #171719;
|
|
||||||
font-size: 15px;
|
|
||||||
}
|
|
||||||
.grow { flex: 1; }
|
|
||||||
.tools {
|
|
||||||
display: flex; gap: 2px; padding: 3px;
|
|
||||||
border: 1px solid #29292d; border-radius: 11px; background: rgba(18,18,20,.86);
|
|
||||||
}
|
|
||||||
.tools i {
|
|
||||||
width: 26px; height: 26px; border-radius: 8px;
|
|
||||||
display: grid; place-items: center;
|
|
||||||
font-style: normal;
|
|
||||||
}
|
|
||||||
.tools i.on { background: #151517; }
|
|
||||||
.tools svg { width: 13px; height: 13px; stroke: currentColor; fill: none; stroke-width: 1.7; }
|
|
||||||
.tools i.on svg { stroke: #f1f1f2; }
|
|
||||||
.tools i:not(.on) svg { stroke: #85858a; }
|
|
||||||
.msgs {
|
|
||||||
padding: 18px 28px 8px;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 12px;
|
|
||||||
}
|
|
||||||
.bubble {
|
|
||||||
max-width: 78%;
|
|
||||||
padding: 10px 14px;
|
|
||||||
border-radius: 18px;
|
|
||||||
line-height: 1.5;
|
|
||||||
font-size: 13px;
|
|
||||||
white-space: pre-wrap;
|
|
||||||
}
|
|
||||||
.me { align-self: flex-end; background: #f1f1ef; color: #1a1a1a; }
|
|
||||||
.them { align-self: flex-start; background: #19191c; }
|
|
||||||
.dock {
|
|
||||||
padding: 8px 22px 16px;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 8px;
|
|
||||||
}
|
|
||||||
.think {
|
|
||||||
display: flex; align-items: center; gap: 8px;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
.think .lbl {
|
|
||||||
background: linear-gradient(90deg, #7a8088 0%, #7a8088 28%, #fff 50%, #7a8088 72%, #7a8088 100%);
|
|
||||||
background-size: 220% 100%;
|
|
||||||
-webkit-background-clip: text;
|
|
||||||
background-clip: text;
|
|
||||||
color: transparent;
|
|
||||||
}
|
|
||||||
.think small { color: #85858a; }
|
|
||||||
.composer {
|
|
||||||
height: 52px;
|
|
||||||
display: flex; align-items: center; gap: 8px;
|
|
||||||
border: 1px solid #29292d;
|
|
||||||
background: #121214;
|
|
||||||
border-radius: 22px;
|
|
||||||
padding: 6px 8px;
|
|
||||||
}
|
|
||||||
.plus, .send {
|
|
||||||
width: 34px; height: 34px; border-radius: 50%;
|
|
||||||
display: grid; place-items: center;
|
|
||||||
}
|
|
||||||
.plus { color: #85858a; font-size: 20px; }
|
|
||||||
.ph { flex: 1; color: #626267; font-size: 13px; }
|
|
||||||
.send { background: #f1f1ef; color: #1b1b1c; font-size: 14px; }
|
|
||||||
.desk {
|
|
||||||
background: #0a0a0b;
|
|
||||||
border-left: 1px solid #171719;
|
|
||||||
padding: 10px 12px 12px;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 8px;
|
|
||||||
}
|
|
||||||
.desk-h {
|
|
||||||
display: flex; align-items: center; gap: 8px;
|
|
||||||
color: #f1f1f2; font-size: 13px;
|
|
||||||
}
|
|
||||||
.desk-h .dot-run {
|
|
||||||
width: 7px; height: 7px; border-radius: 50%; background: #4ecb71;
|
|
||||||
}
|
|
||||||
.desk-h small { color: #85858a; margin-left: auto; font-size: 11px; }
|
|
||||||
.preview {
|
|
||||||
position: relative;
|
|
||||||
flex: 1;
|
|
||||||
border: 1px solid #29292d;
|
|
||||||
border-radius: 12px;
|
|
||||||
background: #101012;
|
|
||||||
overflow: hidden;
|
|
||||||
min-height: 168px;
|
|
||||||
}
|
|
||||||
.desktop {
|
|
||||||
position: absolute; inset: 0;
|
|
||||||
background:
|
|
||||||
radial-gradient(circle at 20% 0%, #1a2430 0%, transparent 42%),
|
|
||||||
linear-gradient(180deg, #15202b, #0d1218);
|
|
||||||
}
|
|
||||||
.win {
|
|
||||||
position: absolute;
|
|
||||||
left: 14px; top: 12px; right: 14px; bottom: 28px;
|
|
||||||
border-radius: 8px;
|
|
||||||
background: #fff;
|
|
||||||
overflow: hidden;
|
|
||||||
box-shadow: 0 10px 28px rgba(0,0,0,.35);
|
|
||||||
display: grid;
|
|
||||||
grid-template-rows: 22px 28px 1fr;
|
|
||||||
}
|
|
||||||
.win-bar { background: #e8e8ea; display: flex; align-items: center; gap: 5px; padding: 0 8px; }
|
|
||||||
.win-bar i { width: 7px; height: 7px; border-radius: 50%; background: #c4c4c8; font-style: normal; }
|
|
||||||
.addr {
|
|
||||||
background: #f4f4f6;
|
|
||||||
display: flex; align-items: center;
|
|
||||||
padding: 0 10px;
|
|
||||||
color: #444; font-size: 10px;
|
|
||||||
border-bottom: 1px solid #ececf0;
|
|
||||||
}
|
|
||||||
.page-body { padding: 12px 14px; color: #222; }
|
|
||||||
.page-body h3 { margin: 0 0 6px; font-size: 13px; font-weight: 400; }
|
|
||||||
.page-body p { margin: 0; color: #666; font-size: 11px; line-height: 1.45; }
|
|
||||||
.progress {
|
|
||||||
margin-top: 10px; height: 6px; border-radius: 99px; background: #ececf0; overflow: hidden;
|
|
||||||
}
|
|
||||||
.progress b { display: block; width: 62%; height: 100%; background: #3ec5a8; }
|
|
||||||
.next {
|
|
||||||
margin-top: 10px; height: 24px; width: 64px;
|
|
||||||
border-radius: 6px; background: #1a1a1c; color: #fff;
|
|
||||||
display: grid; place-items: center; font-size: 11px;
|
|
||||||
}
|
|
||||||
.panel {
|
|
||||||
position: absolute; left: 0; right: 0; bottom: 0; height: 22px;
|
|
||||||
background: #1b242e; border-top: 1px solid #2a3540;
|
|
||||||
}
|
|
||||||
.cap {
|
|
||||||
display: flex; justify-content: space-between;
|
|
||||||
color: #85858a; font-size: 11px;
|
|
||||||
}
|
|
||||||
.btn {
|
|
||||||
height: 28px; padding: 0 10px; border-radius: 8px;
|
|
||||||
border: 1px solid #29292d; color: #dfdfe2; display: grid; place-items: center;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
.btn.cream { background: #f1f1ef; color: #181819; border-color: #f1f1ef; }
|
|
||||||
.actions { display: flex; justify-content: flex-end; gap: 6px; }
|
|
||||||
|
|
||||||
/* blobatar-ish faces */
|
|
||||||
.blob { width: 32px; height: 32px; flex: 0 0 32px; }
|
|
||||||
.blob.sm { width: 22px; height: 22px; flex-basis: 22px; }
|
|
||||||
.blob.md { width: 28px; height: 28px; flex-basis: 28px; }
|
|
||||||
.think-ring { position: relative; }
|
|
||||||
.think-ring:after {
|
|
||||||
content: "";
|
|
||||||
position: absolute; inset: -5px;
|
|
||||||
border-radius: 50%;
|
|
||||||
background: conic-gradient(from 40deg, transparent 0 8%, #ff4fd8 12%, #7c5cff 18%, transparent 26% 48%, #34d9ff 54%, transparent 62%);
|
|
||||||
-webkit-mask: radial-gradient(farthest-side, transparent calc(100% - 2px), #000 calc(100% - 1.5px));
|
|
||||||
mask: radial-gradient(farthest-side, transparent calc(100% - 2px), #000 calc(100% - 1.5px));
|
|
||||||
}
|
|
||||||
</style>
|
</style>
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
<div class="page">
|
<div class="page">
|
||||||
<div class="top">
|
<div class="top">
|
||||||
<div class="brand">
|
<div class="brand">
|
||||||
<svg class="blob md" viewBox="0 0 64 64" aria-hidden="true">
|
<svg viewBox="0 0 64 64" aria-hidden="true">
|
||||||
<path fill="#3ec5a8" d="M33 5c11.5 1 24 11 23 26-1 14-11 27-25 28S6 47 7 31 21 4 33 5z"/>
|
<path fill="#3ec5a8" d="M33 5c11.5 1 24 11 23 26-1 14-11 27-25 28S6 47 7 31 21 4 33 5z"/>
|
||||||
<ellipse cx="25.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/>
|
<ellipse cx="25.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/>
|
||||||
<ellipse cx="40.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/>
|
<ellipse cx="40.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/>
|
||||||
|
|
@ -364,79 +81,43 @@
|
||||||
</div>
|
</div>
|
||||||
<div class="copy">
|
<div class="copy">
|
||||||
<h1>Give an agent a real computer</h1>
|
<h1>Give an agent a real computer</h1>
|
||||||
<p class="lead">Open bots in the browser. They browse, run commands, and learn the flows you demonstrate.<br/>Chats and desktops stay on your host. You choose the model.</p>
|
<p class="lead">
|
||||||
|
Assign work by text, voice, or a quick demo, then watch its own Linux desktop do it.<br/>
|
||||||
|
Chats, memory, and desktops stay on your host. You choose the model.
|
||||||
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="window">
|
<div class="window">
|
||||||
<div class="chrome"><i class="dot r"></i><i class="dot y"></i><i class="dot g"></i></div>
|
<div class="chrome"><i class="dot r"></i><i class="dot y"></i><i class="dot g"></i></div>
|
||||||
<div class="app">
|
<div class="shot"></div>
|
||||||
<aside class="side">
|
|
||||||
<div class="ws"><i>LB</i> Local workspace</div>
|
|
||||||
<div class="search"><svg viewBox="0 0 24 24"><circle cx="11" cy="11" r="7"/><path d="M20 20l-3.5-3.5"/></svg>Search</div>
|
|
||||||
<div class="label">Pinned</div>
|
|
||||||
<div class="bot on">
|
|
||||||
<span class="think-ring">
|
|
||||||
<svg class="blob" viewBox="0 0 64 64"><path fill="#3ec5a8" d="M33 5c11.5 1 24 11 23 26-1 14-11 27-25 28S6 47 7 31 21 4 33 5z"/><ellipse cx="25.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><ellipse cx="40.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><circle cx="26.8" cy="27.2" r="1.3" fill="#f3f3f6"/><circle cx="41.8" cy="27.2" r="1.3" fill="#f3f3f6"/></svg>
|
|
||||||
</span>
|
|
||||||
<div class="copy"><strong>Engineering</strong><small>Page 12, waiting for Next</small></div>
|
|
||||||
<span class="time">14:08</span>
|
|
||||||
</div>
|
|
||||||
<div class="label">Agent</div>
|
|
||||||
<div class="bot">
|
|
||||||
<svg class="blob" viewBox="0 0 64 64"><path fill="#f5a03c" d="M30 7c13-1 26 10 25 24-1 15-12 26-25 27S5 46 7 31 17 8 30 7z"/><ellipse cx="25" cy="30" rx="4" ry="4.8" fill="#1b1b22"/><ellipse cx="40" cy="30" rx="4" ry="4.8" fill="#1b1b22"/><circle cx="26.3" cy="28.3" r="1.2" fill="#f3f3f6"/><circle cx="41.3" cy="28.3" r="1.2" fill="#f3f3f6"/></svg>
|
|
||||||
<div class="copy"><strong>Morning digest</strong><small>Runs tomorrow at 9:00</small></div>
|
|
||||||
</div>
|
|
||||||
<div class="bot">
|
|
||||||
<svg class="blob" viewBox="0 0 64 64"><path fill="#6a6bf5" d="M34 6c12 2 22 12 21 25s-12 26-25 27S6 46 8 32 22 4 34 6z"/><ellipse cx="26" cy="29" rx="4.1" ry="5" fill="#f3f3f6"/><ellipse cx="41" cy="29" rx="4.1" ry="5" fill="#f3f3f6"/><circle cx="24.8" cy="27.4" r="1.2" fill="#1b1b22"/><circle cx="39.8" cy="27.4" r="1.2" fill="#1b1b22"/></svg>
|
|
||||||
<div class="copy"><strong>Support</strong><small>Remembered the reply tone</small></div>
|
|
||||||
</div>
|
|
||||||
</aside>
|
|
||||||
<main class="chat">
|
|
||||||
<header class="topbar">
|
|
||||||
<svg class="blob sm" viewBox="0 0 64 64"><path fill="#3ec5a8" d="M33 5c11.5 1 24 11 23 26-1 14-11 27-25 28S6 47 7 31 21 4 33 5z"/><ellipse cx="25.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><ellipse cx="40.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><circle cx="26.8" cy="27.2" r="1.3" fill="#f3f3f6"/><circle cx="41.8" cy="27.2" r="1.3" fill="#f3f3f6"/></svg>
|
|
||||||
Engineering
|
|
||||||
<span style="color:#85858a;font-size:12px">STAR training ▾</span>
|
|
||||||
<span class="grow"></span>
|
|
||||||
<div class="tools">
|
|
||||||
<i class="on"><svg viewBox="0 0 24 24"><rect x="3" y="5" width="18" height="12" rx="2"/><path d="M8 21h8M12 17v4"/></svg></i>
|
|
||||||
<i><svg viewBox="0 0 24 24"><path d="M12 3a7 7 0 0 0-4 12.7V18h8v-2.3A7 7 0 0 0 12 3z"/><path d="M9 21h6"/></svg></i>
|
|
||||||
<i><svg viewBox="0 0 24 24"><path d="M12 5v2M12 17v2M5 12h2M17 12h2M7 7l1.5 1.5M15.5 15.5L17 17M7 17l1.5-1.5M15.5 8.5L17 7"/><circle cx="12" cy="12" r="3"/></svg></i>
|
|
||||||
</div>
|
|
||||||
</header>
|
|
||||||
<div class="msgs">
|
|
||||||
<div class="bubble me">Finish the STAR course and submit the quiz.</div>
|
|
||||||
<div class="bubble them">Computer is up. Page 12 / 24 — I’ll hit Next when the video ends. I won’t store passwords.</div>
|
|
||||||
</div>
|
|
||||||
<div class="dock">
|
|
||||||
<div class="think">
|
|
||||||
<span class="think-ring"><svg class="blob sm" viewBox="0 0 64 64"><path fill="#3ec5a8" d="M33 5c11.5 1 24 11 23 26-1 14-11 27-25 28S6 47 7 31 21 4 33 5z"/><ellipse cx="25.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><ellipse cx="40.5" cy="29" rx="4.2" ry="5" fill="#1b1b22"/><circle cx="26.8" cy="27.2" r="1.3" fill="#f3f3f6"/><circle cx="41.8" cy="27.2" r="1.3" fill="#f3f3f6"/></svg></span>
|
|
||||||
<span class="lbl">Working…</span>
|
|
||||||
<small>click Next</small>
|
|
||||||
</div>
|
|
||||||
<div class="composer"><div class="plus">＋</div><div class="ph">Message Engineering</div><div class="send">↑</div></div>
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
<aside class="desk">
|
|
||||||
<div class="desk-h">Engineering’s computer <i class="dot-run"></i> <small>Running</small></div>
|
|
||||||
<div class="preview">
|
|
||||||
<div class="desktop">
|
|
||||||
<div class="win">
|
|
||||||
<div class="win-bar"><i></i><i></i><i></i></div>
|
|
||||||
<div class="addr">star.example / course / 12</div>
|
|
||||||
<div class="page-body">
|
|
||||||
<h3>Workplace Safety · page 12</h3>
|
|
||||||
<p>Read this, then press Next. It unlocks when the countdown ends.</p>
|
|
||||||
<div class="progress"><b></b></div>
|
|
||||||
<div class="next">Next</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="panel"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="cap"><span>This screen</span><span>Enlarge</span></div>
|
|
||||||
<div class="actions"><div class="btn cream">Take over</div><div class="btn">Stop task</div></div>
|
|
||||||
</aside>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
<script>
|
||||||
|
// Every layer is a real capture from scripts/capture-hero.mjs, so the banner
|
||||||
|
// never shows an invented UI. The group view is two layers on purpose:
|
||||||
|
// room.png is captured with `--hide .side-card`, which blanks the desktop
|
||||||
|
// panel (an agent's browser holds whatever its owner left open), and
|
||||||
|
// desktop.png drops a neutral one back in at the anchor the same capture
|
||||||
|
// prints. Captures are 2x shots of a 1480px viewport, so a capture-space
|
||||||
|
// number becomes hero-space by multiplying by 1184 / 1480 = 0.8.
|
||||||
|
const layers = {
|
||||||
|
room: [
|
||||||
|
{ src: './hero/room.png', x: 0, y: 0, w: 1184, h: 520 },
|
||||||
|
{ src: './hero/desktop.png', x: 832, y: 0, w: 352, h: 520 },
|
||||||
|
],
|
||||||
|
agent: [{ src: './hero/agent.png', x: 0, y: 0, w: 1184, h: 520 }],
|
||||||
|
};
|
||||||
|
const shot = new URLSearchParams(location.search).get('shot') || 'room';
|
||||||
|
const host = document.querySelector('.shot');
|
||||||
|
(layers[shot] || layers.room).forEach((layer) => {
|
||||||
|
const img = document.createElement('img');
|
||||||
|
img.src = layer.src;
|
||||||
|
img.alt = 'LazyBoy group chat routing one message to an agent on its live desktop';
|
||||||
|
img.style.left = `${layer.x}px`;
|
||||||
|
img.style.top = `${layer.y}px`;
|
||||||
|
img.style.width = `${layer.w}px`;
|
||||||
|
img.style.height = `${layer.h}px`;
|
||||||
|
host.appendChild(img);
|
||||||
|
});
|
||||||
|
</script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 235 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 181 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 242 KiB After Width: | Height: | Size: 310 KiB |
|
|
@ -0,0 +1,314 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
// Capture README hero screenshots from a running LazyBoy instance.
|
||||||
|
//
|
||||||
|
// The banner must show the real app, so this drives a real browser against a
|
||||||
|
// real stack instead of painting a mock-up. No npm dependencies: it speaks
|
||||||
|
// Chrome DevTools Protocol over the browser WebSocket (Node 22+).
|
||||||
|
//
|
||||||
|
// node scripts/capture-hero.mjs --out docs/hero/agent.png
|
||||||
|
// node scripts/capture-hero.mjs --out docs/hero/room.png --pick 測試聊天
|
||||||
|
// node scripts/capture-hero.mjs --out docs/hero/desktop.png --selector .side-card
|
||||||
|
//
|
||||||
|
// A capture that goes into the repository must not photograph whatever the
|
||||||
|
// agent's browser happens to have open. `--hide` blanks a region (its layout
|
||||||
|
// box stays, so nothing reflows) and `--anchor` prints where that region sits
|
||||||
|
// inside the shot, which is what docs/hero.html uses to overlay a clean panel.
|
||||||
|
//
|
||||||
|
// node scripts/capture-hero.mjs --pick 測試聊天 --hide .side-card --anchor .side-card \
|
||||||
|
// --out docs/hero/room.png
|
||||||
|
// node scripts/capture-hero.mjs --pick 阿狗 --selector .side-card --out docs/hero/desktop.png
|
||||||
|
//
|
||||||
|
// Environment:
|
||||||
|
// LB_URL stack to photograph (default http://127.0.0.1:3101)
|
||||||
|
// LB_TOKEN login token (default: LAZYBOY_APP_TOKEN from .env)
|
||||||
|
// CHROME browser binary (default: discovered, see findChrome)
|
||||||
|
import { spawn } from 'node:child_process';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||||
|
|
||||||
|
const option = (name, fallback) => {
|
||||||
|
const index = process.argv.indexOf(`--${name}`);
|
||||||
|
return index > -1 && process.argv[index + 1] ? process.argv[index + 1] : fallback;
|
||||||
|
};
|
||||||
|
|
||||||
|
const config = {
|
||||||
|
base: (process.env.LB_URL || 'http://127.0.0.1:3101').replace(/\/$/, ''),
|
||||||
|
out: option('out', 'docs/hero/app.png'),
|
||||||
|
pick: option('pick', ''),
|
||||||
|
selector: option('selector', '.app-shell'),
|
||||||
|
hide: option('hide', ''),
|
||||||
|
anchor: option('anchor', ''),
|
||||||
|
width: Number(option('width', 1480)),
|
||||||
|
// 1480x650 keeps the 1184x520 image box in docs/hero.html free of cropping.
|
||||||
|
height: Number(option('height', 650)),
|
||||||
|
scale: Number(option('scale', 2)),
|
||||||
|
settle: Number(option('settle', 6000)),
|
||||||
|
};
|
||||||
|
|
||||||
|
function findChrome() {
|
||||||
|
const candidates = [
|
||||||
|
process.env.CHROME,
|
||||||
|
...['chromium', 'chromium-browser', 'google-chrome', 'google-chrome-stable', 'brave', 'microsoft-edge']
|
||||||
|
.map((name) => process.env.PATH.split(path.delimiter).map((dir) => path.join(dir, name)).find((bin) => fs.existsSync(bin))),
|
||||||
|
'/Applications/Brave Browser.app/Contents/MacOS/Brave Browser',
|
||||||
|
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
||||||
|
].filter(Boolean);
|
||||||
|
for (const candidate of candidates) if (fs.existsSync(candidate)) return candidate;
|
||||||
|
const playwright = path.join(os.homedir(), '.cache/ms-playwright');
|
||||||
|
if (fs.existsSync(playwright)) {
|
||||||
|
for (const build of fs.readdirSync(playwright).sort().reverse()) {
|
||||||
|
for (const dir of ['chrome-linux64', 'chrome-mac']) {
|
||||||
|
const bin = path.join(playwright, build, dir, 'chrome');
|
||||||
|
if (fs.existsSync(bin)) return bin;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new Error('no Chromium found; set CHROME=/path/to/chrome');
|
||||||
|
}
|
||||||
|
|
||||||
|
function readToken() {
|
||||||
|
if (process.env.LB_TOKEN) return process.env.LB_TOKEN;
|
||||||
|
const file = path.join(root, '.env');
|
||||||
|
const match = fs.existsSync(file) ? fs.readFileSync(file, 'utf8').match(/^LAZYBOY_APP_TOKEN=(.+)$/m) : null;
|
||||||
|
if (!match) throw new Error('no login token; set LB_TOKEN or create .env with make env');
|
||||||
|
return match[1].trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForVersion(port) {
|
||||||
|
for (let attempt = 0; attempt < 60; attempt += 1) {
|
||||||
|
try {
|
||||||
|
const response = await fetch(`http://127.0.0.1:${port}/json/version`);
|
||||||
|
if (response.ok) return await response.json();
|
||||||
|
} catch {
|
||||||
|
// Chrome is still coming up.
|
||||||
|
}
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||||
|
}
|
||||||
|
throw new Error('Chrome did not expose a DevTools endpoint');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Minimal flat CDP client: one WebSocket, promise-per-id, plus event waiting.
|
||||||
|
class Connection {
|
||||||
|
constructor(socket) {
|
||||||
|
this.socket = socket;
|
||||||
|
this.nextId = 1;
|
||||||
|
this.pending = new Map();
|
||||||
|
this.listeners = new Map();
|
||||||
|
socket.addEventListener('message', (event) => {
|
||||||
|
const message = JSON.parse(event.data);
|
||||||
|
if (message.id) {
|
||||||
|
const waiter = this.pending.get(message.id);
|
||||||
|
if (!waiter) return;
|
||||||
|
this.pending.delete(message.id);
|
||||||
|
if (message.error) waiter.reject(new Error(`${message.method}: ${message.error.message}`));
|
||||||
|
else waiter.resolve(message.result);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const handlers = this.listeners.get(message.method) || [];
|
||||||
|
this.listeners.set(message.method, handlers.filter((handler) => !handler.once));
|
||||||
|
handlers.forEach((handler) => handler.resolve(message.params));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
send(method, params = {}, sessionId) {
|
||||||
|
const id = this.nextId;
|
||||||
|
this.nextId += 1;
|
||||||
|
this.socket.send(JSON.stringify({ id, method, params, ...(sessionId ? { sessionId } : {}) }));
|
||||||
|
return new Promise((resolve, reject) => this.pending.set(id, { resolve, reject }));
|
||||||
|
}
|
||||||
|
|
||||||
|
waitFor(method, sessionId) {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const handlers = this.listeners.get(method) || [];
|
||||||
|
handlers.push({ resolve, once: true });
|
||||||
|
this.listeners.set(method, handlers);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Retry an evaluate call until it returns something other than null. The app
|
||||||
|
// loads its lists over the API, so a single shot races the first render.
|
||||||
|
const retry = async (attempt) => {
|
||||||
|
for (let pass = 0; pass < 40; pass += 1) {
|
||||||
|
const value = await attempt();
|
||||||
|
if (value) return value;
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
};
|
||||||
|
|
||||||
|
const launch = async () => {
|
||||||
|
const profile = fs.mkdtempSync(path.join(os.tmpdir(), 'lazyboy-hero-'));
|
||||||
|
const port = 9400 + Math.floor(Math.random() * 200);
|
||||||
|
const args = [
|
||||||
|
'--headless=new',
|
||||||
|
`--remote-debugging-port=${port}`,
|
||||||
|
`--user-data-dir=${profile}`,
|
||||||
|
'--no-first-run',
|
||||||
|
'--no-default-browser-check',
|
||||||
|
'--disable-gpu',
|
||||||
|
'--hide-scrollbars',
|
||||||
|
'--force-color-profile=srgb',
|
||||||
|
`--window-size=${config.width},${config.height}`,
|
||||||
|
...(process.getuid?.() === 0 ? ['--no-sandbox'] : []),
|
||||||
|
'about:blank',
|
||||||
|
];
|
||||||
|
const chrome = spawn(findChrome(), args, { stdio: 'ignore' });
|
||||||
|
chrome.on('exit', (code) => {
|
||||||
|
if (code && code !== 0) console.error(`chrome exited early with code ${code}`);
|
||||||
|
});
|
||||||
|
const cleanup = () => {
|
||||||
|
chrome.kill('SIGKILL');
|
||||||
|
fs.rmSync(profile, { recursive: true, force: true });
|
||||||
|
};
|
||||||
|
const version = await waitForVersion(port);
|
||||||
|
return { cleanup, version };
|
||||||
|
};
|
||||||
|
|
||||||
|
const { cleanup, version } = await launch();
|
||||||
|
process.on('exit', cleanup);
|
||||||
|
try {
|
||||||
|
const socket = new WebSocket(version.webSocketDebuggerUrl);
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
socket.addEventListener('open', resolve, { once: true });
|
||||||
|
socket.addEventListener('error', () => reject(new Error('DevTools WebSocket failed')), { once: true });
|
||||||
|
});
|
||||||
|
const connection = new Connection(socket);
|
||||||
|
const { targetId } = await connection.send('Target.createTarget', { url: 'about:blank' });
|
||||||
|
const { sessionId } = await connection.send('Target.attachToTarget', { targetId, flatten: true });
|
||||||
|
await connection.send('Page.enable', {}, sessionId);
|
||||||
|
await connection.send(
|
||||||
|
'Emulation.setDeviceMetricsOverride',
|
||||||
|
{ width: config.width, height: config.height, deviceScaleFactor: config.scale, mobile: false },
|
||||||
|
sessionId,
|
||||||
|
);
|
||||||
|
|
||||||
|
const goto = async (url) => {
|
||||||
|
const loaded = connection.waitFor('Page.loadEventFired', sessionId);
|
||||||
|
await connection.send('Page.navigate', { url }, sessionId);
|
||||||
|
await loaded;
|
||||||
|
};
|
||||||
|
|
||||||
|
await goto(`${config.base}/`);
|
||||||
|
const status = await connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `fetch('/api/session',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({token:${JSON.stringify(readToken())}})}).then(r=>r.status)`,
|
||||||
|
awaitPromise: true,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
).then((result) => result.result?.value);
|
||||||
|
if (status !== 200) throw new Error(`login failed with HTTP ${status}`);
|
||||||
|
await goto(`${config.base}/`);
|
||||||
|
|
||||||
|
if (config.pick) {
|
||||||
|
// The sidebar fills in after the first API responses, so retry until the
|
||||||
|
// entry is really there before clicking.
|
||||||
|
const clicked = await retry(async () => connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `(() => {
|
||||||
|
const label = ${JSON.stringify(config.pick)};
|
||||||
|
const hits = [...document.querySelectorAll('button,[role="button"],li,div')].filter((el) => {
|
||||||
|
const text = (el.innerText || '').trim();
|
||||||
|
return text.startsWith(label) && text.length < 90 && el.offsetParent !== null
|
||||||
|
&& el.getBoundingClientRect().width < 420;
|
||||||
|
});
|
||||||
|
if (!hits.length) return null;
|
||||||
|
hits.sort((a, b) => a.innerText.length - b.innerText.length)[0].click();
|
||||||
|
return 'clicked';
|
||||||
|
})()`,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
).then((result) => result.result?.value));
|
||||||
|
if (clicked !== 'clicked') throw new Error(`sidebar entry "${config.pick}" not found`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The noVNC panel is a live stream; give it time to paint a real desktop.
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, config.settle));
|
||||||
|
|
||||||
|
if (config.hide) {
|
||||||
|
const hidden = await connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `document.querySelectorAll(${JSON.stringify(config.hide)}).length`,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
).then((result) => result.result?.value);
|
||||||
|
// A typo in --hide would silently ship a photograph of the live desktop.
|
||||||
|
if (!hidden) throw new Error(`--hide ${config.hide} matched nothing; refusing to capture`);
|
||||||
|
// A stylesheet survives React re-renders; an inline style does not.
|
||||||
|
await connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `(() => {
|
||||||
|
const style = document.createElement('style');
|
||||||
|
const rules = ${JSON.stringify(config.hide)}.split(',')
|
||||||
|
.map((s) => s.trim()).filter(Boolean)
|
||||||
|
.map((s) => s + ' { visibility: hidden !important }').join('\\n');
|
||||||
|
style.textContent = rules;
|
||||||
|
document.head.appendChild(style);
|
||||||
|
return document.querySelectorAll(${JSON.stringify(config.hide)}).length;
|
||||||
|
})()`,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const rect = await connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `(() => {
|
||||||
|
const el = document.querySelector(${JSON.stringify(config.selector)});
|
||||||
|
if (!el) return null;
|
||||||
|
const box = el.getBoundingClientRect();
|
||||||
|
return { x: box.x, y: box.y, width: box.width, height: box.height };
|
||||||
|
})()`,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
).then((result) => result.result?.value);
|
||||||
|
if (!rect) throw new Error(`selector ${config.selector} not found`);
|
||||||
|
|
||||||
|
let placed = '';
|
||||||
|
if (config.anchor) {
|
||||||
|
const box = await connection.send(
|
||||||
|
'Runtime.evaluate',
|
||||||
|
{
|
||||||
|
expression: `(() => {
|
||||||
|
const host = document.querySelector(${JSON.stringify(config.selector)});
|
||||||
|
const el = document.querySelector(${JSON.stringify(config.anchor)});
|
||||||
|
if (!host || !el) return null;
|
||||||
|
const a = el.getBoundingClientRect();
|
||||||
|
const b = host.getBoundingClientRect();
|
||||||
|
return { x: a.x - b.x, y: a.y - b.y, width: a.width, height: a.height };
|
||||||
|
})()`,
|
||||||
|
returnByValue: true,
|
||||||
|
},
|
||||||
|
sessionId,
|
||||||
|
).then((result) => result.result?.value);
|
||||||
|
if (!box) throw new Error(`anchor ${config.anchor} not found`);
|
||||||
|
placed = ` anchor ${box.x.toFixed(1)},${box.y.toFixed(1)} ${box.width.toFixed(1)}x${box.height.toFixed(1)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const shot = await connection.send(
|
||||||
|
'Page.captureScreenshot',
|
||||||
|
{ format: 'png', clip: { ...rect, scale: 1 }, captureBeyondViewport: true },
|
||||||
|
sessionId,
|
||||||
|
);
|
||||||
|
const file = path.resolve(root, config.out);
|
||||||
|
fs.mkdirSync(path.dirname(file), { recursive: true });
|
||||||
|
fs.writeFileSync(file, Buffer.from(shot.data, 'base64'));
|
||||||
|
const bytes = fs.statSync(file).size;
|
||||||
|
console.log(`wrote ${path.relative(root, file)} ${Math.round(rect.width * config.scale)}x${Math.round(rect.height * config.scale)} (${bytes} bytes)${placed}`);
|
||||||
|
} finally {
|
||||||
|
cleanup();
|
||||||
|
}
|
||||||
|
|
@ -1,23 +0,0 @@
|
||||||
#!/usr/bin/env bash
|
|
||||||
# Render docs/diagrams.html slides to docs/diagrams/*.png
|
|
||||||
set -euo pipefail
|
|
||||||
root="$(cd "$(dirname "$0")/.." && pwd)"
|
|
||||||
html="$root/docs/diagrams.html"
|
|
||||||
out="$root/docs/diagrams"
|
|
||||||
brave="/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
|
|
||||||
mkdir -p "$out"
|
|
||||||
if [[ ! -x "$brave" ]]; then
|
|
||||||
echo "Need Brave at $brave" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
for p in map chat sleep look click teach schedule keys folders; do
|
|
||||||
"$brave" \
|
|
||||||
--headless=new \
|
|
||||||
--disable-gpu \
|
|
||||||
--hide-scrollbars \
|
|
||||||
--force-device-scale-factor=2 \
|
|
||||||
--window-size=1280,720 \
|
|
||||||
--screenshot="$out/$p.png" \
|
|
||||||
"file://$html?p=$p"
|
|
||||||
echo "wrote $out/$p.png"
|
|
||||||
done
|
|
||||||
|
|
@ -1,25 +1,45 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Render docs/hero.html to docs/readme-hero.png
|
# Render docs/hero.html to docs/readme-hero.png
|
||||||
# Same idea as Rakazo's README banner: a designed HTML frame (logo + headline
|
#
|
||||||
# + product window), then a browser screenshot. Not an image-model generation,
|
# The banner only frames a real capture (see scripts/capture-hero.mjs), so this
|
||||||
# so every character on the screen stays exact.
|
# step needs any headless Chromium: CHROME=/path/to/chrome, a browser on PATH,
|
||||||
|
# the macOS Brave/Chrome install, or a Playwright download.
|
||||||
|
#
|
||||||
|
# scripts/render-readme-hero.sh # the group shot (docs/hero/room.png)
|
||||||
|
# SHOT=agent scripts/render-readme-hero.sh # the single-agent shot
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
root="$(cd "$(dirname "$0")/.." && pwd)"
|
root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
html="$root/docs/hero.html"
|
html="$root/docs/hero.html"
|
||||||
out="$root/docs/readme-hero.png"
|
out="${OUT:-$root/docs/readme-hero.png}"
|
||||||
brave="/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
|
shot="${SHOT:-room}"
|
||||||
if [[ ! -x "$brave" ]]; then
|
size="${SIZE:-1280,760}"
|
||||||
echo "Need Brave at $brave (or edit this script to your Chromium)." >&2
|
|
||||||
|
browser=""
|
||||||
|
for candidate in "${CHROME:-}" \
|
||||||
|
"$(command -v chromium 2>/dev/null || true)" \
|
||||||
|
"$(command -v chromium-browser 2>/dev/null || true)" \
|
||||||
|
"$(command -v google-chrome 2>/dev/null || true)" \
|
||||||
|
"$(command -v brave-browser 2>/dev/null || true)" \
|
||||||
|
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser" \
|
||||||
|
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"; do
|
||||||
|
if [ -n "$candidate" ] && [ -x "$candidate" ]; then browser="$candidate"; break; fi
|
||||||
|
done
|
||||||
|
if [ -z "$browser" ]; then
|
||||||
|
for cached in "$HOME"/.cache/ms-playwright/chromium-*/chrome-*/chrome; do
|
||||||
|
if [ -x "$cached" ]; then browser="$cached"; break; fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
if [ -z "$browser" ]; then
|
||||||
|
echo "No Chromium found. Set CHROME=/path/to/chrome and try again." >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
"$brave" \
|
|
||||||
--headless=new \
|
args=(--headless=new --disable-gpu --hide-scrollbars --force-device-scale-factor=2
|
||||||
--disable-gpu \
|
--window-size="$size" --screenshot="$out")
|
||||||
--hide-scrollbars \
|
# Root needs the sandbox off; nowhere else does.
|
||||||
--force-device-scale-factor=2 \
|
[ "$(id -u)" = "0" ] && args+=(--no-sandbox)
|
||||||
--window-size=1280,640 \
|
"$browser" "${args[@]}" "file://$html?shot=$shot"
|
||||||
--screenshot="$out" \
|
|
||||||
"file://$html"
|
|
||||||
python3 - "$out" <<'PY'
|
python3 - "$out" <<'PY'
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import struct, sys
|
import struct, sys
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue