Commit Graph

4 Commits

Author SHA1 Message Date
王性驊 3718d4ab21 Benchmark notes: three unseen macOS apps on the Mac mini
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
2026-11-09 14:22:55 +08:00
王性驊 d4745a7a7a The same click twice in a step is done once, a front app that vanishes for a moment is looked at again, and the notes have the Mac and held-out results
- Two identical clicks in one step opened the Mac's Wi‑Fi menu and closed it again: the second is
  skipped with a word on why (count: 2 for a double click).
- macOS Calculator had no focused window for a moment after 「乘」 and the list came back with
  only the menu bar: an empty or failed read is tried once more 0.3 s later.
- Benchmark notes: three more unseen apps on box v9 (16/16, GTK 2 apps 4-10x faster with gail)
  and the Mac mini run with Screen Recording (zoom, Wi‑Fi menu, Calculator checked by eye).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
2026-11-09 13:31:25 +08:00
王性驊 3513982f3a Benchmark notes: the v10 rerun of the field and dialog tasks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
2026-11-09 11:53:46 +08:00
王性驊 bcae66b21a A computer-use benchmark: eight fixed tasks on a throwaway box, timed step by step
bench/computer-use/run.py runs each task through `anybot --ask` with a fresh chat, checks the
result inside the box, and records success, steps, time split between model and tools, route
switches and a per-step transcript. It refuses to run against the real box (needs its own
ANYBOT_BOX_NAME / ANYBOT_BOX_VOLUME and a scratch data folder). --keep-skills carries learned
skills into the next run to measure replay. README has the tasks, how to run, and the
2026-11-09 numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uirha5F5gJt93t3yEx3XcS
2026-11-09 09:00:48 +08:00