feat: enforce path ladder before computerUse
Runtime rejects spawn_subagent kind=computerUse until web/DOM/MCP was tried this turn (or force=true). Prompt + CORE-GAPS document the four usability cores; handoff re-observe guidance tightened.
This commit is contained in:
王性驊
parent
2f6e9b3f7b
commit
4646d2d9a8
|
|
@ -35,10 +35,10 @@ pub const AGENT_SYSTEM: &str = "\
|
|||
You are GrokBoy, a warm, concise local Grok Bot-style agent.
|
||||
|
||||
## How a turn works
|
||||
1. Reply first. On any turn a person opened, your first action is send_message (plain text): answer directly if it is quick, or acknowledge and name the first step if it is real work. Never open such a turn with a tool call. A hidden revival (a background command finishing) may start work silently if there is nothing to report.
|
||||
1. Reply first. On any turn a person opened, your first action is send_message (plain text): answer directly if it is quick, or acknowledge and name the first step if it is real work. Never open such a turn with a non-message tool call. Keep the acknowledgement to one sentence; include independent first work tools after it in the SAME response when their arguments are already known. Do not spend a separate model round only announcing work. A hidden revival (a background command finishing) may start work silently if there is nothing to report.
|
||||
2. Pick the surface. Default to shell and read on your Docker computer; web_search/web_fetch for public information; the box browser for login-gated sites; shell/read to inspect box files; send_message widget or request_user_input only when the user must decide.
|
||||
3. Work out loud. Keep the user posted with send_message on meaningful beats (a result, decision, blocker, change of plan). Do not vanish into a long silent run of tool calls.
|
||||
4. Close the loop. Deliver the result in send_message. Then end the turn by responding with NO tool calls.
|
||||
4. Close the loop. Once the requested work and necessary verification are complete, put the actual result directly in send_message instead of first drafting it in plain assistant text. Do not send a final result while required tools or child results are still pending. Deliver the result in send_message. Then end the turn by responding with NO tool calls.
|
||||
|
||||
## Plans
|
||||
When the work has stages, keep a short update_plan. After you mark a step completed, send_message in that same moment with the finding (what you learned) and the next step. The user should see a report after every completed line, not one dump at the end. Do not mark completed until you actually have the result.
|
||||
|
|
@ -73,8 +73,16 @@ external_read_file pages by line. external_grep uses regex; external_glob finds
|
|||
## MCP
|
||||
Installed connectors are structured access to services (mail, issues, docs). Read a tool's schema with get_mcp_tools first, then invoke it with call_mcp_tool — every call is live. Prefer a service's MCP over its browser UI, including a connector you would have to add first (add_mcp_server). If a call fails or looks like a no-op, refetch the descriptor with get_mcp_tools and compare; if the schema changed, rebuild arguments. Ask the user for secrets rather than guessing. Config lives in ~/.grokboy/mcp.json.
|
||||
|
||||
## Path ladder
|
||||
Pick the cheapest surface that can finish the job. Do not jump to pixels first.
|
||||
1. Prefer MCP / API / local files (call_mcp_tool, shell/read, external_*).
|
||||
2. Public pages: web_search then web_fetch (anonymous HTTP; no cookies).
|
||||
3. Login-gated or JS-heavy pages: browser_* on the shared Docker Chromium (DOM snapshot + selector/role). Re-observe after every navigation.
|
||||
4. screenshot is read-only check-in on MY computer — never a substitute for browser_*.
|
||||
5. spawn_subagent kind=computerUse is last: native GUI, file dialogs, drag, or a site that already defeated DOM. Runtime rejects computerUse until a lower web/DOM/MCP rung was tried this turn, unless you pass force=true for a clear non-DOM task.
|
||||
|
||||
## Collaboration and safety
|
||||
A blocked step is not a failed task. For login/OTP/captcha use browser_handoff on the SAME browser. Never collect passwords in chat. Never publish or take irreversible public actions without explicit approval this turn or request_user_confirm. Traditional Chinese is welcome when the user writes in Chinese. Do not invent tool results.";
|
||||
A blocked step is not a failed task. For login/OTP/captcha use browser_handoff or request_box_help on the SAME shared browser/session — never a new profile. After the human returns, re-observe with browser_snapshot (or one screenshot if you were on the desktop) before the next action; do not assume the prior DOM still holds. Never collect passwords in chat. Never publish or take irreversible public actions without explicit approval this turn or request_user_confirm. Traditional Chinese is welcome when the user writes in Chinese. Do not invent tool results.";
|
||||
|
||||
/// Final verdict from the agent loop (fail-closed when stuck).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
|
|
@ -379,6 +387,7 @@ where
|
|||
Fut: Future<Output = Result<ChatMessage>>,
|
||||
P: FnMut(&str),
|
||||
{
|
||||
let _turn_timing = crate::timing::Timing::new("agent_turn");
|
||||
let tools = if let Some(team) = &tool_ctx.team {
|
||||
crate::team::worker::definitions(team.task.is_none())
|
||||
} else {
|
||||
|
|
@ -439,23 +448,16 @@ where
|
|||
messages.push(ChatMessage::user(SILENCE_REMINDER));
|
||||
silence_reminded = true;
|
||||
}
|
||||
let prepare_timing = crate::timing::Timing::new("request_preparation");
|
||||
runtime.checkpoint(messages, None)?;
|
||||
if let Some(team) = &tool_ctx.team {
|
||||
team.ack_messages(messages)?;
|
||||
}
|
||||
let mut request_messages = messages.clone();
|
||||
let mut request_messages = request_history(
|
||||
messages,
|
||||
tool_ctx.team.as_ref().is_some_and(|team| team.task.is_none()),
|
||||
);
|
||||
if let Some(team) = &tool_ctx.team {
|
||||
if team.task.is_none() {
|
||||
let starts = request_messages
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, m)| m.role == Role::User)
|
||||
.map(|(i, _)| i)
|
||||
.collect::<Vec<_>>();
|
||||
if starts.len() > 20 {
|
||||
request_messages.drain(1..starts[starts.len() - 20]);
|
||||
}
|
||||
}
|
||||
request_messages.insert(
|
||||
1.min(request_messages.len()),
|
||||
ChatMessage::system(team.context()?),
|
||||
|
|
@ -487,6 +489,7 @@ where
|
|||
runtime,
|
||||
&mut on_progress,
|
||||
);
|
||||
drop(prepare_timing);
|
||||
// Grok Bot style first-visible latency: the reply streams in and a text
|
||||
// `send_message` runs the moment its arguments close, instead of after the
|
||||
// model has also finished writing every other tool call in the same reply.
|
||||
|
|
@ -500,6 +503,7 @@ where
|
|||
complete(request_messages, Some(tools.clone())),
|
||||
);
|
||||
let mut eager: HashMap<String, String> = HashMap::new();
|
||||
let model_timing = crate::timing::Timing::new("model_round_including_queue");
|
||||
let outcome = {
|
||||
let eager = &mut eager;
|
||||
let on_progress = &mut on_progress;
|
||||
|
|
@ -545,6 +549,7 @@ where
|
|||
})
|
||||
.await
|
||||
};
|
||||
drop(model_timing);
|
||||
let reply = match outcome {
|
||||
Ok(reply) => reply,
|
||||
Err(error) if runtime.cancelled() => {
|
||||
|
|
@ -940,12 +945,39 @@ fn skipped_tool_result(message: &str) -> String {
|
|||
serde_json::json!({"error":message,"executed":false}).to_string()
|
||||
}
|
||||
|
||||
// Preserve the existing foreground 20-user-message window without cloning
|
||||
// old tool outputs that would immediately be discarded. Durable history stays intact.
|
||||
fn request_history(messages: &[ChatMessage], foreground: bool) -> Vec<ChatMessage> {
|
||||
if foreground {
|
||||
if let Some((start, _)) = messages
|
||||
.iter()
|
||||
.enumerate()
|
||||
.rev()
|
||||
.filter(|(_, m)| m.role == Role::User)
|
||||
.nth(19)
|
||||
{
|
||||
if messages[..start].iter().any(|m| m.role == Role::User) {
|
||||
return messages[..1]
|
||||
.iter()
|
||||
.chain(messages[start..].iter())
|
||||
.cloned()
|
||||
.collect();
|
||||
}
|
||||
}
|
||||
}
|
||||
messages.to_vec()
|
||||
}
|
||||
|
||||
/// A plain-text `send_message` may run while the reply is still streaming: it only
|
||||
/// delivers text, so executing it before its sibling tool calls exist changes nothing.
|
||||
fn eager_deliverable(call: &ToolCall) -> bool {
|
||||
call.function.name == "send_message"
|
||||
&& serde_json::from_str::<Value>(&call.function.arguments)
|
||||
.is_ok_and(|args| args["type"] != "widget" && args.get("widget").is_none())
|
||||
.is_ok_and(|args| {
|
||||
args["type"] != "widget"
|
||||
&& args.get("widget").is_none()
|
||||
&& args.get("task_id").is_none()
|
||||
})
|
||||
}
|
||||
|
||||
async fn execute_tool_batch(
|
||||
|
|
@ -1022,6 +1054,7 @@ async fn run_one_tool(
|
|||
runtime: &crate::Runtime,
|
||||
call: &ToolCall,
|
||||
) -> String {
|
||||
let _tool_timing = crate::timing::Timing::new("tool_execution");
|
||||
runtime.emit(AgentEvent::ToolStarted {
|
||||
id: call.id.clone(),
|
||||
name: call.function.name.clone(),
|
||||
|
|
@ -1085,6 +1118,31 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn foreground_history_matches_old_window_without_mutating_source() {
|
||||
for users in [0, 1, 20, 21, 50] {
|
||||
let mut messages = vec![ChatMessage::system("system")];
|
||||
for i in 0..users {
|
||||
messages.push(ChatMessage::user(format!("user {i}")));
|
||||
messages.push(ChatMessage::assistant("answer"));
|
||||
}
|
||||
let original = serde_json::to_value(&messages).unwrap();
|
||||
let mut expected = messages.clone();
|
||||
let starts: Vec<_> = expected.iter().enumerate()
|
||||
.filter(|(_, m)| m.role == Role::User).map(|(i, _)| i).collect();
|
||||
if starts.len() > 20 { expected.drain(1..starts[starts.len()-20]); }
|
||||
assert_eq!(serde_json::to_value(request_history(&messages, true)).unwrap(), serde_json::to_value(expected).unwrap());
|
||||
assert_eq!(serde_json::to_value(request_history(&messages, false)).unwrap(), original);
|
||||
assert_eq!(serde_json::to_value(&messages).unwrap(), original);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn task_messages_are_not_eager_user_delivery() {
|
||||
assert!(!eager_deliverable(&tc("peer", "send_message", r#"{"task_id":"t1","message":"steer"}"#)));
|
||||
assert!(eager_deliverable(&tc("user", "send_message", r#"{"type":"text","text":"hello"}"#)));
|
||||
}
|
||||
|
||||
// Check the protocol invariant on persisted history, including stopped runs.
|
||||
fn assert_tool_results_paired(messages: &[ChatMessage]) {
|
||||
for (i, msg) in messages.iter().enumerate() {
|
||||
|
|
@ -1477,6 +1535,9 @@ mod tests {
|
|||
assert!(AGENT_SYSTEM.contains("NO tool calls") || AGENT_SYSTEM.contains("no tool calls"));
|
||||
assert!(AGENT_SYSTEM.contains("block_until_ms"));
|
||||
assert!(AGENT_SYSTEM.contains("ack ≠ delivery") || AGENT_SYSTEM.contains("ack != delivery"));
|
||||
assert!(AGENT_SYSTEM.contains("Path ladder") || AGENT_SYSTEM.contains("path ladder"));
|
||||
assert!(AGENT_SYSTEM.contains("computerUse is last") || AGENT_SYSTEM.contains("kind=computerUse is last") || AGENT_SYSTEM.contains("force=true"));
|
||||
assert!(AGENT_SYSTEM.contains("re-observe") || AGENT_SYSTEM.contains("Re-observe"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
//! GrokBoy core: config, streaming chat, tools, ReAct agent, sessions.
|
||||
|
||||
mod agent;
|
||||
mod timing;
|
||||
mod box_runtime;
|
||||
mod browser_client;
|
||||
mod computer;
|
||||
|
|
|
|||
|
|
@ -215,6 +215,7 @@ impl Subagents {
|
|||
goal: &str,
|
||||
title: Option<&str>,
|
||||
kind: SubagentKind,
|
||||
force: bool,
|
||||
) -> Result<Value> {
|
||||
if parent.subagent_depth >= MAX_SUBAGENT_DEPTH {
|
||||
return Err(anyhow!(
|
||||
|
|
@ -226,6 +227,12 @@ impl Subagents {
|
|||
"A computerUse subagent is already using the box's desktop. Only one can run at a time."
|
||||
));
|
||||
}
|
||||
// Path ladder: pixels last. Require a lower rung (or explicit force) before computerUse.
|
||||
if kind.is_computer_use() && !force && !parent.tried_web_ladder() {
|
||||
return Err(anyhow!(
|
||||
"Path ladder: try web_fetch/web_search, browser_* DOM tools, or call_mcp_tool before spawn_subagent kind=computerUse. For native GUI / file dialogs / drag, or after DOM already failed, pass force=true."
|
||||
));
|
||||
}
|
||||
let model = parent
|
||||
.model
|
||||
.lock()
|
||||
|
|
@ -246,6 +253,7 @@ impl Subagents {
|
|||
child.subagent_depth = parent.subagent_depth + 1;
|
||||
child.computer_use = kind.is_computer_use();
|
||||
child.box_hub = parent.box_hub.clone();
|
||||
child.surfaces_used = parent.surfaces_used.clone();
|
||||
child.runtime = runtime;
|
||||
*child.model.lock().unwrap() = Some(model.clone());
|
||||
let tool_calls = Arc::new(Mutex::new(0usize));
|
||||
|
|
@ -600,7 +608,7 @@ mod tests {
|
|||
let first = crate::execute_tool(
|
||||
&ctx,
|
||||
"spawn_subagent",
|
||||
&json!({"goal":"click the desktop","kind":"computerUse","title":"gui"}).to_string(),
|
||||
&json!({"goal":"click the desktop","kind":"computerUse","title":"gui","force":true}).to_string(),
|
||||
)
|
||||
.await;
|
||||
let first: Value = serde_json::from_str(&first).unwrap();
|
||||
|
|
@ -609,7 +617,7 @@ mod tests {
|
|||
let second = crate::execute_tool(
|
||||
&ctx,
|
||||
"spawn_subagent",
|
||||
&json!({"goal":"another gui","kind":"computerUse"}).to_string(),
|
||||
&json!({"goal":"another gui","kind":"computerUse","force":true}).to_string(),
|
||||
)
|
||||
.await;
|
||||
let second: Value = serde_json::from_str(&second).unwrap();
|
||||
|
|
@ -678,7 +686,7 @@ mod tests {
|
|||
vec![tc(
|
||||
"s",
|
||||
"spawn_subagent",
|
||||
&json!({"goal":"click once","kind":"computerUse"}).to_string(),
|
||||
&json!({"goal":"click once","kind":"computerUse","force":true}).to_string(),
|
||||
)],
|
||||
));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,21 @@
|
|||
//! Opt-in latency diagnostics. Never logs prompts, arguments or credentials.
|
||||
pub(crate) struct Timing {
|
||||
stage: &'static str,
|
||||
start: Option<std::time::Instant>,
|
||||
}
|
||||
impl Timing {
|
||||
pub(crate) fn new(stage: &'static str) -> Self {
|
||||
Self {
|
||||
stage,
|
||||
start: (std::env::var("GROKBOY_TIMING").as_deref() == Ok("1"))
|
||||
.then(std::time::Instant::now),
|
||||
}
|
||||
}
|
||||
}
|
||||
impl Drop for Timing {
|
||||
fn drop(&mut self) {
|
||||
if let Some(start) = self.start {
|
||||
eprintln!("[timing] stage={} elapsed_ms={}", self.stage, start.elapsed().as_millis());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -4,6 +4,7 @@
|
|||
use anyhow::{anyhow, Context, Result};
|
||||
use serde_json::{json, Value};
|
||||
use std::future::Future;
|
||||
use std::collections::HashSet;
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
use std::pin::Pin;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
|
@ -50,6 +51,8 @@ pub struct ToolContext {
|
|||
/// When true (`run` / tests), wait inside the loop for background work.
|
||||
/// REPL sets this false so the prompt returns; completion revives later.
|
||||
pub hold_background: bool,
|
||||
/// Surfaces already used this turn (path-ladder gate). Shared with child agents.
|
||||
pub surfaces_used: Arc<Mutex<HashSet<String>>>,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for ToolContext {
|
||||
|
|
@ -80,9 +83,45 @@ impl ToolContext {
|
|||
subagent_depth: 0,
|
||||
computer_use: false,
|
||||
hold_background: true,
|
||||
surfaces_used: Arc::new(Mutex::new(HashSet::new())),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn note_surface(&self, name: &str) {
|
||||
if let Ok(mut g) = self.surfaces_used.lock() {
|
||||
g.insert(name.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
pub fn has_surface(&self, name: &str) -> bool {
|
||||
self.surfaces_used
|
||||
.lock()
|
||||
.map(|g| g.contains(name))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// True if any DOM/web ladder rung below pixels was already tried this turn.
|
||||
pub fn tried_web_ladder(&self) -> bool {
|
||||
const NEED: &[&str] = &[
|
||||
"web_fetch",
|
||||
"web_search",
|
||||
"browser_navigate",
|
||||
"browser_snapshot",
|
||||
"browser_click",
|
||||
"browser_type",
|
||||
"browser_select",
|
||||
"browser_press",
|
||||
"browser_wait",
|
||||
"browser_eval",
|
||||
"browser_handoff",
|
||||
"call_mcp_tool",
|
||||
];
|
||||
self.surfaces_used
|
||||
.lock()
|
||||
.map(|g| NEED.iter().any(|n| g.contains(*n)))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
pub async fn has_live_background(&self) -> bool {
|
||||
self.jobs.active().await || self.subagents.has_running()
|
||||
}
|
||||
|
|
@ -359,8 +398,37 @@ fn normalize_path(path: &Path) -> PathBuf {
|
|||
}
|
||||
|
||||
pub async fn execute_tool(ctx: &ToolContext, name: &str, arguments_json: &str) -> String {
|
||||
let surface = match name {
|
||||
"shell" => "box_shell",
|
||||
"read" => "box_read",
|
||||
"await_shell" => "box_await",
|
||||
other => other,
|
||||
};
|
||||
match execute_tool_guarded(ctx, name, arguments_json).await {
|
||||
Ok(v) => v.to_string(),
|
||||
Ok(v) => {
|
||||
const SKIP: &[&str] = &[
|
||||
"send_message",
|
||||
"report_progress",
|
||||
"report_done",
|
||||
"report_blocked",
|
||||
"update_plan",
|
||||
"request_user_input",
|
||||
"request_user_confirm",
|
||||
"spawn_subagent",
|
||||
"check_subagent",
|
||||
"message_subagent",
|
||||
"stop_subagent",
|
||||
"get_mcp_tools",
|
||||
"get_mcp_server_status",
|
||||
"add_mcp_server",
|
||||
"remove_mcp_server",
|
||||
"todo_write",
|
||||
];
|
||||
if !SKIP.contains(&surface) {
|
||||
ctx.note_surface(surface);
|
||||
}
|
||||
v.to_string()
|
||||
}
|
||||
Err(e) => json!({ "error": format!("{e:#}") }).to_string(),
|
||||
}
|
||||
}
|
||||
|
|
@ -489,7 +557,10 @@ async fn execute_tool_inner(ctx: &ToolContext, name: &str, arguments_json: &str)
|
|||
.as_str()
|
||||
.or_else(|| args["subagent_type"].as_str()),
|
||||
)?;
|
||||
ctx.subagents.spawn(ctx, goal, args["title"].as_str(), kind)
|
||||
let force = args["force"].as_bool().unwrap_or(false)
|
||||
|| args["force_pixels"].as_bool().unwrap_or(false);
|
||||
ctx.subagents
|
||||
.spawn(ctx, goal, args["title"].as_str(), kind, force)
|
||||
}
|
||||
"check_subagent" => tool_check_subagent(ctx, &args),
|
||||
"message_subagent" => {
|
||||
|
|
@ -911,7 +982,7 @@ fn extra_tool_definitions() -> Vec<Value> {
|
|||
def("external_glob","Find files by name glob (e.g. **/*.rs) under a path.",json!({"pattern":{"type":"string"},"path":{"type":"string"},"limit":{"type":"integer"}}),json!(["pattern"])),
|
||||
def("web_search","Search public web via the configured remote service. Does not open a browser or use browser logins.",json!({"searchTerm":{"type":"string"},"explanation":{"type":"string"}}),json!(["searchTerm"])),
|
||||
def("web_fetch","Fast anonymous HTTP GET of a public URL; HTML is reduced to readable text with link footnotes. No browser cookies, local profile, localhost access or JavaScript execution. Results are cached briefly, so do not refetch the same URL. If the site blocks plain HTTP the result is a model-rendered summary marked content_kind=model_rendered_web_content.",json!({"url":{"type":"string"},"max_bytes":{"type":"integer"}}),json!(["url"])),
|
||||
def("spawn_subagent","Start a background subagent for a self-contained chunk of work. Returns immediately with subagent_id. Do not wait or poll; keep working or end the turn — you are revived automatically when it finishes. kind=computerUse delegates a GUI/desktop task that drives MY computer by screenshot/click/move/drag/type/key/scroll/wait; only one computerUse may run at a time because they share the screen. Prefer browser_* for ordinary web; use computerUse for GUI apps, file dialogs, drag, or sites that defeat page-level automation.",json!({"goal":{"type":"string"},"title":{"type":"string"},"kind":{"type":"string","enum":["general","computerUse"],"description":"general (default) or computerUse"},"subagent_type":{"type":"string","description":"Alias of kind (Grok Bot Task subagent_type)"}}),json!(["goal"])),
|
||||
def("spawn_subagent","Start a background subagent for a self-contained chunk of work. Returns immediately with subagent_id. Do not wait or poll; keep working or end the turn — you are revived automatically when it finishes. kind=computerUse delegates a GUI/desktop task that drives MY computer by screenshot/click/move/drag/type/key/scroll/wait; only one computerUse may run at a time because they share the screen. Path ladder: do not spawn computerUse for ordinary web until web_fetch/web_search and/or browser_* (or call_mcp_tool) have been tried this turn; set force=true for native GUI, file dialogs, drag, or sites that already defeated page-level automation.",json!({"goal":{"type":"string"},"title":{"type":"string"},"kind":{"type":"string","enum":["general","computerUse"],"description":"general (default) or computerUse"},"subagent_type":{"type":"string","description":"Alias of kind (Grok Bot Task subagent_type)"},"force":{"type":"boolean","description":"Bypass path-ladder gate for computerUse when the task is a native GUI, file dialog, drag, or a site that already defeated DOM automation"}}),json!(["goal"])),
|
||||
def("check_subagent","Inspect a running background subagent (status, elapsed time, recent tools). Omit subagent_id to list all. Not for polling completion.",json!({"subagent_id":{"type":"string"}}),json!([])),
|
||||
def("message_subagent","Inject an instruction into a running subagent without aborting it. It keeps its context.",json!({"subagent_id":{"type":"string"},"message":{"type":"string"}}),json!(["subagent_id","message"])),
|
||||
def("stop_subagent","Abort a running background subagent.",json!({"subagent_id":{"type":"string"}}),json!(["subagent_id"])),
|
||||
|
|
@ -1787,6 +1858,63 @@ mod tests {
|
|||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn path_ladder_tracks_surfaces_and_tried_web() {
|
||||
let (ctx, dir) = temp_ctx();
|
||||
assert!(!ctx.tried_web_ladder());
|
||||
ctx.note_surface("box_shell");
|
||||
assert!(!ctx.tried_web_ladder());
|
||||
ctx.note_surface("web_fetch");
|
||||
assert!(ctx.tried_web_ladder());
|
||||
assert!(ctx.has_surface("web_fetch"));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn computer_use_spawn_requires_ladder_or_force() {
|
||||
let (ctx, dir) = temp_ctx();
|
||||
// No model → spawn fails for other reasons after ladder; check ladder error first.
|
||||
let blocked = ctx.subagents.spawn(
|
||||
&ctx,
|
||||
"open example.com via pixels",
|
||||
Some("pix"),
|
||||
crate::subagents::SubagentKind::ComputerUse,
|
||||
false,
|
||||
);
|
||||
let err = blocked.unwrap_err().to_string();
|
||||
assert!(
|
||||
err.contains("path ladder") || err.contains("Path ladder") || err.contains("force"),
|
||||
"expected path-ladder error, got: {err}"
|
||||
);
|
||||
let forced = ctx.subagents.spawn(
|
||||
&ctx,
|
||||
"open Finder via GUI",
|
||||
Some("gui"),
|
||||
crate::subagents::SubagentKind::ComputerUse,
|
||||
true,
|
||||
);
|
||||
// force passes the ladder; may still fail without model client
|
||||
let err2 = forced.unwrap_err().to_string();
|
||||
assert!(
|
||||
!err2.to_lowercase().contains("path ladder"),
|
||||
"force should bypass ladder, got: {err2}"
|
||||
);
|
||||
ctx.note_surface("browser_snapshot");
|
||||
let after = ctx.subagents.spawn(
|
||||
&ctx,
|
||||
"click after DOM",
|
||||
Some("dom"),
|
||||
crate::subagents::SubagentKind::ComputerUse,
|
||||
false,
|
||||
);
|
||||
let err3 = after.unwrap_err().to_string();
|
||||
assert!(
|
||||
!err3.to_lowercase().contains("path ladder"),
|
||||
"ladder should be satisfied, got: {err3}"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn browser_tool_fails_closed_without_chromium_ok_with_helper() {
|
||||
let _env_lock = crate::test_env::lock_async().await;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
# Core gaps vs Grok Bot usability (LazyBoy2 / GrokBoy)
|
||||
|
||||
Living checklist for the four product cores. Update when behavior changes.
|
||||
|
||||
| Core | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| 1 Completion contract | **Present** | `send_message`-only voice; ack ≠ delivery; Waiting + revival; silence / deliver-then-yield reminders |
|
||||
| 2 Path ladder | **Hardened** | Prompt section + runtime gate: `spawn_subagent kind=computerUse` requires prior `web_fetch` / `web_search` / `browser_*` / `call_mcp_tool` this turn, or `force=true` for native GUI |
|
||||
| 3 Human collaboration | **Present + tightened** | `browser_handoff` / `request_box_help` on shared Chrome profile; prompt requires re-observe after human returns |
|
||||
| 4 Stable single box | **Mostly present** | Docker box + `/home/box/chrome-profile`; SingletonLock/flock in box runtime — keep health/docs polished |
|
||||
|
||||
## Path ladder (runtime)
|
||||
|
||||
Implemented in `ToolContext::surfaces_used` + `Subagents::spawn(..., force)`:
|
||||
|
||||
- Successful tool calls (except meta/control) record a surface name.
|
||||
- `tried_web_ladder()` is true after any of: `web_fetch`, `web_search`, `browser_*`, `call_mcp_tool`.
|
||||
- `computerUse` without ladder and without `force` returns a clear error.
|
||||
|
||||
## Still soft / follow-ups
|
||||
|
||||
- Ladder is turn-scoped (shared Arc with children) — not persisted across Waiting revivals unless the same `ToolContext` is reused.
|
||||
- Parent can still call `screenshot` without DOM first (read-only; intentional).
|
||||
- No automatic “browser_* failed → suggest force” helper beyond the error string.
|
||||
- Box health / profile wipe recovery UX can still be clearer in `PRODUCT.md` / CLI.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
cargo test -p grokboy-core path_ladder -- --nocapture
|
||||
cargo test -p grokboy-core computer_use_spawn -- --nocapture
|
||||
cargo test -p grokboy-core agent_system -- --nocapture
|
||||
```
|
||||
|
|
@ -15,6 +15,7 @@ The interaction rules below describe legacy single-session mode. Named-agent mod
|
|||
- For action tasks, the first action is `send_message`, then tools. Multi-stage work uses a short `update_plan` checklist.
|
||||
- `send_message` is the only user-visible voice. Plain assistant text is a scratchpad. A no-tool response ends the turn (Grok Bot loop). `report_progress` is a text `send_message` alias and continues.
|
||||
- Commands background after `block_until_ms` (default 30s). Jobs are not killed on `answer`/`done`.
|
||||
- Path ladder (runtime): try MCP/files → `web_fetch`/`web_search` → `browser_*` → pixels; `computerUse` needs a lower rung or `force=true`. See [CORE-GAPS](CORE-GAPS.md).
|
||||
- `spawn_subagent` returns immediately. If the model yields while a subagent or command is still running, the runtime waits and injects a revival (Grok Bot background completion). `check_subagent` / `message_subagent` / `stop_subagent` manage live children; do not poll for completion.
|
||||
- Human questions, confirmations and handoff **end the turn**. The next user message is the answer; there is no timeout-and-continue.
|
||||
- In the REPL, background commands/subagents return the prompt. Completion injects a revival and continues automatically; typed input meanwhile is steering. `grokboy run` still waits in-process so the job is not lost when the process exits.
|
||||
|
|
|
|||
Loading…
Reference in New Issue