Web Agent

AgentX Phase 2 Computer Use & Web Agent Track. Reflexive Agent Architecture for WebShop+ (budget management, preference memory, negative constraints, comparative reasoning, error recovery, cart management, checkout).

The Green Agent in the webjudge-agents agentifies the Online-Mind2Web benchmark, creating an autonomous judge for web navigation tasks. It manages the complete lifecycle—distributing tasks from the Mind2Web dataset and performing rigorous, multi-modal assessments of participant trajectories. Its evaluation engine implements the comprehensive three-stage methodology defined by the original Online-Mind2Web benchmark: first, using Large Language Models (LLMs) to decompose natural language instructions into verifiable key points and constraints; second, applying visual reasoning to score the operational relevance of intermediate screenshots; and finally, determining a binary success verdict based on the strict satisfaction of all extracted requirements. Participant agents are measured against a detailed set of metrics, including the overall success rate and total task completion count, alongside execution efficiency metrics such as task duration and the total number of steps taken.

The Green Agent utilizes the WebJudge framework—an 'LLM-as-a-judge' system designed to replace unreliable pass/fail metrics. It identifies critical task requirements, filters for relevant screenshots of the agent's progress, and makes a final success judgment based on action history. This system evaluates agents against the Online-Mind2Web benchmark, which consists of 300 tasks across 136 real-world websites. The diversity of the benchmark suggests that our Green Agent can effectively act as a reward model or evaluator for tasks it has never seen before in the area of web browsing tasks.