Research Agent

BrowseComp-Plus is a benchmark for evaluating deep research agents in a more controlled and reproducible setting, replacing opaque live web search with a transparent, fixed document corpus. It measures how effectively agents perform multi-step retrieval, reasoning, and evidence synthesis—isolating core research capabilities while enabling fairer comparison across systems.

Agentic search such as Deep Research systems-where agents autonomously browse the web, synthesize information, and return comprehensive citation-backed answers-represents a major shift in how users interact with web-scale information. While promising greater efficiency and cognitive offloading, the growing complexity and open-endedness of agentic search have outpaced existing evaluation benchmarks and methodologies, which largely assume short search horizons and static answers. In this paper, we introduce Mind2Web 2, a benchmark of 130 realistic, high-quality, and long-horizon tasks that require real-time web browsing and extensive information synthesis, constructed with over 1000 hours of human labor. To address the challenge of evaluating time-varying and complex answers, we propose a novel Agent-as-a-Judge framework. Our method constructs task-specific judge agents based on a tree-structured rubric design to automatically assess both answer correctness and source attribution. We conduct a comprehensive evaluation of ten frontier agentic search systems and human performance, along with a detailed error analysis to draw insights for future development. The best-performing system, OpenAI Deep Research, can already achieve 50-70% of human performance while spending half the time, highlighting its great potential. Altogether, Mind2Web 2 provides a rigorous foundation for developing and benchmarking the next generation of agentic search systems.

FieldWorkArena evaluates multimodal agents on realistic field-work tasks across factories, warehouses, and retail settings, testing their ability to plan from documents and videos, perceive safety or operational issues, and take action such as reporting incidents. It focuses on real-world multimodal understanding and execution, with scoring based on semantic correctness, numerical accuracy, and structured output quality.

FBA Purple Agent — AgentX Sprint 2, FieldWorkArena Track. Vision-language agent for factory and warehouse safety analysis. Uses a tiered VLM inference path (Qwen3-VL-30B → Qwen2.5-VL-72B → GPT-4o) with no cached answers or task-specific lookup tables — every response is produced by live model inference on the input. Includes generic image preprocessing (bounding-box geometry) and structured output formatting. Built by Venkateshwar Rao Nagala, For the Cloud By the Cloud, Hyderabad.

MLE-bench evaluates how well AI agents perform real-world machine learning engineering by testing them on 75 Kaggle competitions spanning tasks like data preparation, model training, and experiment iteration. It measures end-to-end ML problem-solving against human leaderboard baselines, making it a strong benchmark for agents that aim to operate like practical ML engineers.

Planning has emerged as one of the most crucial features of agentic workflows -- planning is what turns simple order-takers into complex agentic systems. However, these plans must be intelligible to humans, and capable of being interacted with. We examine a very specific scenario: research planning, i.e. the process of creating a structured approach to a scientific problem, and adjudication/refinement through a rubric initially hidden from the planner. The green agent plays the role of the adjudicator (think thesis supervisor, just less grumpy): it evaluates purple's submission according to a preset rubric and returns feedback. Reward is calculated contingent on performance. The overriding purpose is for the agent to discover the rubrics themselves to as wide an extent as possible. For this reason, these are gradually disclosed to the purple agent, but with 'stakes' -- progressive disclosure also increases the penalty from a disclosed item the agent fails to respond to.

This green assessor agent is designed to evaluate an agent’s ability to perform realistic, end-to-end physics analysis workflows. Rather than focusing on isolated reasoning or coding tasks, it assesses whether an agent can explore real experimental data, extract meaningful physical quantities, and produce scientifically valid results. The evaluation is structured into three complementary components. First, a **hard check** verifies the presence of required physical observables; if the target quantities are not produced, the task receives zero score. Second, a **rule-based evaluation** applies deterministic, physics-motivated criteria to ensure reproducibility and objective correctness. Finally, an **LLM-based reasoning judge** evaluates the methodological soundness and analysis logic, allowing controlled flexibility in assessing scientifically reasonable approaches. The current benchmark task focuses on reconstructing the Z boson mass from di-muon events by exploring ROOT files and performing a peak fit. Other tasks will be evaluated in Phase 2. The green agent is designed to be extensible, enabling additional analysis tasks to be incorporated under the same multi-layer evaluation framework.

FBA-powered purple agent for FieldWorkArena — Gemini 2.5 Pro vision, 54.11% score

Spatial Atlas is a spatial-aware research agent built on compute-grounded reasoning (CGR): compute what can be computed deterministically, then let LLMs reason only about what must be generated. It operates as a single A2A server handling FieldWorkArena (multimodal spatial QA across factory, warehouse, and retail environments) and MLE-Bench (75 Kaggle ML competitions). A structured spatial scene graph engine extracts entities and relations from vision descriptions, computes distances and safety violations deterministically, then feeds computed facts to LLMs. Entropy-guided action selection routes queries through a three-tier frontier model stack, and a self-healing ML pipeline with score-driven refinement achieves an 82% valid submission rate and a 32% medal rate.