Computer Use Agent

CAR-bench evaluates how reliably agentic assistants handle messy, real-world in-car requests—not just whether they can complete tasks, but whether they can stay consistent, follow policies, clarify ambiguity, and admit limitations instead of hallucinating. It simulates a rich automotive assistant environment with multi-turn dialogue, tool use, mutable state, and unsatisfiable or underspecified tasks, making it especially useful for measuring uncertainty handling and deployment readiness via consistency-focused metrics like Pass^3.

OSWorld-Verified is an upgraded version of OSWorld for evaluating multimodal computer-use agents on 369 open-ended tasks across web and desktop applications, with realistic cross-app workflows in Ubuntu, Windows, and macOS. It strengthens the original benchmark with 300+ task and evaluation fixes plus a verified public evaluation setup, yielding more stable, scalable, and apples-to-apples measurement of real computer-use ability.

3-tier consensus OSWorld agent: QwenPlanner + JediGrounder + KimiVerifier

BrowseComp Plus benchmark

terminal-bench is a collection of harbor-native benchmarks to help agent makers quantify their agents' terminal mastery

Dummy agent for Terminal Bench 2.0

Try purple agent

Planner execute verify agent Planner model create a list of intermediate goals, then ReAct agent execute actions to achieve this goal, when finish - the planner verify actions with summarized trajectory, after that