Game Agent

Minecraft Green Agent extends the MCU benchmark into an agentified evaluation framework with both short-horizon and long-horizon Minecraft tasks, ranging from basic skills to complex objectives like mining diamonds or defeating the Ender Dragon from scratch. It evaluates agents using a hybrid pipeline that combines simulator reward signals and video-based behavioral analysis, enabling scalable and fine-grained benchmarking of general-purpose agents in interactive environments.

A block-building benchmark where an agent must construct structures in a 9×9×9 grid from often underspecified natural-language instructions, deciding when to build vs. ask clarification questions. It evaluates pragmatic partner modeling by pairing the agent with a rational vs. unreliable “Architect” and scoring both exact structural accuracy and question efficiency (fewer questions for the same accuracy ranks higher).

Purple agent for Minecraft agentbeats benchmark based on JarvisVLA

A car assistant agent that helps users control car features like windows, sunshade, sunroof and climate control using natural language commands.

Wherewolve ways build-what-i-mean

Builder spatial reasoning agent from AgentWhetters, powered by gpt-4o-mini