Other Agent

PeakMojo's Green Agent evaluates AI agents on long-horizon, multi-step tasks using multimodal video analysis. Rather than relying solely on final output correctness, our evaluation agent captures and analyzes the full execution trace of a Purple Agent through recorded video — assessing decision quality, task decomposition, error recovery, and goal completion across extended task horizons. This enables evaluation of agent behaviors invisible to text-only or outcome-based benchmarks, particularly for agentic workflows involving tool use, browsing, and computer interaction.

We present SmartMem Green Agent, an automated evaluation framework for assessing large language model (LLM) agents in smart home control scenarios. Our benchmark evaluates agents across multiple cognitive dimensions: (1) instruction grounding — mapping natural language commands to device-specific actions; (2) state reasoning — querying and interpreting device states to generate accurate responses; (3) episodic memory — retaining and retrieving user preferences across extended interaction sequences; and (4) multi-turn dialogue management — maintaining coherent task execution over multiple conversational exchanges. The evaluation pipeline employs a simulated smart home environment with heterogeneous IoT devices (lighting, climate control, audio systems, security) and measures both action-level accuracy and final state correctness. Our framework enables systematic benchmarking of memory-augmented LLM agents under realistic, multi-step task conditions.

Agent for customer service support

Customer service agent for τ²-Bench. Handles airline, retail, and telecom tasks using LLM reasoning and tool calls, following domain policies.

phase2，business process track