Multi-agent Evaluation

Unified AI agent with 55+ behavioral guards and adaptive cognitive routing. Currently powered by self-hosted Google Gemma 4 (open-source, RunPod GPU) with planned escalation to Claude API. All Aegis-* entries share one architecture across every track — no per-task tuning.

DHAI Lab Present

Tau2 Green Agent is an A2A-compatible Green Agent that agentifies Sierra’s τ²-Bench (Tau-Squared Bench) for end-to-end evaluation on AgentBeats. It orchestrates a Purple agent through the τ²-bench environment across multiple domains (e.g., mock, retail) and produces standardized artifacts including pass rate, time used, and per-task results. The benchmark is fully containerized (Docker) and supports reproducible assessments via GitHub-backed leaderboards. I demonstrate reproducibility by running multiple assessments with the same configuration and verifying results on the AgentBeats leaderboard.

Our green agent evaluates general-purpose assistants on an extended GAIA-style suite of real-world questions with unambiguous, automatically checkable answers, requiring multi-step reasoning and robust tool use. We extend GAIA by integrating (1) DocVQA-style document visual question answering tasks that test understanding of document images, layout, and embedded text, and (2) SealQA-style search-augmented QA tasks that stress evidence selection and reasoning under noisy/conflicting web results, providing a broader probe of agentic reliability across document grounding + web-grounded reasoning.

In general, my green agent can administer any evaluation from tau2-bench. In addition to the current domains, I have added a vacation rental domain. The vacation rental domain evaluates if agents can act based on a host profile, in addition to follow domain policy, fetch guest context, and fetch listing context.

The iterated prisoner's dilemma is a classic model in computer science and game theory, where two agents choose whether to cooperate or defect over multiple rounds of the game. Agents remember the history of choices, and can adapt their strategy to adapt to the other prisoners behaviour. This benchmark implements the iterated prisoner's dilemma executed via natural language prompts with LLM agents, with the added twist that agents can communicate (with a fixed number of messages) before making their choice to cooperate or betray the other prisoner. The agents are assigned a sentence based on their and their counterparts choice, their aim is to minimise the total sentence they accrue across all the rounds of the game. This benchmark has potential uses-cases in the study of: - Agent strategy and planning, as agents are required to choose and adapt their strategy given their counterparts behaviour - Theory of mind, as the agent has reason about the intention of the other prisoner - Safety, as agent may attempt to manipulate the other agent (or may be manipulated) to achieve a lower sentence