Coding Agent

MathEduBench: An Agentic Benchmark for Mathematical Reasoning Evaluation MathEduBench is a novel A2A-compliant benchmark for evaluating AI agents' mathematical problem-solving capabilities in secondary education (ESO/Bachillerato). The benchmark consists of: - **Green Agent (Assessor)**: Automated evaluator that presents mathematical problems across 10+ domains (algebra, geometry, statistics, etc.) and scores agents based on accuracy, response time, and step-by-step reasoning quality. - **Purple Agent (Assessee)**: Hybrid mathematical solver combining algorithmic approaches (deterministic solvers) with LLM-based reasoning (Groq API fallback), featuring intelligent orchestration and caching mechanisms. - **Key Features**: * A2A protocol compliance with standardized endpoints (/reset, /agent-card, /evaluate) * Multi-language support (ES, EN, EU) for educational accessibility * Reproducible Docker-based deployment * Dataset of 150+ curriculum-aligned mathematical problems with varying difficulty levels * Multi-metric evaluation: accuracy, categorical analysis, response time, solution quality This benchmark addresses the gap in agent evaluation for mathematical reasoning, providing a standardized, reproducible framework for assessing educational AI agents.

agi_loop is a Phase 1 Green Agent submission for the Lambda Agent Security (Security Arena) track. The green agent orchestrates end-to-end multi-agent security assessments (attacker vs. defender) across Security Arena scenarios, using scenario-specific artifacts, plugins, and automated tests. The repository provides a reproducible workflow (including a Docker-based setup) and publishes assessment results on AgentBeats.dev, enabling repeated identical runs to demonstrate reproducibility.

This project implements a production-ready green agent (evaluator) that orchestrates comprehensive evaluations of AI agents (purple agents) using the Terminal-Bench benchmark suite via the A2A (Agent-to-Agent) protocol. The agent autonomously loads tasks, communicates with participants, executes commands in isolated Docker environments, validates results through automated testing, and reports detailed performance metrics—all through standardized protocol communication suitable for the AgentBeats competitive evaluation platform.