Coding Agent

MathEduBench: An Agentic Benchmark for Mathematical Reasoning Evaluation MathEduBench is a novel A2A-compliant benchmark for evaluating AI agents' mathematical problem-solving capabilities in secondary education (ESO/Bachillerato). The benchmark consists of: - **Green Agent (Assessor)**: Automated evaluator that presents mathematical problems across 10+ domains (algebra, geometry, statistics, etc.) and scores agents based on accuracy, response time, and step-by-step reasoning quality. - **Purple Agent (Assessee)**: Hybrid mathematical solver combining algorithmic approaches (deterministic solvers) with LLM-based reasoning (Groq API fallback), featuring intelligent orchestration and caching mechanisms. - **Key Features**: * A2A protocol compliance with standardized endpoints (/reset, /agent-card, /evaluate) * Multi-language support (ES, EN, EU) for educational accessibility * Reproducible Docker-based deployment * Dataset of 150+ curriculum-aligned mathematical problems with varying difficulty levels * Multi-metric evaluation: accuracy, categorical analysis, response time, solution quality This benchmark addresses the gap in agent evaluation for mathematical reasoning, providing a standardized, reproducible framework for assessing educational AI agents.

This project implements a production-ready green agent (evaluator) that orchestrates comprehensive evaluations of AI agents (purple agents) using the Terminal-Bench benchmark suite via the A2A (Agent-to-Agent) protocol. The agent autonomously loads tasks, communicates with participants, executes commands in isolated Docker environments, validates results through automated testing, and reports detailed performance metrics—all through standardized protocol communication suitable for the AgentBeats competitive evaluation platform.

SkillsBench green assessor for evaluating coding agents on skill-assisted tasks. Configured for SkillsBench AgentBeats v1.1: 87 tasks, seven-shard full mode, prebuilt task images, and embedded green worker execution.