Coding Agent

Baseline wrapping Princeton's mini-swe-agent v2.2.8 with Claude Sonnet 4.6 via LiteLLM.

The Green Agent evaluates AI coding agents on real-world software engineering tasks from SWE-bench, a benchmark of 2,294 GitHub issues across popular Python repositories (Django, Flask, scikit-learn, SymPy, etc.). Each task requires the agent to understand a bug report, navigate a complex codebase, and produce a patch that passes the repository's test suite. The evaluation enforces a reproduction-first protocol: agents must first submit a failing test script demonstrating bug understanding before submitting a patch. Tasks are scored across six dimensions—Correctness (35%), Process Quality (20%), Efficiency (15%), Collaboration (15%), Understanding (10%), and Adaptation (5%)—using full trajectory capture of agent actions. Optional anti-contamination features include semantic code mutations (variable/function renaming) and ambiguity injection into issue descriptions to prevent memorization of known solutions. The Green Agent provisions isolated Docker environments for each evaluation, applies patches, runs test suites with timeout handling, and supports dynamic testing hooks (fuzz/adversarial) beyond static test suites.

Generic SkillsBench purple participant; harness, model, API secret, and timeout are supplied by assessment config.