Coding Agent

WorkMemEval is a specialized benchmark designed to evaluate the working memory capabilities of autonomous agents. Unlike traditional benchmarks that focus on outcome correctness or "needle in the haystack" search and retrieval, WorkMemEval shifts focus towards agent behavioral analysis. It measures an agent's ability to maintain Memory Fidelity (retention), Contextual Relevance (filtering noise), and Behavioral Integrity (adapting to dynamic rule changes) over extended multi-step tasks.

The Purple Coding Agent is a high-performance, autonomous software engineering agent optimized for repository-level reasoning and complex bug resolution in competitive environments like SWE-Bench Pro and AIMO2026. Operating on a stateful Phase 2 architecture, the agent moves beyond static code analysis by utilizing a live, execution-grounded environment. It autonomously explores codebases, reproduces issues within isolated Docker containers, and verifies its own repairs through a mechanical test gate to ensure production-grade reliability. Key Capabilities Stateful Bash REPL: Maintains a persistent, 50-turn interactive session that allows the agent to explore, edit, and verify code iteratively within a single unified context. Mechanical Ground Truth: Utilizes a Docker-out-of-Docker (DooD) bridge to spawn sibling containers, allowing it to run test suites natively and generate its own diagnostic logs. Inference-Time Scaling (GRPO): Employs group sampling strategies to generate and evaluate multiple diagnostic hypotheses simultaneously, prioritizing leads based on real-world execution feedback. Graph-Based RAG: Leverages Tree-Sitter for AST-based repository mapping, providing the agent with a structural "skeleton" of the codebase to prevent context wandering in large repositories. Relative Reward Verification: Implements a smarter QA gate that compares post-fix execution results against a baseline state to prevent regressions and ensure the core issue is resolved. Automated Tooling: Seamlessly integrates specialized models (e.g., DeepSeek-v4-flash) with local bash utilities to perform batched file reads and robust Python-based edits.