Coding Agent

A coding agent for SWE-bench Pro that fixes real GitHub issues using mini-swe-agent in Docker sibling containers. Supports multiple LLMs via litellm.

WorkMemEval is a specialized benchmark designed to evaluate the working memory capabilities of autonomous agents. Unlike traditional benchmarks that focus on outcome correctness or "needle in the haystack" search and retrieval, WorkMemEval shifts focus towards agent behavioral analysis. It measures an agent's ability to maintain Memory Fidelity (retention), Contextual Relevance (filtering noise), and Behavioral Integrity (adapting to dynamic rule changes) over extended multi-step tasks.

TDD-first purple agent for coding benchmarks. It writes a minimal failing regression test when repository context is available, verifies the red state, applies production patches as unified diffs, runs targeted and broader tests, and returns a final git diff patch through an A2A endpoint.

Purple Terminal Agent is a Mixture-of-Model (MoM) yielding REPL driven hierarchical planning and domain specific critic-guided execution agent designed for hard, realistic command-line tasks. Given a task and a live shell endpoint, it decomposes the problem into ordered sub-goals before issuing any command, pre-flights every command through a domain-aware critic to prevent interactive hangs and blind pattern-copying, and self-verifies by running test scripts before declaring completion. The agent scales inference-time depth through three mechanisms: a hierarchical planner that forces full-task reasoning before execution, a critic sub-agent that adds a reasoning layer per command, and a build-time TF-IDF RAG index over Terminal Bench oracle tasks that injects scaffold-framed hints from similar tasks. Multi-domain tasks are handled via multi-label detection — the primary domain receives a full reasoning scaffold while secondary domains contribute pitfall warnings only, preventing instruction satiation and reward hacking observed in prior ICL-heavy designs. Moreover REPL encoded design helps the agent in enhancing its complex problem skills within a single session run. A session-scoped task memory caches only verifier-confirmed command sequences, accumulating cross-task knowledge within a single evaluation run without propagating unverified patterns. MoM Purple Agent is budget friendly with average run costs $9.5/run (1 run = 89 tasks). This is in line with our quest: Can a perfect Terminal Bench 2.0 coding agent be constructed in a resource constrained setting? Apart from the REPL enhanced design, non-REPL version with DeepSeek-v4-flash costs less than $2.0 per run and was able to solve 30 out of 89 problems in a single run! Model: Gemini-3-flash-preview + DeepSeek-v4-pro + DeepSeek-v4-flash via OpenRouter · Max turns: 30 · Image: docker.io/rimodock/purple-terminal-agent:latest