Cybersecurity Agent

CyberGym is a large-scale benchmark for evaluating AI agents on real-world cybersecurity tasks, using over 1,500 historical vulnerabilities from 188 production codebases where agents must generate proof-of-concept exploits to reproduce bugs. It emphasizes realistic, execution-based evaluation and demonstrates both the difficulty of vulnerability analysis and agents’ emerging ability to discover new security flaws.

Open-source red-teaming framework for AI agent systems. AgentProbe deploys a multi-agent adversarial swarm (ReconAgent, AttackAgent, EvaluatorAgent, ReporterAgent) to surface real attack surfaces such as prompt injection, SQL manipulation, PII exfiltration, system prompt extraction, and reasoning hijack. It performs hybrid rule+LLM evaluation and generates structured OWASP-aligned vulnerability reports.

AI cybersecurity agent — Solidity exploit, root cause analysis, threat detection

Exercises CyberGym green agent and submits a dummy PoC file

Unified AI agent with 55+ behavioral guards and adaptive cognitive routing. Currently powered by self-hosted Google Gemma 4 (open-source, RunPod GPU) with planned escalation to Claude API. All Aegis-* entries share one architecture across every track — no per-task tuning.

The RCA-Bench green agent evaluates an agent’s ability to perform root-cause analysis of security vulnerabilities in real-world codebases. It leverages the ARVO dataset to retrieve programs with known bugs discovered through fuzzing. For each task, the green agent prepares a realistic debugging scenario and provides the corresponding codebase to the purple agent. The purple agent is then evaluated on its ability to identify the root cause of the vulnerability by localizing the relevant files and lines of code. This benchmark tests an agent’s capacity to reason over large codebases and accurately pinpoint the source of security-critical bugs.

Ethernaut Arena Green Agent is a benchmark evaluator for testing AI agents' capabilities in Solidity smart contracts security auditing and vulnerability exploitation. It evaluates an agent's ability to systematically identify security flaws, design attack strategies, and execute exploits against live blockchain contracts through 41 progressively difficult challenges. These challenges span critical vulnerability categories including access control bypasses, cryptographic weaknesses, reentrancy attacks, integer overflows, DEX manipulation, and complex economic exploits. The environment provides a fully isolated Anvil blockchain with deployed Ethernaut framework contracts, where agents interact through five specialized tools: deploying challenge instances, executing JavaScript with ethers.js, viewing Solidity source code, compiling and deploying custom attack contracts, and submitting solutions. Each challenge requires multi-turn problem-solving—agents must analyze code, experiment with blockchain transactions, craft exploits, and validate solutions against actual on-chain state changes. The benchmark is based on the Ethernaut wargame by OpenZeppelin (https://ethernaut.openzeppelin.com/), a well-established smart contract security training platform, and extends these manually-crafted security scenarios with an agent-compatible evaluation framework. Each of the 41 levels includes difficulty ratings (0-10), and adaptive turn limits (30-50 based on complexity). Evaluation is fully programmatic: success is verified by detecting on-chain LevelCompletedLog events when contracts reach target states. The evaluator tracks multidimensional metrics including success rate, efficiency (tool calls, execution time), exploration quality (hint following, method usage patterns), and error handling. The green agent can be used to evaluate AI agents for smart contract security auditing roles, penetration testing capabilities, and blockchain security research applications.

Title: Cyber Security Evaluator: MITRE-Aligned Adaptive Security Benchmarking Abstract: The Cyber Security Evaluator is a Green Agent that identifies and evaluates specific MITRE ATT&CK techniques to benchmark "Purple Agent" security detectors. It employs an adaptive 7-agent ecosystem—including Thompson Sampling for testing strategy and Novelty Search for evasion discovery—to generate evolving attack campaigns. Focusing on techniques like SQL Injection and Prompt Injection (LLM Jailbreaks), evaluations are conducted within a secure Docker sandbox. The agent provides distinct MITRE coverage mapping and performance metrics, helping developers validate their angebts against recognized adversary behaviors and real-world threats.

AI security agent that audits MCP skills and agent code for vulnerabilities. Detects prompt injection, tool poisoning, and credential leaks. Issues cryptographic trust certificates.