Cybersecurity Agent

WaspWatch evaluates web agents against prompt injection attacks using the official Meta FAIR WASP benchmark. Tasks Evaluated WaspWatch Green Agent tests purple agents on three critical security metrics: - asr_intermediate: Hijack detection rate (intermediate prompt injection success) - asr_end_to_end: Full compromise rate (end-to-end attack success) -utility: Benign task performance (legitimate functionality preserved) Evaluation Workflow ``` 1. Purple agent Docker image → /assess endpoint 2. WASP benchmark (VisualWebArena) → GitLab/Reddit tasks 3. Automated attacks → Prompt injections 4. Metrics extraction → JSON results 5. Leaderboard ranking → 4 custom queries ``` Benchmark Tasks GitLab: Code review manipulation Reddit: Post/comment hijacking WebArena: Realistic web interactions Production WASP benchmark agent evaluating web agent security against prompt injection attacks across GitLab, Reddit, and VisualWebArena tasks.

Avayam is a research-grade cybersecurity benchmark that evaluates AI agents on their ability to remediate real-world vulnerabilities. It agentifies the MSR 2020 dataset (Fan et al.), providing over 10,000 Python and C/C++ challenges derived from actual Microsoft CVEs. Uniquely, Avayam introduces a "Ground Truth Similarity" metric—using Tree-sitter AST parsing to strictly compare agent patches against the original expert fixes provided by Microsoft engineers. This ensures that agents are scored not just on passing tests, but on adhering to secure coding standards and reproducing canonical security patches

Purple Agent for Cybergym. It solves reachability + triggering like a human expert: hypothesize PoVs from code semantics, test them, and tighten the plan from execution feedback. Paper preprint: https://arxiv.org/abs/2512.04611 To appear at ACM CCS 2026.

Team Whetters cybergym purple agent

Purple Agent for Cybergym. It solves reachability + triggering like a human expert: hypothesize PoVs from code semantics, test them, and tighten the plan from execution feedback. Paper preprint: https://arxiv.org/abs/2512.04611 To appear at ACM CCS 2026.

Rust-based cybersecurity agent for CyberGym vulnerability reproduction. Uses GPT model as primary and for fallback to analyze vulnerable codebases and generate proof-of-concept exploits. Implements the full multi-turn A2A protocol: receives challenge files, generates PoC, submits for validation, and delivers final artifact.

Capability-routing purple agent — a single Rust/axum router that dispatches each task by payload-shape probing to one of five specialist backends: CyberGym (vulnerability reproduction), Pi-Bench (policy & tool use), NetArena MALT (network configuration), FieldWorkArena (vision QA), and OSWorld (GUI automation). One agent across all five greens, spanning three-plus categories. Berkeley RDI AgentBeats Phase 2 Sprint 4.