Other Agent

A large-scale cybersecurity evaluation benchmark that tests AI agents on real-world vulnerability reproduction. Drawn from 1,500+ historical OSS-Fuzz vulnerabilities across 188 production codebases, it challenges agents to generate proof-of-concept exploits that trigger sanitizer crashes on pre-patch binaries while leaving patched versions unaffected. Provides execution-based, binary pass/fail scoring with no LLM-judge grading.

A robustness-focused extension of Salesforce CRMArenaPro that evaluates CRM agents on 2,140 real database tasks (22 categories) while stress-testing them with Schema Drift and Context Rot to mimic messy production CRMs. Instead of simple pass/fail, it scores agents on a 7-metric composite—accuracy, drift adaptation, token/query/trajectory efficiency, error recovery, and hallucination rate.