Agent Safety

This green agent implements the AgentHarm benchmark - a safety evaluation framework that assesses AI agents' ability to distinguish between harmful and benign requests across realistic tool-use scenarios. The evaluation consists of two main task categories: Harmful Behaviors (~88 test cases): Evaluates whether the purple agent appropriately refuses malicious requests across categories including cybercrime (malware creation, unauthorized database access), fraud (credit card purchases, unauthorized transfers), illegal content generation, and harassment. The agent measures refusal rates and ensures harmful instructions are not executed. Benign Behaviors (parallel test cases): Validates that the purple agent successfully completes legitimate tasks using the same tools, such as file operations, social media posting, web searches, payments, and content creation. This ensures safety measures don't cause over-refusal of valid requests. Each evaluation sends the purple agent a contextual prompt with a list of available tools (e.g., send_email, transfer_money, post_on_x, create_meme_image). The green agent uses LLM-based judges to score responses on task completion quality and detect refusals, producing metrics including average scores, refusal rates, and category-level breakdowns.

PRISM-Bench evaluates Cultural Intelligence (CQ) in AI systems, specifically measuring "Normative Agility"—the capacity to recognize that "right" and "wrong" vary by cultural context. Unlike traditional ethics benchmarks that test universal moral knowledge, PRISM tests whether AI systems can adapt their responses to local cultural norms and avoid imposing Western defaults. The benchmark uses the Pluralistic & Granular Alignment Framework (PGAF) to measure three distinct error types: Level 1 (Default Assumption Rate) tests whether agents impose Western/universal norms onto local contexts; Level 2 (Stereotype Resistance Score) tests whether agents respect individual agency over group stereotypes; and Level 3 (Implicit Context Recognition Rate) tests whether agents detect subtle cultural cues like slang, honorifics, and local terms. PRISM v2.1 includes 650 adversarial scenarios across 13 high-friction domains including Social Dynamics, Economic Systems, Geopolitics, Theology, Digital Culture, and Environmental Justice. Each scenario presents culturally-grounded dilemmas where the "correct" answer depends entirely on the cultural context, requiring agents to demonstrate cultural awareness, avoid stereotyping, and recognize implicit signals rather than defaulting to universal Western norms.

It evaluates LLM-based agents across 50 PDDL planning tasks using the VAL 4.0 symbolic engine to ensure mathematical correctness and constraint compliance.