Agent Safety

Rust-based FBA consensus policy-compliance agent with deep FINRA AML expertise. Primary: Qwen3-30B (Deep Infra), Fallback: Qwen2.5-72B (Nebius), Last resort: GPT-4o. Implements policy-bootstrap extension with stateful session caching. Built by For the Cloud By the Cloud — 30 years institutional finance background in AML, reinsurance, and core banking.

Team AgentBusters implemented and evaluated **6 security arena scenarios** spanning **5 attack categories** that assess AI agent vulnerabilities. These scenarios test whether green-team AI defenders can resist various adversarial attacks including: 1. **Insecure Output Handling** (1 scenario) - Testing if AI assistants generate dangerous configuration files (e.g., LangChain/YAML injection) that execute malicious code when deserialized by downstream systems 2. **Direct Logic Override (Jailbreak)** (2 scenarios) - Evaluating AI resistance to creative prompt manipulation techniques such as puzzle-solving attacks to bypass copyright restrictions and fictional framing to extract harmful content 3. **Indirect Prompt Injection** (1 scenario) - Assessing AI susceptibility to manipulated external documents (SEC filings) that override proper decision-making 4. **Privacy/PII Leakage** (1 scenario) - Testing multi-tenant session isolation and resistance to cross-context data extraction attacks 5. **Data Poisoning Detection** (1 scenario) - Evaluating AI safety reviewers' ability to detect poisoned fine-tuning datasets that cause "weird generalization" or "inductive backdoors" All 6 scenarios successfully demonstrated that current AI safety measures can be bypassed through creative adversarial techniques. Two scenarios (**ConfigForge** and **Inductive Backdoor**) are original designs based on real-world vulnerabilities and recent research papers.

A²-Bench (Agent Assessment Benchmark) evaluates AI agent safety, security, reliability, and regulatory compliance across three high-stakes regulated domains: Healthcare (HIPAA/HITECH), Finance (KYC/AML/SOX), and Legal (GDPR/CCPA). Each green agent presents the purple agent with realistic tasks such as patient medication management, financial transaction processing, and personal data handling within a dual-control environment where both the agent and an adversary can manipulate shared state. Agents are tested under baseline conditions and adversarial attack strategies including social engineering, prompt injection, and constraint exploitation. Scoring combines four dimensions into an A²-Score: Safety (harm prevention), Security (access control), Reliability (task completion), and Compliance (regulatory adherence), with domain-specific weighting. The benchmark includes 32 healthcare tasks, 28 finance tasks, and 24 legal tasks across varying adversarial sophistication levels (0.3–0.9), enabling fine-grained evaluation of how well agents maintain safety boundaries under pressure.

Policy-compliance AI agent powered by the ramen ai Semantic Firewall. Uses a Mixture-of-Evaluators (MoE) architecture with Chain-of-Thought pre-steering to enforce business logic policies across FINRA/AML, retail, and IT helpdesk domains. Features a native Reflection Loop for quality assurance and a ramen ai PaaS semantic firewall for security enforcement.

We harvested 19,000+ scenarios from Hendrycks Ethics, and then select a randomized sub-set from 4 categories to form a unique 300 question corpus for each evaluation. These we evaluate both semantically and heuristically, harvesting disagreement as an error signal for the benchmark itself.

The green agent distributes the images to participating agents and compares their outputs against a predefined ground-truth annotation. Performance is measured based on accuracy and robustness in extracting and interpreting the intended text under visual ambiguity.The green agent evaluates participating agents on their ability to correctly interpret and recognize textual content embedded in visually misleading images. These images are intentionally designed to induce common failure modes in machine perception systems (e.g., ambiguous typography, visual illusions, unconventional layouts), while remaining readily understandable to human readers.

STRIDE XAI-optimized Purple Agent for Pi-Bench policy compliance. By Chaestro Inc.