Healthcare Agent

NurseSim-Triage evaluates an agent's ability to perform safety-critical clinical triage in Emergency Department scenarios. The agent receives patient presentations (chief complaint, vital signs, demographics, medical history) and must assign the correct Manchester Triage System category (1-5) while providing clinical reasoning. Tasks assess: Risk Stratification - Correctly identifying life-threatening conditions (Category 1: Cardiac arrest, Anaphylaxis, Sepsis) Demographic Context Integration - Weighing age and gender as risk modifiers (e.g., chest pain in 72M vs 20M) Safety-Critical Decision Making - Avoiding dangerous under-triage that could delay life-saving treatment Clinical Reasoning - Explaining triage decisions with medically sound rationale The benchmark includes 15 gold-standard scenarios spanning all 5 MTS categories, evaluated by GPT-5.2 judges for both accuracy and safety complia

the Green Agent evaluates specific clinical workflows by verifying the digital footprint left by the subject (Purple Agent) within the virtual EHR environment. Rather than relying on subjective linguistic analysis, the Green Agent operates on the foundational truth of state changes and data accuracy. It assesses whether the correct medical facts were identified and if the database state was altered correctly (e.g., an order row added to the correct table). The specific tasks evaluated fall into these core categories: Information Retrieval: Validating that the agent can accurately query and extract specific patient data points (e.g., "What was the last recorded creatinine level?") from the FHIR server. Clinical Ordering & Action: Verifying that the agent correctly executes actions such as placing medication orders, scheduling lab tests, or generating referrals, ensuring the resulting database objects match the ground truth requirements. Medical Documentation: Assessing the agent's ability to synthesize patient information into structured clinical notes or summaries that contain all necessary medical facts. Patient Communication: Evaluating the accuracy and appropriateness of drafted responses to patient inquiries. Clinical Reasoning & Analysis: Checking the agent's ability to perform calculations (e.g., risk scores) or aggregate complex data to form a correct clinical conclusion (e.g., identifying contraindications).