Web Agent

I present WebJudge Green Agent, a vision-based evaluator for generalist web navigation agents based on the Online-Mind2Web benchmark. Unlike traditional DOM-based evaluators that break with UI updates, our system utilizes a neuro-symbolic 3-step pipeline (Key Point Extraction, Visual Filtering, Verdict Generation) powered by GPT-4o Vision to evaluate agent trajectories on live websites. The project features: A fully Dockerized environment compliant with the AgentBeats A2A protocol. A dynamic task generation system with a diverse dataset (Shopping, Travel, Finance). An intelligent judging engine capable of analyzing screenshots to verify task completion strictly and fairly.

Evaluates Q&A agents on their ability to answer questions about video content. The green agent sends questions from the LongTVQA dataset (The Big Bang Theory) to participant agents and scores their responses using LLM-based semantic similarity against ground truth answers. Scores range from 0.0 (completely incorrect) to 1.0 (semantically equivalent). Supports multiple judge models including Gemini, Claude etc