Other Agent

Loads the Design2Code dataset from Hugging Face (SALT-NLP/Design2Code-hf) Sends screenshot tasks to the purple agent Parses the generated HTML from the agent's response Evaluates the HTML using visual similarity metrics: CLIP similarity between generated and reference screenshots Block-level matching (position, color, text similarity) Overall visual quality assessment Produces evaluation metrics and artifacts

As AI agents are increasingly deployed in real-world applications, evaluating their reliability in domain-specific technical tasks becomes critical for safe deployment. We present \emph{WirelessBench}, a benchmark of $2,592$ curated problems across three Wireless Communication tasks: HomeWork (WCHW), Network Slicing (WCNS), and Mobile Service Assurance (WCMSA). Our benchmark features a tolerance-aware scoring mechanism that accommodates the precision requirements of engineering calculations, distinguishing acceptable numerical errors from catastrophic unit or formula mistakes. Experiments reveal significant reliability gaps: frontier Large Language Models (LLMs) achieve only $58$-$62\%$ accuracy on computational tasks with direct prompting, while our proposed WirelessAgent achieves $80.65\%$ through workflow optimization. WirelessAgent can avoid prevalent failure modes, including unit conversion errors, order-of-magnitude mistakes, and formula misapplication, by calling domain-specific tools.

The Contempletiva agent gets it's name from the book by Hannah Arendt, The Human Condition. Like the vita contempletiva in Arendt's work, this agent examines what it means for an agent to be "thinking." I would argue the agents I built are acting, not thinking (vita activa). In my experience, the challenge is when neither the human nor the agent is contemplating the work.  Here are a few of my experiences with that during this project. To get started, I cloned an instance of τ²-Bench (thanks to generous resources provided by Lambda). I spent several days (and hundreds of dollars of Nvidia GPU compute!) testing the agent capabilities of τ²-Bench in each of its existing domains. Yes, I forgot to turn off my instance. You can read more about my journey with that below: https://www.linkedin.com/posts/ctempleton_youre-not-getting-catfished-my-latest-activity-7406375384281362432-EsrC In another moment of human thoughtlessness, I asked one of the coding agents I was using (Google's Antigravity and Jules) to clean up files I was no longer using but forget to tell it to stop. When I checked back in later than week, all my files were gone. Agents are like that friend at the party who stays up to clean when everyone goes to sleep and then the house is empty when you wake up and you can’t find your keys. That's not to say I didn't have my moments of thoughtful reflection. After all, the orginal idea was to explore The Human Condition of contemplation! As I was preparing to submit a new τ²-Bench domain (product / project management), I reviewed the issue backlog to see if my idea was already in progress. I went through each of the ~50 issues including the 15 that had been resolved - I couldn't tell. I thought of Ethan Mollick's recent advice to pay less attention to the benchmarks and more attention to the bottlenecks. The issues were largely community-contributed and the format varied significantly (despite Sierra's contributing guidelines being very clear). The bottleneck appears to be dispositioning these issues and identifying solutions. Rather than adding to the backlog, I decided to propose a solution to groom it. i.e. Make it easier for community members to propose solutions. PMs who cannot do, delegate. Of ~50 issues submitted for the τ²-Bench repo over the last six months (mid June through mid December 2025), less than 15% follow the structured issue template proposed by the repo owners. This makes it difficult for users (and repo owners) to identify duplicate issues and propose solutions. As this repo experiences increased usage in the future (it's common for open source frameworks like this to grow to thousands of followers rapidly), a more scalable approach to issue management would be helpful. To walk the talk (and not just submit an issue), I contributed to this repo's .github folder an issue template to be used for each new issue submitted. The template also includes an internal section to be used by repo owners to label and assign issues. The impact I anticipate this process will have is fewer duplicate issues, a greater number of community-contributed solutions and a significant draw-down of the existing backlog (35 issues). I hope that this creates a Jevons Paradox of sorts for Sierra - a flywheel of increased community contributions resulting in increased usage of τ²-Bench resulting in increased community contributions and so on. Check out the video below for a sneak peek of what I will be writing about in my upcoming newsletter. As part of this, I propose a remix on Arendt's idea: I call it The Agent Condition.