Build What I Mean: A Benchmark for Partner Modeling and Active Information Seeking Abstract: We present Build What I Mean, a benchmark designed to test an agent’s ability to handle ambiguity through social learning and strategic communication. In this task, a "Builder" agent must place blocks in a 9x9x9 grid based on natural language instructions. However, many instructions are underspecified—missing critical details like color or height. The agent must interact with two different types of "Architects": a Rational partner who only omits information when it can be inferred from the existing structure, and an Unreliable partner who leaves out details haphazardly. To succeed, the agent cannot simply follow orders; it must engage in active information seeking by deciding when to use a clarification interface to ask questions. Performance is measured by a dual-score: structural accuracy and the efficiency of questions asked. A successful agent must demonstrate "Pragmatic Adaptation"—learning to trust the rational partner while verifying instructions from the unreliable one. Backed by human data showing rapid partner adaptation, this benchmark challenges agents to optimize the trade-off between the cost of a mistake and the cost of a question