Social Translucence Benchmark: Evaluating Visibility, Awareness, and Accountability for Collaborative Sequencing Tasks in Human-Agent Interactions

Existing benchmarks for AI agents primarily evaluate task completion, ignoring how agents collaborate with humans and one another during those tasks. We introduce the Social Translucence Benchmark (STB), a benchmark for evaluating social behaviors in collaborative human–agent interactions. Inspired by the theory of Social Translucence from computer-supported cooperative work, STB operationalizes three behavioral dimensions essential for effective collaboration—visibility, awareness, and accountability—within collaborative sequencing tasks where humans and AI agents jointly construct ordered solutions. The benchmark will include a suite of collaborative sequencing tasks inspired by real-world planning and co-creation activities, such as travel itinerary planning, curriculum design, and collaborative writing. Beyond conventional measures of task success, STB evaluates whether agents expose their reasoning, maintain shared situational awareness, communicate uncertainty, negotiate conflicting preferences, and coordinate responsibilities. By shifting evaluation from task competence to collaborative behavior, STB complements existing agent benchmarks and provides a framework for assessing whether frontier AI agents exhibit effective, trustworthy collaborative behavior.