ECHO: Evidence-Calibrated Human Outcomes from Synthetic Users for Interaction Design

This position paper proposes ECHO, a protocol for treating simulated users built with AI as calibrated measurement instruments for scoped interaction-design claims, not as synthetic replacements for human participants. A synthetic user is an echo, not the person who produced the original signal; it gains evidentiary force only when it supports evidence-calibrated human outcomes: claims about human-facing outcomes bounded by human data. The issue matters now because LLMs and GUI agents generate logs, rationales, personas, and usability reports at product speed, while NLP and HCI lack shared norms for when such traces count as evidence. We contribute an evidentiary standard specifying when synthetic traces may support discovery, rehearsal, pilot triage, or bounded decision support. The standard scopes a design claim, records governed human seed traces, documents simulator policy, estimates design effects with human-corrected evidence, and reports uncertainty and rejection flags. It synthesizes usability inspection, personas, user simulation, agent benchmarks, social simulation, validity, correction, documentation, privacy, and human-AI interaction. The central position is narrow: synthetic users can help interaction design when they improve issue discovery, pilot triage, or bounded decision support under calibration. Without calibration, fluent traces turn model plausibility into counterfeit user evidence.