SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery

Scaling qualitative data collection requires automating semi-structured interviews. However, existing LLM-based systems largely focus on covering predefined topics and struggle to follow up on emergent themes that arise organically in participant responses. This limitation stems from the absence of an explicit objective that formalizes the trade-off between covering a predefined topic guide and adaptive exploration. In this work, we frame adaptive semi-structured interviewing as an optimization problem and define interview utility as balancing coverage of a predefined topic guide, discovery of emergent themes, and interview cost. Based on this formulation, we introduce SparkMe, a multi-agent LLM interviewer that performs proactive planning via simulated conversation rollouts to select high-utility questions. SparkMe achieves higher topic guide coverage ($+4.7$% over the best baseline) and elicits richer emergent insights while using fewer conversational turns. We further validate SparkMe in a user study with $70$ participants across $7$ professions, where domain experts rate it as producing high-quality adaptive interviews that surface profession-specific insights not captured by baseline systems. The code and datasets for SparkMe will be released upon acceptance.