On Epistemic Diversity in Large Language Models

The increasing adoption of LLMs has been accompanied by growing concerns about their ability to accurately reflect diverse perspectives. Consequently, a plethora of research has focused on measuring and enhancing diversity in LLM outputs. Through a survey of prior work, we show that most existing studies focus on diversity when it is defined in terms of groups of people, such as those based on gender, race, national origin, or religious views. In this work, we identify an often overlooked but essential aspect of diversity, which we term epistemic diversity, that is, the diversity of knowledge that LLMs present to users. The need for epistemic diversity arises in contexts where a single information need can be fulfilled in multiple valid ways, e.g., proving that there are infinitely many primes or writing an article about a famous physicist. We propose a framework for conceptualizing epistemic diversity. This framework sheds light on when and why multiple valid outputs arise, and how epistemic diversity can be meaningfully characterized and measured. We then operationalize this framework to assess epistemic diversity in multiple domains. Our analysis shows that LLM outputs can lack epistemic diversity, potentially potentially restricting the knowledge surfaced to users.