Risk vs. Reward: Supporting Practitioners’ Understanding of LLM Privacy Risks

A growing body of research has highlighted significant privacy risks associated with Large Language Models (LLMs). However, little is known about how LLM practitioners - those developing and deploying LLM-based products - perceive and manage these risks. Further, support for such practitioners in choosing model providers with privacy practices that align with their needs is limited. We conducted semi-structured interviews with 11 practitioners who had experience deploying LLM-based products. Our study explored their perceptions of privacy risks, informing the iterative development and evaluation of the LLM Selection Guide, a prototype tool that helps practitioners translate privacy requirements into model recommendations. Participants found the tool to be a helpful starting point for evaluating the privacy practices of LLM providers, and suggested additional features that could help better integrate the tool into a larger workflow for addressing privacy risks in LLM-based products.