Tracing LLM Social Reasoning to Pretraining Data: A Provenance Audit for Social Simulation
Abstract
LLM-based social simulations depend on social-reasoning capabilities whose pretraining evidence is usually opaque. We treat training-data attribution as a provenance audit: for an open model and corpus, which training regions are associated with social reasoning, how do those profiles differ from adjacent capabilities, and does intervention confirm that influential data is load-bearing? We compute gradient-based attribution (TrackStar via Bergson) over a working set from the de-duplicated Dolma3 mix, aggregate influence across WebOrganizer’s 24-format ×24-topic taxonomy (576 bins), and contrast SocialIQA and MMLU Social Sciences against ARC-Challenge and MMLU STEM. In OLMo3-7B, SocialIQA is the profile outlier: its positive attribution is more diffuse and shifted toward interpersonal and narrative regions rather than documentation-heavy ones. Influence-targeted unlearning validates this association most clearly for SocialIQA, while the comparison tasks show weaker, null, or reversed effects. A held-out social-probe suite tests the breadth and limits of the pattern without claiming population-level simulation fidelity.