EVALUATING THE ROBUSTNESS OF FACT-CHECKING LLMS ON MULTILINGUAL EVIDENCE

The prevalence of online misinformation has increased for various reasons, including low publishing barriers, limited media literacy, and the tendency of social media platforms to amplify emotionally charged content. As a result, the need for automated fact-checking systems that can verify claims using available evidence has increased greatly. However, real-world evidence articles are rarely clean and complete: they may be incomplete, misleading, or written in a low-resource language. This work evaluates the robustness of two LLMs on fact-checking tasks under realistic conditions, for artificially degraded evidence and multilingual inputs. We construct a dataset of 190 claims per language across four languages with different resource levels (English, German, Greek, Romanian) using the EuroVerdict dataset. Additionally, Romanian claims are machine-translated into English and treated as a fifth language condition. For each claim, evidence articles are retrieved and ranked. The most relevant articles are used to form gold evidence sets. We propose different corruption methods to obtain noisy documents, including the removal, replacement, and edit of evidence sentences. The two LLMs classify each claim into label categories based on the claim's truth value. To evaluate model reasoning quality, an LLM judge scores the model responses using a bias evaluation rubric. Results show that LLMs are highly sensitive to both evidence quality and language. For Llama 8B Instruct, model accuracy dropped from 80\% on English (gold evidence) to 17.89\% on Romanian (gold evidence). On English alone, Llama's accuracy dropped from 80\% on gold evidence to 16.84\% on misleading edit 70\% corruption. Further, translation can serve as a strategy to somewhat improve robustness in low-resource settings, shown through Qwen's improvement from 34.21\% on Romanian (gold evidence) to 42.63\% on Romanian machine-translated to English (gold evidence). However, translation alone may not be able to completely close the gap between model performance on low versus high resource languages.