Evidence Engine: An open-source, human-in-the-loop platform for LLM-assisted evidence synthesis and reviews

Literature synthesis for systematic review (SR) is labor-intensive, and existing screening tools are often proprietary, cloud-dependent, and lack transparency. We developed Evidence Engine, an open-source, locally deployable, end-to-end platform for LLM-assisted evidence synthesis, and benchmarked its abstract-screening configurations against gold-standard data. The platform is a three-layer web application (React/TypeScript frontend, FastAPI backend, provider-pluggable LLM orchestrator) covering the full SR workflow. We benchmarked six prompting architectures and nine model tiers, from a domain-fine-tuned 7B local model (LEADS-Mistral-7B) to two frontier API models (DeepSeek-V3.2 and Claude Opus 4.8), at temperature 0 against four gold-standard SR datasets (three from SYNERGY; 1,139 abstracts; 123 positives), reporting recall and specificity at a calibrated operating point. The default LEADS-Mistral-7B reached recall ≥0.80 across three held-out review domains without retraining, and the two frontier models matched it (mean recall 0.909 each) at substantially higher per-abstract cost. Few-shot, multi-persona, cascade, and majority-voting variants did not improve over the native LEADS prompt. On the independent CSMeD benchmark the default reached recall 0.906, and the deterministic table parser reached 0.96 TEDS on PubTabNet. Across all 26 SYNERGY reviews, the default sustained mean recall 0.986 and mean AUROC 0.849 without retraining. Matching each model to its instruction-tuned format was a stronger determinant of accuracy than architectural elaboration, suggesting that model-prompt-task fit may be more important than architectural complexity for this screening task. A task-specific, domain-fine-tuned 7B local model thus achieved high-recall screening across multiple review domains, comparable to the frontier models evaluated, at near zero per-paper marginal cost, offering a transparent, reproducible, and locally deployable alternative to proprietary SR screening tools.