ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-art Discovery

Scientific artifacts such as models and datasets are foundations for research. With the rapid growth of platforms like HuggingFace, researchers now have access to a large number of artifacts. Yet, a key challenge remains: how can we automatically discover the state-of-the-art (SOTA) model for a given dataset by fully leveraging existing artifacts? We formalize this task as automatic SOTA discovery by modeling HuggingFace as an artifact graph, where nodes are models/datasets and edges represent evaluations. We propose ArtifactLinker, a two-stage framework: (1) ranking promising unobserved model--dataset links using Graph Neural Networks (GNNs) and graph-augmented Large Language Models (LLMs), and (2) verifying top-ranked links via fully automatic and reproducible coding experiments with LLM-based agents. We further introduce a benchmark named ArtifactBench with 14,050 artifacts and 51,335 relations to evaluate the performance of both stages. Results show that (1) graph structures between existing artifacts are effective for missing link prediction; (2) end-to-end ranking and verification with ArtifactLinker help discover surprising SOTA results and research insights.