Journal article

Predicting reliability through structured expert elicitation with repliCATS (Collaborative Assessments for Trustworthy Science)

Hannah Fraser, Martin Bush, Bonnie Wintle, Fallon Mody, Eden T Smith, Anca Hanea, Elliot Gould, Victoria Hemming, Daniel George Hamilton, Libby Rumpff, David Peter Wilkinson, Ross Pearson, Felix Singleton Thorn, raquel Ashton, Aaron Willcox, Charles T Gray, Andrew Head, Melissa Ross, Rebecca Groenewegen, Alexandru Marcoci Show all

Center for Open Science

Abstract

Replication is a hallmark of scientific research. As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce a new technique to evaluating replicability, the repliCATS (Collaborative Assessments for Trustworthy Science) process, a structured expert elicitation approach based on the IDEA protocol. The repliCATS process is delivered through an underpinning online platform and applied to the evaluation of research claims in social and behavioural sciences. This process can be deployed for both rapid assessment of small numbers of claims, and assessment of high volumes of claims over an extended period. Pilot data suggests..

View full abstract

Citation metrics