Science loves replication: We conclude an effect is real if we believe replications would also show the effect. It is therefore crucial to understand replication. However, there is strong evidence of severe, widespread misconception about p values and confidence intervals, two of the main statistical tools that guide us in deciding whether an observed effect is real. I propose we teach about replication directly. I describe three approaches: Via confidence intervals (What is the chance the original confidence interval will capture the mean of a repeat of the experiment?); Via p values (Given an initial p value, what is the distribution of p values for replications of the experiment?): and via Peter Killeen's 'prep', which is the average probability that a replication will give a result in the same direction. In each case I will demonstrate an interactive graphical simulation designed to make the tricky ideas of replication vividly accessible.
The CAUSE Research Group is supported in part by a member initiative grant from the American Statistical Association’s Section on Statistics and Data Science Education