P2-26: Characterizing Common Errors in Simulation-Based Inference

Information

Before modern computing power allowed for rapid simulations, introductory statistics courses necessarily relied on theoretical methods like z tests and t tests to introduce the logic of inference. Today, at the recommendation of many statistics educators (notably, George Cobb at USCOTS in 2005), simulation-based inference methods are increasingly common as a complement or substitute for traditional inference in introductory statistics courses (ASA, 2016; Rossman & Chance, 2014). Adoption of simulation-based inference methods is inspired by exciting proposed advantages and “a generation of adventurous authors” (Cobb, 2007, p. 13) who have published curricula and resources to support instruction. However, simulation-based inference is not a panacea. This poster identifies errors that commonly arise among students who use simulation-based inference methods and characterizes the statistical conceptions underlying those errors.

Specifically, this poster will illustrate a new framework for conceptualizing the logic of inference. Using the framework, errors in simulation-based inference can be described largely in terms of two challenges. First, students struggle to coordinate the multi-level scheme, which includes the population or true underlying relationship, the distribution of a single sample, and the distribution of statistics collected from multiple samples. Second, students struggle to coordinate two perspectives: the real-world where the sample data was collected and the hypothetical perspective where the null hypothesis is assumed to be true. Samples of student work, collected as part of a larger qualitative study, will be presented to support these conclusions. A brief description of the study’s methods will also be included.