By Stacey Hancock (Montana State University); Wendy Rummerfield (University of California, Irvine)
Information
Despite a vast body of research on how students learn and understand sampling distributions, students in introductory statistics courses at post-secondary institutions continue to have misconceptions on sampling distributions, a fundamental topic for understanding inference. Two commonly suggested instructional methods for teaching sampling distributions are via computer or tactile simulation activities, but there is limited empirical research on these methods.
We designed a study to conduct in two sections of a large introductory statistics course at a large research university in fall 2015 quarter. Each section, with approximately 220 students enrolled, met three times a week for 50 minutes of lecture. The lecture sections broke into discussion sections of 55 students and met once per week for 50 minutes of activities led by graduate student teaching assistants. The course followed a fairly traditional curriculum (textbook Mind on Statistics, 5th ed., Utts and Heckard, Wiley 2015): descriptive statistics (including correlation and least-squares regression lines), sampling and study design, probability, random variables (normal and binomial), one- and two-sample z-tests for proportions, one- and two-sample t-tests for means, chi-squared tests for 2x2 tables. Students were introduced to inference early in the 10-week quarter through confidence intervals for a single proportion and chi-squared tests for 2x2 tables.
In our experiment, different discussion sections were randomly assigned to a sequence of three in-class activities (weeks 4, 7, and 8 of a 10-week quarter) on sampling distributions using either computer simulations alone, or computer simulations preceded by tactile simulations. We used the free Rossman-Chance applets at rossmanchance.com for all activities. Each type of activity sequence (computer simulation alone or computer simulation preceded by tactile simulation) included the same time-on-task and students received instruction on how to use the online applets through in-class worksheets and instructor guidance. Though sampling distributions were not explicitly covered in the textbook/lecture until week 8, students were introduced to the concept of variability of a statistic in a discussion activity as early as week 4 through a randomization test for a difference in proportions.
Our analysis uses exam scores as our assessment instrument, though other assessments, both qualitative and quantitative, were also measured and will be used in future work (e.g., in-class clicker responses, specific exam items targeted to sampling distributions, written student reflections after each discussion activity). Three exams were given throughout the quarter: Midterm 1 in week 4 (prior to exposure to sampling distributions), Midterm 2 in week 7, and the final exam. We used a linear mixed effects model to determine if there was an effect of the treatment (type of activity sequence) on the average exam performance throughout the quarter. Linear mixed effects models are similar to regression models, but allow for modeling response variables that are correlated. This was applicable in our analysis since the three exam score measurements are correlated within students. The resulting model led us to believe that students who took part in a hands-on activity before moving to online simulation tools had better retention of the concept of a sampling distribution over the quarter compared to those who only did computer simulations.