Using assessment to study the development of students' reasoning about sampling distributions

In order to investigate the impact of simulation software on students' understanding of sampling distributions, the Sampling Sim program (delMas, 2001) was developed. The use of this software with students has been the subject of several classroom research studies conducted in a variety of settings (see Chance, delMas, and Garfield, in press). This paper examines the effect of several versions of a structured activity on students' understanding of sampling distributions. The first version of the activity was created to guide the students' interaction with the simulation software based on ideas from previous studies as well as the research literature. Two subsequent versions introduced a sticker and scrapbook activity that allowed students to keep a visual record of the effects of change in population shape and sample size on the resulting distributions of sample means. Four questions guided the studies reported in this paper: how can the simulations be utilized most effectively, how can we best integrate the technology into instruction, which particular techniques appear to be most effective, and how is student reasoning of sampling distributions impacted by use of the program and activities. A variety of assessment tasks were used to determine the extent of students' conceptual understanding of sampling distributions. As classroom researchers, a main goal was to document student learning while providing feedback for further development and improvement of the software and the learning activity. Ongoing collection and analysis of assessment data indicated that despite students' engagement in the activity and apparent understanding of sampling distributions and the Central Limit Theorem,<br>they were unable to apply this knowledge to solve novel problems. In particular, they had<br>difficulty solving graphical items that resembled tasks in the activity as well as well as applying the Central Limit Theorem to different situations. We comment on the lessons we have learned from this research, our explanations for why so many students continue to have difficulties, and our plans for revising the activity.