P1-19: Introductory Statistics Students’ Conceptual Understanding of Study Design and Conclusions

Information

One of the goals recommended by the 2016 GAISE guidelines is that “students should recognize and be able to explain the central role of randomness in designing studies and drawing conclusions” (GAISE College Report ASA Revision Committee, 2016). This includes being able to identify when it is appropriate for results of studies to be generalized to the population from which the sample was taken, and when it is appropriate to make cause-and-effect conclusions. However, it can be challenging for students to distinguish between the purposes of random sampling and random assignment in statistical studies, sometimes resulting in confusion between the two (e.g. Derry et al., 2000).

In this project, a two-and-a-half week unit on study design and conclusions was designed and implemented in four sections of a 3-credit undergraduate introductory statistics course during spring semester 2016. The primary audience of this course was liberal arts students fulfilling a mathematical thinking requirement at the University of Minnesota. Students were not required to have had any prior experience with statistics before taking this course, but were required to have completed a high school algebra course. Students were also required to have some basic computer skills (e.g. web browsing, Microsoft Word, opening/saving files), as the course used technology on a regular basis. The software used by students in the course was TinkerPlots^TM (http://www.tinkerplots.com). This software was also used in all activities designed for this unit.

The study design and conclusions unit was implemented in four sections of this course – three sections taught in person (two 75-minute periods per week) and one taught entirely online. Each section had between 29 and 42 students enrolled (class sizes were capped at 45), and each section was taught by a graduate student in the statistics education PhD program in the Department of Educational Psychology. Each section also had one teaching assistant who was another graduate student in the Department of Educational Psychology. The teaching assistant helped with grading and, for the in-class sections, attended class approximately once per week. The course was designed so that students would discover statistical concepts by working collaboratively on activities, with very little lecture. In class, most of the period was spent with students working on activities in randomly assigned groups of size 2-3 while the instructor (and teaching assistant, if present) walked around helping groups. Online, students had weekly activities which they completed on a GoogleDoc in randomly assigned groups of 5-6, with regular intervention from the online instructor and/or teaching assistant. Assessments used in this class included regular group quizzes and individual homework assignments.

The goal of the study design unit was to aid the development of students’ conceptual understanding of the distinct roles that random sampling and random assignment play in gathering data, and in the ability to make certain conclusions. Four activities were either developed or modified from the Change Agents in Teaching and Learning Statistics curriculum (CATALST; Zieffler & Catalysts for Change, 2015). Two readings were developed for students to complete prior to certain activities. A forced choice assessment called Inferences from Design Assessment (IDEA) was developed, consisting mainly of modified items from other statistics education assessments and resources such as the Comprehensive Assessment of Outcomes in a First Statistics Course (CAOS; delMas, Garfield, Ooms & Chance, 2007), and the Assessment Resource Tools for Improving Statistical Thinking (ARTIST; https://apps3.cehd.umn.edu/artist/index.html). IDEA was taken by the students as a pretest and posttest. Students also completed two free response assignments throughout the unit, one group quiz and one individual homework assignment, which were also developed as a part of this study. Results from the implementation of this unit will be presented on this poster, including results from assessments that show some areas of improved understanding and some misconceptions that persist.

An outline of the five-day in-class unit appears below. For the online class, the curriculum spanned three weeks (although the first week included a homework assignment from the previous unit, so only half of the first week’s material included study design). During the first week, the “Sampling Countries” activity was completed. During the second week, the “Strength Shoe” and “Murderous Nurse” activities were completed. During the third week, students completed the “Survey Incentives” activity and then the group quiz. At the conclusion of the unit, students turned in a homework assignment.

¹Inspired by the “Sampling” (“Gettysburg Address”) activity from the CATALST curriculum (Zieffler et al., 2015, p. 162-176)

²Modified from the “Strength Shoe” activity from the CATALST curriculum (Zieffler et al., 2015, p. 147-157)

³Modified from the “Murderous Nurse” activity from the CATALST curriculum (Zieffler et al., 2015, p. 187-192)

Although two and a half weeks were available for this unit, not all statistics courses can devote that much time to learning about study design and conclusions. The “Survey Incentives” activity is about random sampling, random assignment, and distinguishing between the two. Although this final activity was designed to put together the material previously learned in the unit, this activity could easily be modified to stand alone as a one-day or two-day introduction to study design and conclusions. Also, although the activities rely on TinkerPlots^TM, they could also be modified to be used with other dynamic technology tools or web applets.

For copies of the activities, lesson plans, readings, and assessments used, contact Elizabeth Fry at fryxx069@umn.edu.