Resource Library

A song to introduce the basic idea of using simulation to calculate a P-value for a randomization test (by simulating lots of group assignments and seeing what proportion of them give more extreme test statistics than observed with the actual group assignments).  The lyrics were written in November 2018 by Larry Lesser from The University of Texas at El Paso and Dennis Pearl from Penn State University. May be sung to the tune of the 1980 number #1 song “Celebration” by Kool and the Gang. Audio of the parody was produced and sung by students in the commercial music program of The University of Teas at El Paso.

Funded by the National Science Foundation, workshops were held over a three-year period, each with about twenty participants nearly equally divided between mathematics educators and statisticians. In these exchanges the mathematics educators presented honest assessments of the status of mathematics education research (both its strengths and its weaknesses), and the statisticians provided insights into modern statistical methods that could be more widely used in such research. The discussions led to an outline of guidelines for evaluating and reporting mathematics education research, which were molded into the current report. The purpose of the reporting guidelines is to foster the development of a stronger foundation of research in mathematics education, one that will be scientific, cumulative, interconnected, and intertwined with teaching practice. The guidelines are built around a model involving five key components of a high-quality research program: generating ideas, framing those ideas in a research setting, examining the research questions in small studies, generalizing the results in larger and more refined studies, and extending the results over time and location. Any single research project may have only one or two of these components, but such projects should link to others so that a viable research program that will be interconnected and cumulative can be identified and used to effect improvements in both teaching practice and future research. The guidelines provide details that are essential for these linkages to occur. Three appendices provide background material dealing with (a) a model for research in mathematics education in light of a medical model for clinical trials; (b) technical issues of measurement, unit of randomization, experiments vs. observations, and gain scores as they relate to scientifically based research; and (c) critical areas for cooperation between statistics and mathematics education research, including qualitative vs. quantitative research, educating graduate students and keeping mathematics education faculty current in education research, statistics practices and methodologies, and building partnerships and collaboratives.

A song about the different assumptions needed for parametric statistical methods and the importance of checking how well they hold and what effect they may have on the results and conclusions. The lyrics were written in 2017 by Dennis K. Pearl from Penn State University and may be sung to the tune of "Every Breath You Take" written by Sting and made popular by The Police on their 1983 album "Synchronicity." Audio of the parody was produced and sung by students in the commercial music program of The University of Teas at El Paso.

This limerick was written by Dr. Nyaradzo Mvududu of the Seattle Pacific University School of Education. The poem was given an honorable mention in the 2007 A-Mu-sing competition.

A cartoon suitable for use in discussing the interpretation of p-values of different levels. The cartoon is number 1478 (January, 2015) from the webcomic series at xkcd.com created by Randall Munroe. Free to use in the classroom and on course web sites under a creative commons attribution-non-commercial 2.5 license.

A cartoon suitable for use in teaching ideas about independence and conditional probability. The cartoon is number 795 (September, 2010) from the webcomic series at xkcd.com created by Randall Munroe. Free to use in the classroom and on course web sites under a creative commons attribution-non-commercial 2.5 license.

A cartoon suitable for a course website that makes use of a boxplot to display an outlier and also uses the term "statistically significant" in its punch line. The cartoon is number 539 (February, 2009) from the webcomic series at xkcd.com created by Randall Munroe. Free to use in the classroom and on course web sites under a creative commons attribution-non-commercial 2.5 license.

A cartoon suitable for use in teaching about the idea of a falsifiable hypothesis. The cartoon is number 2078 (November, 2018) from the webcomic series at xkcd.com created by Randall Munroe. Free to use in the classroom and on course web sites under a creative commons attribution-non-commercial 2.5 license.

A cartoon suitable for use in teaching about publication bias and the small sample caution in hypothesis testing. The cartoon is number 2020 (July, 2018) from the webcomic series at xkcd.com created by Randall Munroe. Free to use in the classroom and on course web sites under a creative commons attribution-non-commercial 2.5 license.

Summary: This High School AP activity examines whether students can tell the difference between Coke^TM and Pepsi^TM by taste? During the “tasting part”, data are collected and the class keeps track of how many students can differentiate between Coke and Pepsi. During the “simulation part” of the activity, a simulation is conducted with dice. Finally, students compare their classroom results in the taste test with the simulated results about what would happen when subjects just guess randomly from the three possible choices. The activity is described in F. Bullard, “AP Statistics: Coke Versus Pepsi: An Introductory Activity for Test of Significance: AP Central – The College Board,” 2017 on the AP Central website at https://apcentral.collegeboard.org/courses/ap-statistics/classroom-resources/coke-versus-pepsi-introductory-test-significance

Specifics: The activity is performed in the following steps:

1. The Tasting part:
   1. First, two students will label three cup positions “A,” “B,” and “C.” Then they will roll a die and pour drinks into the cups such that all combinations of two of one drink and one of the other are represented, and the die roll makes each combination equally likely and keep track of the treatment.
   2. Students will be called out into the hall one by one to taste the three drinks and decide which cup contains the different drink. They do not need to identify the drinks as Coke or Pepsi, they only have to identify the cup containing the different soda, either A, B, or C.
2. The Simulation part:
   1. The next stage of the discussion is to ask the students how many correct identifications they need before they can conclude that people were not just randomly guessing: “11 out of 30 is more than a third, but not enough more to be convincing, right?” Students will probably volunteer different dividing lines, but they will not be good at defending them. At the point when all the students understand the question but are unsure of how to answer it, the dice should be introduced into the activity.
   2. The students can suggest a simulation in which two die outcomes (say, 1 and 2) are considered a correct cup identification, and the other four die outcomes (say, 3, 4, 5, and 6) are considered incorrect cup identifications. Demonstrate by rolling a set of dice or one die many times. You should have as many die rolls as there are subjects in the study. Count the 1s and 2s. Suppose there are 8 out of 30 that “guessed correctly.” On your number line at the blackboard, make an X over the number 8. The students or group of students should do five or 10 simulations each (it’s good to have about 100—200 simulations) and then come to the blackboard and stack their Xs over the appropriate integers, making a histogram of the distribution of “number of correct cup identifications if everyone is randomly guessing.”
3. Conclusion:
   1. Upon the conclusion of the tasting, the number of correct identifications is then counted. At this point, if the number is unusually high (say, 18 out of 30), then most students are prepared to conclude (correctly) that there is evidence that at least some people can tell the difference between Coke and Pepsi.
   2. Some statement like this would be great: “If everyone were randomly guessing, we would almost never see 18 students get it right by luck, because we did that 100 times with dice, and the highest we ever got was 16, and that was only once.”
   3. In the author’s experience, usually, about half or a little more will identify the correct drink. When the author, did this activity with a class: 13 out of 21 students correctly identified the different drinks.

(Resource photo illustration by Barbara Cohen, 2020; this summary compiled by Bibek Aryal)