Significance Testing Principles

This activity illustrates the convergence of long run relative frequency to the true probability. The psychic ability of a student from the class is studied using an applet. The student is asked to repeatedly guess the outcome of a virtual coin toss. The instructor enters the student's guesses and the applet plots the percentage of correct answers versus the number of attempts. With the applet, many guesses can be entered very quickly. If the student is truly a psychic, the percentage correct will converge to a value above 0.5.

By means of a simple story and a worksheet with questions we guide the students from research question to arriving at a conclusion. The whole process is simply reasoning, no formulas. We use the reasoning already done by the student to introduce the standard vocabulary of testing statistical hypotheses (null & alternative hypotheses, p-value, type I and type II error, significance level). Students need to be familiar with binomial distribution tables. After the ducks story is finished, the class is asked to come up with their own research question, collect the data, do the hypotheses testing and answer their own research question. The teaching material is intended to be flexible depending of the time available. Instructors can choose to do just the interactive lecture type, interactive lecture + activity, or even add the optional material.

In this hands-on activity, students count the number of chips in cookies in order to carry out an independent samples t-test to see if Chips AhoyŒ¬ cookies have a higher, lower, or different mean number of chips per cookie than a supermarket brand. First there is a class discussion that can include concepts about random samples, independence of samples, recently covered tests, comparing two parameters with null and alternative hypotheses, what it means to be a chip in a cookie, how to break up the cookies to count chips, and of course a class consensus on the hypotheses to be tested. Second the students count the number of chips in a one cookie from each brand, and report their observations to the instructor. Third, the instructor develops the independent sample t-test statistic. Fourth, the students carry out (individually or as a class) the hypothesis test, checking the assumptions on sample-size/population-shape.