Re: [SBI] A sample item

Thought I'd just report about an assignment I gave a couple weeks ago, prompted by a Morning Edition segment on NPR: Does the p-value affect the value of the research? Listen to this NPR segment: http://www.npr.org/2015/01/28/382056490/studies-critical-of-bilingual-benef… (Links to an external site.)<http://www.npr.org/2015/01/28/382056490/studies-critical-of-bilin… and write a few paragraphs (no more than one page) response/commentary. As part of your response, either give some examples of kinds of studies where a high p-value (so "negative results") might actually be useful public knowledge or explain why only studies with low p values are worthy of publication. If you wish, you could read the posted comments as a prompt for your thinking. This was an extra credit opportunity, but (as I had hoped) it made the students who chose to do it (about 2/3 of them) think carefully about what p-value means. Jennifer ________________________________ From: sbi-bounces(a)causeweb.org [sbi-bounces(a)causeweb.org] on behalf of Josh Tabor [joshtabor(a)hotmail.com] Sent: Wednesday, February 18, 2015 4:06 PM To: sbi(a)causeweb.org Subject: [SBI] A sample item Hello Everyone— I wanted to share an item I included on my recent AP Statistics test. We had just finished traditional one sample tests for a proportion and one sample tests for a mean. I introduced the chapter with a simulation-based analysis of a test for a proportion, and we had done several similar activities earlier in the year. I like the item because it addresses the logic of inference, but doesn’t match up with one of our traditional tests. Also, because it is about baseball! I pasted it below and in the attachment. I also included a brief answer key and comments on my student performance. Feel free to use it with your students if you are interested. Thanks, Josh Tabor In a recent Sports Illustrated article, author Michael Rosenberg addresses “America’s Wait Problem.” That is, he discusses how fans of some teams have to wait many, many years for their team to win a championship. In Major League Baseball, fans should expect to wait an average of 30 years for a championship—assuming all 30 teams are equally likely to win a championship each season. But is it reasonable to believe that all teams are equally likely to win a championship? In the last 19 seasons, only 10 different teams won a championship. Does this provide convincing evidence that some teams are more likely than others to win a championship? We can find out by testing the following hypotheses: H0: All 30 teams are equally likely to win a championship Ha: Some teams are more likely to win a championship than others. (a) Describe how to use slips of paper to simulate the number of different teams to win a championship in 19 seasons, assuming that each team is equally likely to win the championship each season. Write the name of each team on a slip of paper. Shuffle slips in a hat. Select one team, record, and replace slip. Repeat for a total of 19 champions. Record number of different champions. Repeat many times. Comments on student performance: Many students were able to describe how to prepare the slips. Most of these correctly selected 19 slips with replacement, although some left this out. However, many students did not discuss what they would do after the 19 selections (e.g., record the number of different champs). One hundred trials of a simulation were conducted, assuming that each team is equally likely to win the championship each season. The number of different teams to win a championship in 19 seasons was recorded for each trial on graphed on the dotplot. (b) There is one dot at 10. Explain what this dot represents. [cid:image005.png@01D04B8C.6F08B290] Number of difference champions In one simulated 19-year stretch, there were only 10 different champions. Comments: Most students did well on this part. (c) Based on the simulation and the data from the previous 19 seasons, is there convincing evidence that some teams are more likely to win a championship than others? Explain. Because it is very unlikely to get 10 or fewer different winners by chance alone (p-value [cid:image006.png@01D04B8C.6F08B290] 1/100), we reject H0. There is convincing evidence that some teams are more likely to win a championship than others. Comments: As expected, this was the hardest part for students. About 10-15% had no idea what to do or tried a traditional z or t test. About 50% focused on the wrong part of the sampling distribution (e.g., said the evidence was convincing because there were no dots at 19, said the evidence was convincing because the distribution was centered at 14-15, which was less than 19 or 30). The remaining students focused on the small probability that there would be 10 or fewer different champions by chance alone. Some even called this a p-value! A few argued that 10 different winners did not provide convincing evidence of Ha because 10 could happen by chance, even though it is would be fairly unusual.