SBI

sbi@causeweb.org

64 discussions

Data for Fisher's Exact and some ethical considerations

by Olson Hunt, Megan

Good evening, So this isn't about SBI per se, but is with regard to the blog posts about finding real data (which are great, by the way). I happened upon this article in Lancet that I like for a couple of reasons. It's from 1998, but I think it's still worth considering: Wolkenstein et al. (1998). Randomised comparison of thalidomide versus placebo in toxic epidermal necrolysis. The Lancet 352, 1586-1589. It's a small sample size (appropriate for Fisher's Exact Test), and the study had to be stopped because more people were dying from the active treatment than placebo (10/12 vs. 3/10). Thus, it's a nice opportunity to talk about ethics when studying human subjects and provides a real (and I think interesting as well) example of Fisher's Test. Megan Megan J. Olson Hunt, PhD ASSISTANT PROFESSOR, STATISTICS ............................................................................. Department of Natural and Applied Sciences, LS 465 University of Wisconsin-Green Bay 2420 Nicolet Drive Green Bay, WI 54311 Office: LS 427 | blog.uwgb.edu/olsonhunt | olsonhum(a)uwgb.edu

9 years

MAA PREP 3-day workshop on Simulation-Based Inference: July 14-17 in Roanoke

by Allan Rossman

Hello SBI folks, This message is to make you aware of a 3-day workshop on "Teaching the Process of Statistical Investigations with a Randomization-Based Curriculum" that my colleagues and I will present at Hollins University in Roanoke, Virginia on Tues July 14 - Fri July 17, 2015. Many of you on this list have already participated in a similar workshop, but we would appreciate your helping to spread the word to colleagues who might be interested. The registration fee for the workshop is $325 before June 2, which includes meals and lodging. More information is available from a link at: http://www.maa.org/programs/faculty-and-departments/prep-workshops/schedule and a link to register is available at: https://www.signup4.net/Public/ap.aspx?EID=MAAP10E Additional questions can be directed to me (arossman(a)calpoly.edu). Thanks very much, Allan Rossman -- Allan J. Rossman Professor and Chair Statistics Department Cal Poly San Luis Obispo, CA 93407 arossman(a)calpoly.edu http://statweb.calpoly.edu/arossman/

9 years

Re: [SBI] favorite assessment/exam questions

by Nathan Tintle

Thanks everyone for your great questions! I've attached one that I've been using for a while that gets at a situation where simulation is OK, but the normal approximation (theory-based) approach is not. It's a bit too context specific and specific to the applets our group has developed to necessarily use it 'as is' in your course, but adapting it a bit would seem to be fairly straightforward. The "note" was added after the first time I included the question and a number of students explained the difference due to chance variation in the simulation distribution. I like this question because it really gets at whether students can apply the idea of 'validity conditions' in context and the difference in p-values is somewhat measurable so it matters Have a great weekend! Nathan On Wed, Mar 25, 2015 at 2:53 PM, Eric Reyes <reyesem(a)rose-hulman.edu> wrote: > I loved Allan's questions (and in fact even included one on an exam this > week). Given that this reduced my exam writing time, I thought I would > pass along a few of my favorites as well. The full questions are attached. > > The first pair of questions makes use of a side-by-side boxplot. I like > the first pair of questions because it tests the idea that the strength of > evidence is dependent on both the value of the statistic as well as the > variability in the data. In addition, it relies on their ability to read > and interpret graphics. The follow-up question then considers whether > assumptions for a particular analysis are reasonable. > > I believe I stole the second pair of questions from Roger Woodard at > NCSU when I was a graduate student? It has been my go-to for testing the > difference between the distribution of a random sample and a sampling > distribution. Presenting the population in terms of a boxplot also > requires students to interpret graphics by comparing boxplots to > histograms. While many students get one of the two questions right, we > often find students wanting to give the same answer for both questions. > > Eric > > *Eric M. Reyes | **Assistant Professor* > *Department of Mathematics* > *ROSE-HULMAN INSTITUTE OF TECHNOLOGY* > > 5500 Wabash Ave | Terre Haute, IN 47803-3999 > Phone: 812.877.8287 | Fax: 812.877.8883 > www.rose-hulman.edu > > On Sat, Mar 21, 2015 at 2:02 PM, Allan Rossman <arossman(a)calpoly.edu> > wrote: > >> Hello Simulation-Based Inference (SBI) group, >> >> You might recall that I wrote to you on Groundhog Day last month, so I >> thought I would check in again on this, the first full day of spring. Now >> with the benefit of nearly seven weeks of hindsight, what do you think of >> Punxsutawney Phil's prediction? >> >> My colleagues and I thought this might be a good time of year to write >> about favorite assessment/exam questions for introductory statistics. >> We've just finished final exams for the Winter quarter at Cal Poly, and >> those of you on a semester calendar will need to give final exams in >> another 4-6 weeks or so. But the best reason for writing now is that this >> provides me with a good excuse to procrastinate on grading my exams! >> >> I am going to identify and comment on my all-time favorite assessment >> question, but first I'll mention two "honorable mention" questions that I >> also like. I'll count down my items from #3 to #1, along with some >> commentary on each. Think of #3 and #2, which are quite short, as opening >> acts for the main event, which is fairly long. Oh, and in case you don't >> make it to the end of this message, let me now invite all of you to respond >> to the SBI list with one of your own favorite assessment items. Here we go >> ... >> >> 3. I ask students: Suppose that 60% of graduate students at Cal Poly have >> an iPad and that 20% of undergraduate students at Cal Poly have an iPad. >> Does it necessarily follow that 40% of all students at Cal Poly have an >> iPad? Explain your answer. >> >> I like this question because I think it gets at a basic skill of >> quantitative literacy. I certainly don't intend that students cite the Law >> of Total Probability in answering the question, and I would hope that many >> students could answer this well even before they take my class. But if >> they leave my class believing that the answer is yes, then I feel that >> they're leaving without some basic quantitative awareness. Almost all of >> my students realize that Cal Poly has far more undergraduate students than >> graduate students, so the overall percentage with an iPad would be much >> closer to 20% than 60% in my made-up scenario. I don't expect students to >> say this part about the overall percentage being closer to 20% than to 60%, >> but I'm glad when they do. I'm pleased that very few of my students fall >> into the trap of answering yes (but wait for my next question). >> >> If you want to modify this question for your local situation, you could >> replace undergrad/grad student with any binary variable that is likely not >> to be equally represented in your population, perhaps male/female or >> math/other major or dog/cat person. (Most well-educated people are cat >> people, right? Just kidding!) >> >> 2. I ask students: Suppose that you take a random sample of 100 houses >> currently for sale in California. Does the Central Limit Theorem (CLT) >> suggest that a histogram of the house prices in the sample will display an >> approximately normal distribution? Explain. >> >> I suspect that you know how this one turns out. To my dismay most >> students answer yes, pointing out for their explanation that the sample >> size is larger than 30. I even put "house prices" in bold in an attempt to >> draw their attention to the variable that I'm asking about. Perhaps I >> should add a note to point out that there's no mention of words such as >> "mean" or "average" anywhere in the question. >> >> So why do I like this question? I think it's more informative of what >> students have learned (or not) about the CLT than asking them to perform >> calculations and hoping that they'll remember to use sigma/sqrt(n) rather >> than just sigma in the denominator of the z-score. If my students leave >> the course thinking that the CLT says that sample data for all variables >> display a normal distribution when the sample size is large, then they've >> fundamentally not understood what the CLT or a sampling distribution is all >> about. I suspect that many of my students realize, or could reason out for >> themselves, that the distribution of house prices is typically skewed to >> the right. But my question deliberately puts the CLT in their minds, which >> leads their thinking toward a common misconception about that result. >> >> 1. My all-time, no-doubt, bar-none favorite exam question is ... (drum >> roll please) ... the investigative task (question #6) from the 2009 AP >> Statistics exam. I want to say at the outset that I had nothing to do with >> writing this question, so I take no credit whatsoever. Before you read my >> comments about it, please go and read this question (remember to scroll >> down to #6) at: >> >> http://apcentral.collegeboard.com/apc/public/repository/ >> ap09_frq_statistics.pdf >> >> Now let me talk through the four parts of this question. I admit that >> the context here is not very exciting, and I'll also say that part (a) is >> pretty straight-forward and perhaps even boring. Nevertheless, I still >> think it's a good question because students do not find it easy to identify >> the parameter in a given situation. And it's certainly important to >> understand what the parameter is before trying to make inferences about the >> value of that parameter. But part (a) does not make this my all-time >> favorite question. >> >> I think part (b) is a very nice question, because it asks students to >> apply something they know to a situation they've probably never thought >> of. Students should know a lot about means and medians, but I bet it's >> never occurred to them to calculate the ratio of the mean to the median. >> This question tests whether they realize that they can use their knowledge >> that the mean is typically larger than the median with a right-skewed >> distribution to conclude that this new ratio statistic will be larger than >> one with a right-skewed distribution. >> >> I grant that parts (a) and (b) do not yet qualify for my short list of >> all-time favorite questions, but there's more ... >> >> Part (c) epitomizes the logic of simulation-based inference. It assesses >> whether students have a firm enough understanding to be able to apply what >> they know to a situation they've never encountered: testing whether sample >> data provide convincing evidence that a population distribution is skewed >> to the right. Unfortunately, many students focus on the symmetric shape of >> the distribution of simulated ratio statistics, and others focus on this >> null distribution being centered around 1. But students who truly >> understand how simulation-based inference works know to look for where the >> observed sample value of the ratio statistic falls in the distribution of >> simulated ratio statistics. >> >> If this question stopped there, it might rank as my all-time favorite >> question, but I'm not sure about that. And you may recall that I used the >> phrases "no-doubt" and "bar-none" above. So, what gives? Well, part (d) >> clinches the title like Secretariat racing down the stretch at the Belmont >> Stakes. >> >> This part of the question invites students to realize that they possess >> the intellectual power to create their own statistic to measure skewness. >> Half an hour previously, it might not have occurred to students that using >> a statistic to measure skewness was even in the realm of possibility. But >> now students are asked to devise their own statistic to do just that. >> They're given just enough of a hint to give them a foundation on which to >> build, as they are told to use components of the five-number summary. >> Students produced many good statistics for this question on the AP exam. >> You can find some sample student responses and also scoring guidelines at: >> >> http://apcentral.collegeboard.com/apc/members/exam/exam_ >> information/8357.html >> >> You can also find an investigation of the power of some of skewness >> statistics in Josh Tabor's JSE article: >> >> http://www.amstat.org/publications/jse/v18n2/tabor.pdf >> >> Thanks very much for reading all of this! I'm afraid that I must now >> heed the beck and call of my own students' final exams that need grading. >> Please do reply to this SBI list with one (or more) of your own favorite >> assessment/exam questions. Rest assured that there's no need for you to >> write as lengthy a message as I have, unless you also have a desire to >> procrastinate from grading of your own. >> >> -- Allan >> >> -- >> Allan J. Rossman >> Professor and Chair >> Statistics Department >> Cal Poly >> San Luis Obispo, CA 93407 >> arossman(a)calpoly.edu >> http://statweb.calpoly.edu/arossman/ >> >> _______________________________________________ >> SBI mailing list >> SBI(a)causeweb.org >> https://www.causeweb.org/mailman/listinfo/sbi >> > > -- Nathan Tintle, Ph.D. Associate Professor of Statistics and Dept. Chair Director for Research and Scholarship Dordt College Sioux Center, IA 51250 nathan.tintle(a)dordt.edu Phone: (712) 722-6264 Office: SB1612

9 years, 1 month

Re: [SBI] favorite assessment/exam questions

by Eric Reyes

I loved Allan's questions (and in fact even included one on an exam this week). Given that this reduced my exam writing time, I thought I would pass along a few of my favorites as well. The full questions are attached. The first pair of questions makes use of a side-by-side boxplot. I like the first pair of questions because it tests the idea that the strength of evidence is dependent on both the value of the statistic as well as the variability in the data. In addition, it relies on their ability to read and interpret graphics. The follow-up question then considers whether assumptions for a particular analysis are reasonable. I believe I stole the second pair of questions from Roger Woodard at NCSU when I was a graduate student? It has been my go-to for testing the difference between the distribution of a random sample and a sampling distribution. Presenting the population in terms of a boxplot also requires students to interpret graphics by comparing boxplots to histograms. While many students get one of the two questions right, we often find students wanting to give the same answer for both questions. Eric *Eric M. Reyes | **Assistant Professor* *Department of Mathematics* *ROSE-HULMAN INSTITUTE OF TECHNOLOGY* 5500 Wabash Ave | Terre Haute, IN 47803-3999 Phone: 812.877.8287 | Fax: 812.877.8883 www.rose-hulman.edu On Sat, Mar 21, 2015 at 2:02 PM, Allan Rossman <arossman(a)calpoly.edu> wrote: > Hello Simulation-Based Inference (SBI) group, > > You might recall that I wrote to you on Groundhog Day last month, so I > thought I would check in again on this, the first full day of spring. Now > with the benefit of nearly seven weeks of hindsight, what do you think of > Punxsutawney Phil's prediction? > > My colleagues and I thought this might be a good time of year to write > about favorite assessment/exam questions for introductory statistics. > We've just finished final exams for the Winter quarter at Cal Poly, and > those of you on a semester calendar will need to give final exams in > another 4-6 weeks or so. But the best reason for writing now is that this > provides me with a good excuse to procrastinate on grading my exams! > > I am going to identify and comment on my all-time favorite assessment > question, but first I'll mention two "honorable mention" questions that I > also like. I'll count down my items from #3 to #1, along with some > commentary on each. Think of #3 and #2, which are quite short, as opening > acts for the main event, which is fairly long. Oh, and in case you don't > make it to the end of this message, let me now invite all of you to respond > to the SBI list with one of your own favorite assessment items. Here we go > ... > > 3. I ask students: Suppose that 60% of graduate students at Cal Poly have > an iPad and that 20% of undergraduate students at Cal Poly have an iPad. > Does it necessarily follow that 40% of all students at Cal Poly have an > iPad? Explain your answer. > > I like this question because I think it gets at a basic skill of > quantitative literacy. I certainly don't intend that students cite the Law > of Total Probability in answering the question, and I would hope that many > students could answer this well even before they take my class. But if > they leave my class believing that the answer is yes, then I feel that > they're leaving without some basic quantitative awareness. Almost all of > my students realize that Cal Poly has far more undergraduate students than > graduate students, so the overall percentage with an iPad would be much > closer to 20% than 60% in my made-up scenario. I don't expect students to > say this part about the overall percentage being closer to 20% than to 60%, > but I'm glad when they do. I'm pleased that very few of my students fall > into the trap of answering yes (but wait for my next question). > > If you want to modify this question for your local situation, you could > replace undergrad/grad student with any binary variable that is likely not > to be equally represented in your population, perhaps male/female or > math/other major or dog/cat person. (Most well-educated people are cat > people, right? Just kidding!) > > 2. I ask students: Suppose that you take a random sample of 100 houses > currently for sale in California. Does the Central Limit Theorem (CLT) > suggest that a histogram of the house prices in the sample will display an > approximately normal distribution? Explain. > > I suspect that you know how this one turns out. To my dismay most > students answer yes, pointing out for their explanation that the sample > size is larger than 30. I even put "house prices" in bold in an attempt to > draw their attention to the variable that I'm asking about. Perhaps I > should add a note to point out that there's no mention of words such as > "mean" or "average" anywhere in the question. > > So why do I like this question? I think it's more informative of what > students have learned (or not) about the CLT than asking them to perform > calculations and hoping that they'll remember to use sigma/sqrt(n) rather > than just sigma in the denominator of the z-score. If my students leave > the course thinking that the CLT says that sample data for all variables > display a normal distribution when the sample size is large, then they've > fundamentally not understood what the CLT or a sampling distribution is all > about. I suspect that many of my students realize, or could reason out for > themselves, that the distribution of house prices is typically skewed to > the right. But my question deliberately puts the CLT in their minds, which > leads their thinking toward a common misconception about that result. > > 1. My all-time, no-doubt, bar-none favorite exam question is ... (drum > roll please) ... the investigative task (question #6) from the 2009 AP > Statistics exam. I want to say at the outset that I had nothing to do with > writing this question, so I take no credit whatsoever. Before you read my > comments about it, please go and read this question (remember to scroll > down to #6) at: > > http://apcentral.collegeboard.com/apc/public/repository/ > ap09_frq_statistics.pdf > > Now let me talk through the four parts of this question. I admit that the > context here is not very exciting, and I'll also say that part (a) is > pretty straight-forward and perhaps even boring. Nevertheless, I still > think it's a good question because students do not find it easy to identify > the parameter in a given situation. And it's certainly important to > understand what the parameter is before trying to make inferences about the > value of that parameter. But part (a) does not make this my all-time > favorite question. > > I think part (b) is a very nice question, because it asks students to > apply something they know to a situation they've probably never thought > of. Students should know a lot about means and medians, but I bet it's > never occurred to them to calculate the ratio of the mean to the median. > This question tests whether they realize that they can use their knowledge > that the mean is typically larger than the median with a right-skewed > distribution to conclude that this new ratio statistic will be larger than > one with a right-skewed distribution. > > I grant that parts (a) and (b) do not yet qualify for my short list of > all-time favorite questions, but there's more ... > > Part (c) epitomizes the logic of simulation-based inference. It assesses > whether students have a firm enough understanding to be able to apply what > they know to a situation they've never encountered: testing whether sample > data provide convincing evidence that a population distribution is skewed > to the right. Unfortunately, many students focus on the symmetric shape of > the distribution of simulated ratio statistics, and others focus on this > null distribution being centered around 1. But students who truly > understand how simulation-based inference works know to look for where the > observed sample value of the ratio statistic falls in the distribution of > simulated ratio statistics. > > If this question stopped there, it might rank as my all-time favorite > question, but I'm not sure about that. And you may recall that I used the > phrases "no-doubt" and "bar-none" above. So, what gives? Well, part (d) > clinches the title like Secretariat racing down the stretch at the Belmont > Stakes. > > This part of the question invites students to realize that they possess > the intellectual power to create their own statistic to measure skewness. > Half an hour previously, it might not have occurred to students that using > a statistic to measure skewness was even in the realm of possibility. But > now students are asked to devise their own statistic to do just that. > They're given just enough of a hint to give them a foundation on which to > build, as they are told to use components of the five-number summary. > Students produced many good statistics for this question on the AP exam. > You can find some sample student responses and also scoring guidelines at: > > http://apcentral.collegeboard.com/apc/members/exam/exam_ > information/8357.html > > You can also find an investigation of the power of some of skewness > statistics in Josh Tabor's JSE article: > > http://www.amstat.org/publications/jse/v18n2/tabor.pdf > > Thanks very much for reading all of this! I'm afraid that I must now heed > the beck and call of my own students' final exams that need grading. > Please do reply to this SBI list with one (or more) of your own favorite > assessment/exam questions. Rest assured that there's no need for you to > write as lengthy a message as I have, unless you also have a desire to > procrastinate from grading of your own. > > -- Allan > > -- > Allan J. Rossman > Professor and Chair > Statistics Department > Cal Poly > San Luis Obispo, CA 93407 > arossman(a)calpoly.edu > http://statweb.calpoly.edu/arossman/ > > _______________________________________________ > SBI mailing list > SBI(a)causeweb.org > https://www.causeweb.org/mailman/listinfo/sbi >

9 years, 1 month

favorite assessment/exam questions

by Allan Rossman

Hello Simulation-Based Inference (SBI) group, You might recall that I wrote to you on Groundhog Day last month, so I thought I would check in again on this, the first full day of spring. Now with the benefit of nearly seven weeks of hindsight, what do you think of Punxsutawney Phil's prediction? My colleagues and I thought this might be a good time of year to write about favorite assessment/exam questions for introductory statistics. We've just finished final exams for the Winter quarter at Cal Poly, and those of you on a semester calendar will need to give final exams in another 4-6 weeks or so. But the best reason for writing now is that this provides me with a good excuse to procrastinate on grading my exams! I am going to identify and comment on my all-time favorite assessment question, but first I'll mention two "honorable mention" questions that I also like. I'll count down my items from #3 to #1, along with some commentary on each. Think of #3 and #2, which are quite short, as opening acts for the main event, which is fairly long. Oh, and in case you don't make it to the end of this message, let me now invite all of you to respond to the SBI list with one of your own favorite assessment items. Here we go ... 3. I ask students: Suppose that 60% of graduate students at Cal Poly have an iPad and that 20% of undergraduate students at Cal Poly have an iPad. Does it necessarily follow that 40% of all students at Cal Poly have an iPad? Explain your answer. I like this question because I think it gets at a basic skill of quantitative literacy. I certainly don't intend that students cite the Law of Total Probability in answering the question, and I would hope that many students could answer this well even before they take my class. But if they leave my class believing that the answer is yes, then I feel that they're leaving without some basic quantitative awareness. Almost all of my students realize that Cal Poly has far more undergraduate students than graduate students, so the overall percentage with an iPad would be much closer to 20% than 60% in my made-up scenario. I don't expect students to say this part about the overall percentage being closer to 20% than to 60%, but I'm glad when they do. I'm pleased that very few of my students fall into the trap of answering yes (but wait for my next question). If you want to modify this question for your local situation, you could replace undergrad/grad student with any binary variable that is likely not to be equally represented in your population, perhaps male/female or math/other major or dog/cat person. (Most well-educated people are cat people, right? Just kidding!) 2. I ask students: Suppose that you take a random sample of 100 houses currently for sale in California. Does the Central Limit Theorem (CLT) suggest that a histogram of the house prices in the sample will display an approximately normal distribution? Explain. I suspect that you know how this one turns out. To my dismay most students answer yes, pointing out for their explanation that the sample size is larger than 30. I even put "house prices" in bold in an attempt to draw their attention to the variable that I'm asking about. Perhaps I should add a note to point out that there's no mention of words such as "mean" or "average" anywhere in the question. So why do I like this question? I think it's more informative of what students have learned (or not) about the CLT than asking them to perform calculations and hoping that they'll remember to use sigma/sqrt(n) rather than just sigma in the denominator of the z-score. If my students leave the course thinking that the CLT says that sample data for all variables display a normal distribution when the sample size is large, then they've fundamentally not understood what the CLT or a sampling distribution is all about. I suspect that many of my students realize, or could reason out for themselves, that the distribution of house prices is typically skewed to the right. But my question deliberately puts the CLT in their minds, which leads their thinking toward a common misconception about that result. 1. My all-time, no-doubt, bar-none favorite exam question is ... (drum roll please) ... the investigative task (question #6) from the 2009 AP Statistics exam. I want to say at the outset that I had nothing to do with writing this question, so I take no credit whatsoever. Before you read my comments about it, please go and read this question (remember to scroll down to #6) at: http://apcentral.collegeboard.com/apc/public/repository/ap09_frq_statistics… Now let me talk through the four parts of this question. I admit that the context here is not very exciting, and I'll also say that part (a) is pretty straight-forward and perhaps even boring. Nevertheless, I still think it's a good question because students do not find it easy to identify the parameter in a given situation. And it's certainly important to understand what the parameter is before trying to make inferences about the value of that parameter. But part (a) does not make this my all-time favorite question. I think part (b) is a very nice question, because it asks students to apply something they know to a situation they've probably never thought of. Students should know a lot about means and medians, but I bet it's never occurred to them to calculate the ratio of the mean to the median. This question tests whether they realize that they can use their knowledge that the mean is typically larger than the median with a right-skewed distribution to conclude that this new ratio statistic will be larger than one with a right-skewed distribution. I grant that parts (a) and (b) do not yet qualify for my short list of all-time favorite questions, but there's more ... Part (c) epitomizes the logic of simulation-based inference. It assesses whether students have a firm enough understanding to be able to apply what they know to a situation they've never encountered: testing whether sample data provide convincing evidence that a population distribution is skewed to the right. Unfortunately, many students focus on the symmetric shape of the distribution of simulated ratio statistics, and others focus on this null distribution being centered around 1. But students who truly understand how simulation-based inference works know to look for where the observed sample value of the ratio statistic falls in the distribution of simulated ratio statistics. If this question stopped there, it might rank as my all-time favorite question, but I'm not sure about that. And you may recall that I used the phrases "no-doubt" and "bar-none" above. So, what gives? Well, part (d) clinches the title like Secretariat racing down the stretch at the Belmont Stakes. This part of the question invites students to realize that they possess the intellectual power to create their own statistic to measure skewness. Half an hour previously, it might not have occurred to students that using a statistic to measure skewness was even in the realm of possibility. But now students are asked to devise their own statistic to do just that. They're given just enough of a hint to give them a foundation on which to build, as they are told to use components of the five-number summary. Students produced many good statistics for this question on the AP exam. You can find some sample student responses and also scoring guidelines at: http://apcentral.collegeboard.com/apc/members/exam/exam_information/8357.ht… You can also find an investigation of the power of some of skewness statistics in Josh Tabor's JSE article: http://www.amstat.org/publications/jse/v18n2/tabor.pdf Thanks very much for reading all of this! I'm afraid that I must now heed the beck and call of my own students' final exams that need grading. Please do reply to this SBI list with one (or more) of your own favorite assessment/exam questions. Rest assured that there's no need for you to write as lengthy a message as I have, unless you also have a desire to procrastinate from grading of your own. -- Allan -- Allan J. Rossman Professor and Chair Statistics Department Cal Poly San Luis Obispo, CA 93407 arossman(a)calpoly.edu http://statweb.calpoly.edu/arossman/

9 years, 1 month

Re: [SBI] favorite assessment/exam questions

by Tim Erickson

> 1. My all-time, no-doubt, bar-none favorite exam question is ... (drum roll please) ... the investigative task (question #6) from the 2009 AP Statistics exam. I want to say at the outset that I had nothing to do with writing this question, so I take no credit whatsoever. Before you read my comments about it, please go and read this question (remember to scroll down to #6) at: > > http://apcentral.collegeboard.com/apc/public/repository/ap09_frq_statistics… <http://apcentral.collegeboard.com/apc/public/repository/ap09_frq_statistics…> Oooh, I DO love the question Allan referred to. Me, I’m in semester-land here so have just given a midterm mini-project. It will probably never be anyone’s all-time, bar-none, hands-down, suitably-hyphenated favorite, but it’s interesting, so I thought I’d share. Our class has spent some time studying inhabitants of “The Island,” an ingenious place invented by Michael Bulmer and his colleagues in Australia. If you have never been there, you should visit! (details below) In their travels and explorations, some students noticed that, among married couples, it seems that it’s "more likely that either both people smoke or neither smokes." Of course what they MEANT by that is not exactly what that statement says. In class discussion, it came out that they meant something more complex and harder to say, namely, some conditional-probability version of the statement, such as, “if you smoke, you’re more likely to be in a relationship with someone who smokes than you would be if you didn’t smoke.” All this means that they have some challenges ahead. I told them that part of the point of the assignment was to use some technique for comparing two proportions. This means that they had to think about the situation and decide what two proportions to compare. In some online discussion in advance of the due date, lots of problems emerged, such as comparing the proportion of couples who both smoke to the proportion where neither smokes. Only about 20% of Islanders smoke overall, so showing that this difference is not equal to zero is not very interesting. More importantly, doing so says nothing about any association between choice of partner and smoking status. But among some students, the task may have helped some light bulbs go on about association, randomization, scrambling attribute values, and all those things we like about SBI. It was good for them to grapple with a problem where they had to think hard about the measure they were looking at in order to decide what exactly it measured; then construct and see its sampling distribution; and connect that to their observed values. Working on the Island was interesting too. We put together a Google form so that each student in the class went out to a village or two and randomly sampled ten households and record the smoking status — and some other information — about the inhabitants. It was a valuable change for the students NOT to have a big data set all clean and prepped for them, but rather to have to cope with collecting and entering the data themselves. Doing 10 each was not too hard, so thanks, Google, for making it possible for us to leverage the whole class’s efforts to getting us a decent-sized data set. Cheers to all, happy grading… Tim The Island: You could contact Michael Bulmer in Australia (m dot bulmer at uq dot edu dot au) and ask, or you can contact me; I’ve set up what I HOPE would be a “class” for SBI members, but I need to submit your email addresses for you to get an invite link. There is probably a page to request admission, but I don’t know where it is! Email me off-list: eepsmedia at gmail dot com. Michael has written about his invention, for example, in this TISE article: https://escholarship.org/uc/item/2q0740hv <https://escholarship.org/uc/item/2q0740hv> Our Data: Or you might just want to look at the results of our data collection, and do some analysis yourself: https://docs.google.com/spreadsheets/d/10BhVD7CoAMeSG8b-Hss93gAdsyXlhozf_7H… <https://docs.google.com/spreadsheets/d/10BhVD7CoAMeSG8b-Hss93gAdsyXlhozf_7H…>

9 years, 1 month

A sample item

by Josh Tabor

Hello Everyone- I wanted to share an item I included on my recent AP Statistics test. We had just finished traditional one sample tests for a proportion and one sample tests for a mean. I introduced the chapter with a simulation-based analysis of a test for a proportion, and we had done several similar activities earlier in the year. I like the item because it addresses the logic of inference, but doesn't match up with one of our traditional tests. Also, because it is about baseball! I pasted it below and in the attachment. I also included a brief answer key and comments on my student performance. Feel free to use it with your students if you are interested. Thanks, Josh Tabor In a recent Sports Illustrated article, author Michael Rosenberg addresses "America's Wait Problem." That is, he discusses how fans of some teams have to wait many, many years for their team to win a championship. In Major League Baseball, fans should expect to wait an average of 30 years for a championship-assuming all 30 teams are equally likely to win a championship each season. But is it reasonable to believe that all teams are equally likely to win a championship? In the last 19 seasons, only 10 different teams won a championship. Does this provide convincing evidence that some teams are more likely than others to win a championship? We can find out by testing the following hypotheses: H0: All 30 teams are equally likely to win a championship Ha: Some teams are more likely to win a championship than others. (a) Describe how to use slips of paper to simulate the number of different teams to win a championship in 19 seasons, assuming that each team is equally likely to win the championship each season. Write the name of each team on a slip of paper. Shuffle slips in a hat. Select one team, record, and replace slip. Repeat for a total of 19 champions. Record number of different champions. Repeat many times. Comments on student performance: Many students were able to describe how to prepare the slips. Most of these correctly selected 19 slips with replacement, although some left this out. However, many students did not discuss what they would do after the 19 selections (e.g., record the number of different champs). One hundred trials of a simulation were conducted, assuming that each team is equally likely to win the championship each season. The number of different teams to win a championship in 19 seasons was recorded for each trial on graphed on the dotplot. (b) There is one dot at 10. Explain what this dot represents. Number of difference champions In one simulated 19-year stretch, there were only 10 different champions. Comments: Most students did well on this part. (c) Based on the simulation and the data from the previous 19 seasons, is there convincing evidence that some teams are more likely to win a championship than others? Explain. Because it is very unlikely to get 10 or fewer different winners by chance alone (p-value 1/100), we reject H0. There is convincing evidence that some teams are more likely to win a championship than others. Comments: As expected, this was the hardest part for students. About 10-15% had no idea what to do or tried a traditional z or t test. About 50% focused on the wrong part of the sampling distribution (e.g., said the evidence was convincing because there were no dots at 19, said the evidence was convincing because the distribution was centered at 14-15, which was less than 19 or 30). The remaining students focused on the small probability that there would be 10 or fewer different champions by chance alone. Some even called this a p-value! A few argued that 10 different winners did not provide convincing evidence of Ha because 10 could happen by chance, even though it is would be fairly unusual.

9 years, 2 months

Re: [SBI] Groundhog Day, and a question about simulation methods

by Nathan Tintle

Homer- Sorry I didn't reply sooner, but I wanted to comment on your final paragraph which I've copied here for convenience: "Where does that leave us in teaching? Do we stick with simulation methods that model the role of chance intuitively? In that case we may get rather far into the weeds (after all, three options for simulation in the two-factor is alot for students to handle so early in the course), and we also end up, from time to time, using methods that won't be recommended in data analysis applications down the road. On the other hand if we employ simulation methods without regard to how intuitively they model the presumed role of chance variation in the production of the data, then we are back to using statistical procedures as black boxes that don't convey insight to students at the introductory level. My dilemma may be due in part to a lack of formal statistical training. Has anyone else found themselves puzzled by similar questions?" I'll give a two-pronged response to your questions. First, my personal opinion is to focus in a first course more on the intuitive and conceptual ideas, and less on technical details. Thus, I'm a lot less worried about matching the exact 'best practices' statistical methods with what I teach in Stat 101. For example, using Welch-Sattherwaite to approximate degrees of freedom or a +4 confidence interval or the exactly correct bootstrap, etc. The reality is that best practices (a) change frequently, (b) are extremely technical and (c) are rarely actually agreed upon by everyone (as you point out earlier in your email). So, I'd prefer to stay out of the weeds and focus on simple/intuitive approaches that model the logic of inference, with enough cautions to students that more precise/technical methods are available in practice. Secondly, with specific regards to simulation-based inference, what does this mean for me? It means, I'm actually not as concerned re: precisely matching the data production (e.g., random sampling vs. randomized experiment, etc.) with the simulation 'crank' used (e.g., bootstrap vs. permutation) and am comfortable with keeping students using only a single 'crank' (e.g., permutation test) or maybe two, even though this may present a bit of a disconnect between data production and analysis. In the ISI text, we navigate this by highlighting to students how study design impacts scope of conclusions (Can I generalize? Can I conclude cause-effect?) but don't worry about changing the analysis strategy as well. We find this keeps things more straightforward for students avoiding the 'too many technical details' problem that becomes the focus of a course for many students once you get into it. Finally, I'd point out that for a more advanced group of students, or in a second (third, beyond) course this might be exactly the kind of stuff you *do* want to talk about. But, for my Stat 101 students, this just seems like too much detail to be worth it. All of this being said, it was not an easy decision for our author team, nor one that we all necessarily would have made if we were doing so individually. I'm sure there are lots of other opinions out there and would love to hear from others on the listserv! Perhaps we should have some blog posts on this soon! Nathan On Tue, Feb 3, 2015 at 7:49 PM, Homer White < Homer_White(a)georgetowncollege.edu> wrote: > Hello Everyone, > > > It has been a joy to read about of the great data sets and activities! > I did not have time to respond yesterday, but here goes now. > > > I've got an introductory class with 26 students, and we meet all the > time in a computer classroom. Inspired by the Mosaic Project folks, we > teach with R in the R Studio Server environment. Our approach to > simulation-based inference is about 60% if of the way from traditional > inference towards Locke5/Intro to Statistical Investigations. > > > We do some simulation-based inference very early in the course -- > binomial stuff, on the first day of class, in fact -- and we try again to > work it in early with the chi-square test for the relationship between two > factor variables. We're in that unit now, so students have played a couple > of times with a "slow-simulation" app to get an idea of the null > distribution of the chi-square statistic; subsequently they have been > exposed to the standard chi-square test where the null distribution is > approximated by a chi-square density curve. They are told that it was > quite a godsend for Mr. Pearson to have stumbled on such a family of > approximating curves, because Mr. Pearson had no access whatsoever to > computing machinery. > > > Groundhog Day began with a little come-to-Jesus chat about the first > data analysis report, which the students tackled over the weekend. The > project involved looking at some data from a Current Population Survey and > to investigate the relationship between hourly wages and such factors as > sex, union membership status, race, etc. Apparently there is a rule at my > College that during Greek Rush Week critical thinking is forbidden, > including the act of determining the type of variables involved in your > research question prior to choosing your analytical tools. Accordingly > about a third of the students had attempted to make bar charts and > cross-tables to investigate, for example, whether men or women earn more, > even though wage is a numerical variable. So we cleared that up, I hope. > > > Harrumph. On with the intended show. > > > Today's plan is to revisit simulation one more time, in a situation > where you really need it (rather small number of observations). I bring > up the "ledge-jump" data (#59 in the classic Handbook of Small Data Sets). > a social psychologist studied 21 incidents in Britain involving a person > threatening to leap from the ledge of a building or other high structure. > The idea was to see what factors might affect the behavior of the crowd > that gathers in the street below. > > > To forestall morbid thoughts among students I get up on a nearby table > and assure them through role-play that nobody really gets hurt: the fire > truck comes right away and the firemen teeter back and forth with their big > yellow trampoline, your rabbi, psychiatrist and spouse are phoned and > within minutes they are leaning out of one nearby windows, soothing words > are spoken, sage advice is given, hope is restored. Eventually they talk > you back inside. > > > But in the meantime a crowd has gathered. Sometimes they wait more or > less in silence, appropriately mindful of the seriousness of the situation > playing out above. Sometimes, though, they begin to bait the would-be > jumper (muster up Cockney accent): "Go on, jump will ya?" > > > In the 21 incidents under study, the following was found: > > > weather/crowd Baiting Polite > > > cool 2 7 > > warm 8 4 > > > Discussion, with students ( me still on table): > > > Me: Somebody could say that, for one reason or another, a crowd is more > likely to bait in warm weather. Others might say that the outcome of these > 21 incidents had nothing to do with weather, but is just the result of > random variation in other factors -- above and beyond the weather at each > particular incident. Can you think of any other things besides the weather > that might affect crowd behavior? > > > Student: How many people show up to watch. The more that show up, the > more likely there jerk who will yell "jump' and get the rest of 'em started. > > > Student: How long they have to wait. Maybe if they stand around a long > time they'll get impatient. > > > Student (looking right at me, still on table): What about the dorkiness > of the would-be jumper? > > > Me: Uh, maybe. Gee, thanks, Zach. > > > I get down and we work it out on the blackboard: if weather has nothing > to do with crowd behavior, then our best guess based on the data is that in > each incident, regardless of weather, there is a 10/21 chance for the crowd > to bait. The chi-square test function that the class uses has built-in > provisions for simulation, of three possible types: > > > > - "fixed": the rows sums in the simulated table are constrained to be > the rows sums of the observed table, and you determined the probability of > each outcome in the columns by pooling the data, as we just did to get the > 10/21 figure. > - "double-fixed": both the row and column sums of the simulated table > are constrained to be equal to the row and column sums of the observed > table. > - "random": neither row nor column sums are constrained, and the > probability of a simulated observation landing in a particular cell is the > observed cell count divided by the grand total of the table. > > "random" make sense when the observed data are a random ample a larger > population, and chance comes into play just in the matter of who gets into > the sample. For example, if you randomly sample people and ask their sex > and where they prefer to sit a in classroom (front, middle or back), then > chance is not in how a fixed person will respond but in whether or not that > person gets into the sample. > > > "double-fixed" (corresponding to the way simulation is done in R's > chisq.test(), and probably in many other software systems as well), appears > to be ideally suited for randomized experiment in which the Null hypothesis > imagines that a subject's response is the same regardless of which > treatment group one is placed into. In that case a different random > assignment of subjects to treatment groups might result in a different > table, but the row and column sums would be the same as for the table we > observe in the actual experiment. > > > "fixed" seems to be the right thing for the ledge-jump situation, if we > assume that the 21 incidents weren't sampled randomly out of some larger > population, that they were the only 21 incidents that occurred under the > period of study in the region under study. In that case the weather at the > time of each incident simply was it was, and chance comes into the > production of the observed table through random variation in all other > factors (conditional upon the weather). > > > So we do simulation with the "fixed" option. Everybody's P-value comes > in around 5%, so we decide that don't have overwhelming evidence that > weather and crowd behavior are related. > > > Now I come round to my questions. > > > When we teach inference though simulation, we don't want it to become > another "black box" for students. We want them to see that the simulation > method generates simulated data that reasonably could occur if the > study were to be conducted in a hypothetical world where the Null > hypothesis is definitely true. Hence we the simulation method has to > model, quite transparently, the role that we think chance played -- if the > Null is true -- in giving us the data we actually see. > > > But there appears to be controversy among statisticians as to which > simulation method is best to use for contingency tables. (See e.g., > Agresti, Categorical Data Analysis Third Edition section 3.5.6). I suppose > that sometimes it's possible for a particular simulation method not to > model the role of chance very well, but to possess superior statistical > properties nonetheless, maybe even be the state-of-the art method. (This > situation seems to occur also in bootstrap hypothesis testing, where the > more preferred re-sampling method is rather more difficult to justify > intuitively than is the "naive" re-sampling method.) > > > Where does that leave us in teaching? Do we stick with simulation > methods that model the role of chance intuitively? In that case we may get > rather far into the weeds (after all, three options for simulation in the > two-factor is alot for students to handle so early in the course), and we > also end up, from time to time, using methods that won't be recommended in > data analysis applications down the road. On the other hand if we employ > simulation methods without regard to how intuitively they model the > presumed role of chance variation in the production of the data, then we > are back to using statistical procedures as black boxes that don't convey > insight to students at the introductory level. > > > My dilemma may be due in part to a lack of formal statistical training. > Has anyone else found themselves puzzled by similar questions? > > > Homer S. White > Professor of Mathematics > Georgetown College, KY 40324 > 502-863-8307 > > *Notice*: *This message may contain confidential information and is > intended for the person/entity to whom it was originally addressed. Any > use by others is strictly prohibited. If you received this email in error, > please permanently delete it and disregard.* > -- Nathan Tintle, Ph.D. Associate Professor of Statistics and Dept. Chair Director for Research and Scholarship Dordt College Sioux Center, IA 51250 nathan.tintle(a)dordt.edu Phone: (712) 722-6264 Office: SB1612

9 years, 2 months

Anyone have access to the data on this study?

by Kevin Rees

Dear all, One of my colleagues passed around this New York Times article on gender roles in the workplace. http://www.nytimes.com/2015/02/08/opinion/sunday/sheryl-sandberg-and-adam-g… In the article they refer to a study in the Journal of Applied Psychology, in the following paragraph: In a study <http://psycnet.apa.org/journals/apl/90/3/431/> led by the New York University <http://topics.nytimes.com/top/reference/timestopics/organizations/n/new_yor…> psychologist Madeline Heilman, participants evaluated the performance of a male or female employee who did or did not stay late to help colleagues prepare for an important meeting. For staying late and helping, a man was rated 14 percent more favorably than a woman. When both declined, a woman was rated 12 percent lower than a man. Over and over, after giving identical help, a man was significantly more likely to be recommended for promotions, important projects, raises and bonuses. A woman had to help just to get the same rating as a man who didn’t help. I was interested in using this data during class, but the only way to access the data is to pay $12 for the article. Does anyone have the data that led to the above values that we could use in a simulation experiment? Thanks for your help, Kevin -- Kevin Rees Math Department Chair Marin Academy www.ma.org 415-482-3260

9 years, 2 months

Re: [SBI] Happy Groundhog Day! What happened in your introductory statistics class today?

by michelle_sisto

Hi Everyone, It has been a pleasure reading this thread all week. Unfortunately I'm not teaching Intro Stats right now. However, I am teaching VBA programming in Excel to Master of Finance students and we use an awful lot of simulation and statistics. The class consists of 3 hour sessions in a computer lab, once a week over 6 weeks. Yesterday in class (week 4) the students worked with 10 years of monthly adjusted closing prices from two stocks. They had to write a program to find monthly returns of the stocks, the means and variances of each, and then generate portfolios of the two stocks by varying weights to each in increments of 10% or an increment input by use. Lastly, they found the minimum variance portfolio. After getting the programming down and recalling the basic gist of a two asset portfolio, they created histograms of each stock's returns to see if modeling the returns as a normal distribution may be reasonable. After which, they simulated new stock return data by generating random returns from a normal distribution and a t-dist with same mean and variance of original sample returns. They examined how the efficiency frontier and minimum variance portfolio varied with new data. Homework for next time is to expand the exercise further by introducing investment in two stocks and a "risk free" asset. Here we'll play around a good bit with how varying the interest rate of the risk free asset impacts the portfolio composition. While it isn't quite on topic - thought I'd share. Also would love to have Robin's data set on population of countries to predict land area. I'm quite sure Monaco is an outlier on this one! Michelle Sisto EDHEC Business School Nice, France From: sbi-bounces(a)causeweb.org To: sbi(a)causeweb.org Cc: Date: Mon, 02 Feb 2015 21:50:45 -0500 Subject: Re: [SBI] Happy Groundhog Day! What happened in your introductory statistics class today?  Allan, et al - Nice idea! Saw your message right after I came out of class and should have responded then when it was fresh. This was day 9 of a 42 class semester and we're finishing up the chapter on descriptive statistics/graphics (after initial classes on data production, sampling, experiments). First exam is later this week and first project (describing their own dataset) is also due at the end of the week. Today's topic was least squares regression (after doing scatterplots and correlation last class). Started with a Fathom demo where students put a movable line on a scatterplot to show the trend (we meet full time in a computer classroom), had Fathom show the "squares" and find SSE to see whose line was best. Then let them move their lines around to try to lower SSE until they found essentially the least squares line. Once they saw how to find the fit automatically in Fathom we looked at several situations: Hgt to predict Wgt (interpreting the slope), population of countries to predict land area (showing influential points/outliers), presidential approval rating as a predictor of election margin, page number in Consumer Report Guide to predict fuel capacity of a car (no relationship). They then did another Fathom demo, dragging a point around to see the effect on the least squares line, especially when the the point became very influential. We finished up by introducing r^2 as the proportion of variability in response explained by the predictor. This involved one more Fathom demo where they made the movable line horizontal to see that the "best" constant is the mean, giving the total variability as its sum of squared errors, then found the "improvement" with the least squares line to measure the variability explained by the predictor. Lots of Fathom today, but that changes next class (Wednesday) when we switch to StatKey and start in on simulations to lead to bootstrap confidence intervals - a topic more relevant to this list. Robin Lock St. Lawrence University On 2/2/2015 12:34 AM, Allan Rossman wrote: Happy Groundhog Day! I continue to find it inexplicable that neither private colleges nor public universities see fit to cancel classes out of respect for this august occasion. But this year I've decided to try to make the best of this lamentable oversight, and I need your help! I think it might be fun to ask introductory statistics teachers to compare notes on what's happening in their classes on one particular day. What better day than Groundhog Day for revisiting the same question over and over, and over and over, and over and over, from multiple perspectives? I'm writing this after Groundhog Day has officially begun in Punxsutawney, Pennsylvania, but it's shortly after 9pm on Super Bowl Sunday here in California. So, to get the ball rolling on this whimsical idea (I strongly prefer the word "whimsical" to "silly" in this context), I'll use future tense to anticipate what will happen in my class on Monday. I plan to be sound asleep when Punxsutawney Phil makes his celebrated prognostication. (Too much information: Thirty years ago I did indeed make the trek to Gobbler's Knob with my future bride before sunrise on February 2, but I won't be up so early or anywhere near Punxsutawney this year!) My introductory students and I in STAT 217-09 at Cal Poly will begin the fifth week of our ten week term on February 2 by finishing up a discussion of principles of well-designed experiments. We’ll discuss a study conducted at Harvard about whether students spend $50 differently depending on whether they’re told that it’s a “tuition rebate” or “bonus income.” Then we’ll consider one of the first studies of the drug AZT for reducing mother-to-child transmission of HIV. We’ll culminate this discussion by collecting some in-class data on a very simple randomized experiment investigating whether grouping of letters can affect memory. All students will receive the same 30 letters in the same order, but some will find convenient, recognizable three-letter groupings and others will see more irregular groupings of letters. Then I expect to have time to introduce a study about whether swimming with dolphins is beneficial to patients who suffer from clinical depression. We'll discuss the design of the study and do a quick exploration of the 2x2 table of results, setting the stage for simulating a randomization test to assess whether the difference between success proportions in two treatment groups is statistically significant. Carrying out this simulation in class, using cards and then an applet, will have to wait until February 3 when the excitement of the momentous day has passed. (Or who knows, perhaps my students and I will find when we awake on Tuesday that we are destined to magically relive Monday again and again...) Please indulge me in this fanciful exercise by replying to this Simulation-Based Inference listserv with a description of what happened, or will happen, in your introductory statistics class on Groundhog Day 2015. Maybe we statistics teachers will learn something interesting by exchanging this information and reflecting on the variety of responses. Even if not, we can honor the grand tradition of Groundhog Day by engaging in a substantially less grand but only marginally more silly (oops, I mean whimsical) one. With best wishes for the special day and for an early spring (to those of you who must endure winter), Allan Rossman -- Allan J. Rossman Professor and Chair Statistics Department Cal Poly San Luis Obispo, CA 93407 arossman(a)calpoly.edu http://statweb.calpoly.edu/arossman/ _______________________________________________ SBI mailing list SBI(a)causeweb.org https://www.causeweb.org/mailman/listinfo/sbi -- Robin Lock Burry Professor of Statistics St. Lawrence University

9 years, 2 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

SBI