Two studies investigated upper elementary school students' informal understanding of sampling issues in the context of interpreting and evaluating survey results. The specific focus was on the children's evaluation of sampling methods and means of drawing conclusions from multiple surveys. In Study 1, 17 children were individually interviewed to categorize children's conceptions. In Study 2, 110 children completed paper-and-pencil tasks to confirm the response categories identified in Study 1 and to determine the prevalence of the response categories in a larger sample. Children evaluated sampling methods focusing on potential for bias, fairness, practical issues, or results. All children used multiple types of evaluation rationales, and the focus of their evaluations varied somewhat by context and type of sampling method (restricted, self-selected, or random). Children used affective (fairness) rationales more often in school contexts and rationales focused on results more often in out-of-school contexts. Children had more difficulty detecting bias with self-selected sampling methods than with restricted sampling methods because self-selection was initially the most fair (i.e., everyone had a chance to participate). Children preferred stratified random sampling to simple random sampling because they wanted to ensure that all types of individuals were included. When drawing conclusions from multiple surveys, children: (1) considered survey quality; (2) aggregated all surveys regardless of quality; (3) used their own opinions and ignored all survey data; or (4) refused to draw conclusions. Even when children were able to identify potential bias, they often ignored survey quality when drawing conclusions from multiple surveys.
The CAUSE Research Group is supported in part by a member initiative grant from the American Statistical Association’s Section on Statistics and Data Science Education