Literature Index

One objective of this chapter is to introduce a diagnostic approach to assess how well students answer questions about a research report. A second objective is to show how the proposed assessment tools can be used to identify teaching strategies for overcoming students' errors in interpreting reports. A third objective is to suggest how the interpretation-of-research assessment questions can be used to help students identify what information needed for interpretation is missing in journal or media reports.

As part of the assessment after an introductory statistics course, students had to do a small project and submit a written report describing their methods, results and conclusions. We supported the report writing and the project work by several means. Among others, we developed an "exemplary project report" they were introduced to. This project report was written in two columns. In the first column the report about a question concerning a data set is with our best knowledge, in the accompanying second column, we reflect on the choices and options to be made in the respective stages of the report. The aim is to stimulate meta-cognitive activity and to help the students seeing the general in the particular of the exemplary report. We got several dozens of project reports and analyzed them carefully. We developed a grading scheme with several dimensions, including the quality of introductory and concluding sections, the quality of method choice and the quality of analysis and conclusions. We did not only pay attention to statistical quality but also to questions of style of writing such as whether the project question is introduced in a motivating manner and whether clear and convincing conclusions are presented to the reader with good communicative means including adequate graphs. The grading scheme was used to provide feed-back to the students. On the other hand we used this scheme for a systematic analysis of the available project reports. Weaknesses and strengths, most difficult areas for our students were identified and we were able to reflect on the adequacy and the shortcomings of our guiding "exemplary report" and our grading scheme. I will present those findings of our study that seem to be generalizable with regard to the question of assessing students' knowledge by means of projects and of report writing.

The ability to interpret and predict from data presented in graphical form is a higher-order thinking skill that is a necessity in our highly technological society. Recent recommendations for the mathematics and science education communities have therefore stressed the importance of engaging learners in real life statistical tasks given in a setting that will promote effective problem solving. Since the small-group setting has been shown to be a fertile environment in which problem solving can occur, we have used that setting for engaging students in data analysis tasks. However, there is a dearth of ideas related to how to assess students' behavior, thinking, and performance in such a setting. The purpose of this chapter is to describe a framework for assessing students' problem solving behaviors on a graph task as they work within a small-group setting.

Statistical literacy, reasoning, and thinking may be the most prominent objectives of statistics education; they are unsatisfactorily defined and demarcated. Therefore, they are difficult to monitor, and assess. As a consequence they are impractical as educational goals. Instead, assessment could be focused on those aspects of specific statistical knowledge that are indicative for different levels of understanding. Factual knowledge directly derived from sources of information indicates a superficial level of understanding; a comprehensive, coherent knowledge structure indicates a more profound level of understanding, and the ability to transfer knowledge (the ability to flexibly engage statistical knowledge in novel tasks) indicates an expert level of understanding. This classification of hierarchically related levels of statistical understanding may produce adequate ways of measurement and assessment.