This study investigated the knowledge base necessary for choosing appropriate statistical techniques in applied research. In this study, we compared knowledge used by six experts and six novices in two types of statistical tasks. The tasks were: 1) comparing research scenarios from the perspective of choosing a statistical technique, and 2) direct comparison of statistical techniques. The framework was based on expert knowledge in inferential statistics using the repertory grid technique for data collection. A qualitative analysis of data showed that of the three types of expert knowledge, research design knowledge comprised the biggest portion, with theoretical and procedural knowledge comprising relatively smaller parts. Little difference was observed between experts and novices in extensiveness of knowledge use, although experts' knowledge use was found to be more integrated than novices'. Finally, two implications were drawn regarding how to better teach selection skills in statistics education: (1) statistical techniques should be taught in relation to relevant research designs, and (2) conceptual connections between statistical techniques should be explicitly taught.
The CAUSE Research Group is supported in part by a member initiative grant from the American Statistical Association’s Section on Statistics and Data Science Education