Despite widespread use of significance testing in empirical research, its interpretation and researchers' excessive confidence in its results have been criticized for years. In this article, the logic of statistical testing in the Fisher and Neyman-Pearson approaches are described, some common misinterpretations of basic concepts behind statistical tests are reviewed, and the philosophical and psychological issues that can contribute to these misinterpretations are analyzed. Some frequent criticisms against statistical tests are revisited, with the conclusion that most of them refer not to the tests themselves but to the misuse of tests on the part of researchers. In accordance with Levin (1998a), statistical tests should be transformed into a more intelligent process that helps researchers in their work. Possible ways in which statistical education might contribute to the better understanding and application of statistical inference are suggested.
The CAUSE Research Group is supported in part by a member initiative grant from the American Statistical Association’s Section on Statistics and Data Science Education