Shaping up the practice of null hypothesis significance testing

Recent criticisms of null hypothesis significance testing (NHST) have appeared in education and psychology research journals (e.g., Cohen, 1990, 1994; Kupfersmid, 1988; Rosenthal, 1991; Rosnow & Rosenthal, 1989; Shaver, 1985; Sohn, 2000; Thompson, 1994, 1997; see also Research in the Schools [1998]). In this article we discuss these criticisms for both current use of NHST and plausible future use. We suggest that the historical use of such procedures was reasonable and that current users might spend time profitably reading some of Fisher's applied work. However, we also believe that modifications to NHST and to the interpretations of its outcomes might better suit the needs of modern science. Our primary conclusion is that NHST is most often useful as an adjunct to other results (e.g., effect sizes) rather than as a stand-alone result. We cite some examples, however, where NHST can be profitably used alone. Last, we find considerable experimental support for a less rigid attitude toward the interpretation of the probability yielded from such procedures.