|
The Hoped-for Demise of Significance Testing: Why and How |
| Sunday, May 25, 2014 |
| 11:00 AM–11:50 AM |
| W178a (McCormick Place Convention Center) |
| Area: TPC; Domain: Theory |
| Instruction Level: Intermediate |
| CE Instructor: Marc N. Branch, Ph.D. |
| Chair: M. Jackson Marr (Georgia Tech) |
| MARC N. BRANCH (University of Florida) |
 Marc N. Branch was introduced to behavioral approaches while an undergraduate at Stanford University in the 1960s. After graduate-school stints at Arizona State University and the University of Maryland, followed by a post-doctoral year at the Worcester Foundation for Experimental Biology, Dr. Branch took a position as a faculty member in the Psychology Department at the University of Florida in fall of 1973. During his time there he served, among other duties, as editor of the Journal of the Experimental Analysis of Behavior and The Behavior Analyst, as chairman of the Psychology Department, and as president of ABAI. He retired from teaching in the summer of 2012 and is now professor emeritus of psychology. |
| Abstract: Despite more than 60 years of published information clearly showing that null-hypothesis significance tests (NHSTs) and the p values associated with them provide essentially no information about the reliability (i.e., probability of replication) of research outcomes, they remain at the core of editorial decision-making in the behavioral sciences, including psychology, with statistical significance serving as the major gateway to publication of research results. Two reasons appear to contribute to the continuing practice. One, information available suggests that a majority of psychological researchers incorrectly believe that p values do provide information about the reliability of research results. Two, among the minority thatare aware that p values do no such thing, a position sometimes taken is that even though p values do not provide the information many think they do, using them to make decisions about whether to believe in research results is and has been essentially benign. This paper addresses both reasons. Because the first has been pointed out many times, it is briefly covered, because of the apparent persistence of the misunderstanding. The second, that NHSTs have no significant negative effects on behavioral sciences, is the focus of the major portion of the paper, which describes seven “side-effects” of NHSTs that continue to retard effective development of psychological science. The paper makes an appeal to journal reviewers and editors to de-emphasize or eliminate the role of NHSTs, and it closes by offering a few suggestions about alternatives that could be considered and with a challenge to psychological researchers to develop new methods that more fully assess the reliability and generality of research findings. |
| Target Audience: Anyone interested in data-analysis techniques. |
| Learning Objectives: 1. Define a p value. 2. Indicate why a p value provides no information about the probability that research results are "due to chance." 3. Indicate at least one way in which significance testing has hindered the development of behavioral science. |
| Keyword(s): Null-hypothesis, P-value, Reliability, Statistical significance |
|
| |