|
| Hierarchies of Evidence? Randomized Trials, Professional Judgment, and Single Subject Design |
| Sunday, May 27, 2007 |
| 9:00 AM–10:20 AM |
| America's Cup C |
| Area: EDC; Domain: Applied Research |
| Chair: Ronnie Detrich (Wing Institute) |
| Discussant: Timothy A. Slocum (Utah State University) |
| Abstract: Evidence-based practice has been defined as the best available research evidence combined with professional judgment (Sackett, 2000). As always the devil is in the details and there is no common agreement of what constitutes best available evidence and the role of professional judgment. Most organizations that have developed standards of evidence have defined randomized clinical trials as the “gold standard” and have given considerably less standing to evidence derived from single subject research. Placing randomized trials at the top of the hierarchy is not without problems particularly as it applies to special education research. The purpose of this symposium is to consider the relative strengths and weaknesses of randomized trials, single subject designs, and professional judgment and their role in evidence-based practice. |
| |
| Professional Judgment: Fallibility, Inevitability, and Manageability. |
| RANDY KEYWORTH (Wing Institute), Ronnie Detrich (Wing Institute), John E. States (Wing Institute) |
| Abstract: Sackett (2000) defined the evidence-based medicine as being comprised of the best available research evidence combined with professional judgment. This definition serves as the cornerstone of evidence-based education as well (Whitehurst, 2005). The question remains: what role does professional judgement play in a decision-making approach which depends to a great extent on empirical evidence. This question is very important because of the large literature base that suggests that professional judgment is subject to a wide variety of biases that may result in faulty decision-making. In this paper we will review the status of professional judgment, the sources of biases, how these biases can be minimized, and explore how judgment can serve as a valuable adjunct to the empirical evidence, especially when the best available evidence is not very strong. |
| |
| Randomized Clinical Trials: The Versatility and Malleability of the “Gold Standard". |
| JOHN E. STATES (Wing Institute), Randy Keyworth (Wing Institute), Ronnie Detrich (Wing Institute) |
| Abstract: As the evidence-based practice movement has grown across disciplines randomized clinical trials have usually been accepted as the gold standard for evaluating the quality of evidence. Most organizations that have established standards of evidence have clearly shown a preference for randomized trials. In this paper we will review the basic assumptions, the strengths and limitations, and the types of questions that are best answered by randomized trials. We will highlight the problems of using randomized trials in special education at the level of participant selection, potential ethical constraints, logistics, and costs. We will also discuss some of the strategic costs to the field of behavior analysis for failing to conduct randomized trials as part of the effort to develop widely accepted evidence-based behavioral interventions. |
| |
| Single Subject Research and Evidence-Based Interventions: Are SSDs Really the Ugly Stepchild? |
| RONNIE DETRICH (Wing Institute), Randy Keyworth (Wing Institute), John E. States (Wing Institute) |
| Abstract: As the movement for evidence-based interventions grows in the field of education, single subject designs have been largely relegated to a low status within the hierarchy of evidence. The risk to the field of education for not giving greater consideration of evidence developed from single subject research is that many very powerful interventions will be ignored because they do not meet the standards of evidence established by many of the professional organizations. In this paper we will review the basic assumptions of single subject designs, the basic strengths of these designs, and their limitations, specifically addressing issues related to generality (external validity). Single subject designs offer great flexibility for answering questions for special education populations when drawing representative population samples can be difficult. Models for how single subject research can be incorporated into decision rules for determining the evidence-base for an intervention will be reviewed. |
|
| |