|
| What Does "ResearchBased" Really Mean? Evaluating and Predicting the Likely Success of Behavioral Programs |
| Saturday, May 29, 2004 |
| 8:00 AM–11:00 AM |
| Beacon A |
| Area: PRA; Domain: Applied Research |
| CE Instructor: Janet S. Twyman, M.D. |
| JANET S. TWYMAN (Headsprout), T. V. JOE LAYNG (Headsprout) |
| Description: Current uses of the term research-based as applied to educational and behavioral programs range in meaning from claims that: Programs contain elements found by research to be effective; pretest vs. post-test or simple comparison studies have provided evidence of effectiveness; or the program has undergone some form of scientifically controlled study, often involving randomized control groups. Further adding to the confusion, all of these uses fail to distinguish between the scientific development of a program, and the scientific evaluation of outcomes after a program is developed. This latter use of researchbased might more properly be considered researchfiltered. That is, a program, regardless of how it was designed, is measured against an alternative form of instruction, or at times, no instruction at all. This use of the term researchbased would find in its referent an emphasis on summative evaluation. Yet another use of the term researchbased might be more properly considered as pertaining to a behavioral intervention or instructional program that has been scientifically developed. That is, it is designed and constructed often-following design test revise retest cycles until each individual is shown to succeed. This use of the term researchbased would find in its referent an emphasis on formative evaluation (after Scriven, 1974). This workshop will provide an overview of how the term research-based needs to be examined using criteria based on the type of formative and summative evaluation that has been conducted in developing behavioral intervention and instructional programs. The workshop will present and provide practice in using a method educators can employ 1) to evaluate claims made by consultants, vendors and program authors, 2) to judge data supplied by reading program publishers, and 3) to make predictions of the likely success of a program. A 3X3 Matrix will be used to describe the various levels of interaction between each subtype of evaluation and the effect it has on predicting the success of both individuals and groups using research-based behavioral and instructional programs. |
| Learning Objectives: At the end of the workshop, participants will be able to: Distinguish between three types of formative evaluation (experiential, evidence-based, and scientific), and three types of summative evaluation (experiential, evidence-based, and scientific). Use a provided 3X3 Matrix to describe the various levels of interaction between each subtype of evaluation and the effect it has on predicting the success of both individuals and groups using research-based behavioral or instructional programs. Use the seven characteristics of the formative evaluation process to better understand the important role formative evaluation may play in developing and evaluating student success in research-based behavioral or instructional programs. Make projections of the likely success of interventions based upon the "research-base" offered as evidence of effectiveness of selected programs. |
| Activities: Audio visual presentation involving computer projected slides, animations, and video; small group application of evaluation guidelines to a range of intervention examples. |
| Audience: All who use or consume behavioral or educational programs or products. |
| Content Area: Practice |
| Instruction Level: Basic |