|Applied Research on Measurement and Instrumentation
|Sunday, May 25, 2014
|9:00 AM–10:50 AM
|W187ab (McCormick Place Convention Center)
|Area: DDA; Domain: Applied Research
|Chair: Kevin C. Luczynski (Munroe-Meyer Institute, University of Nebraska Medical Center)
|Discussant: Brian A. Iwata (University of Florida)
|CE Instructor: Kevin C. Luczynski, Ph.D.
Measurement systems and data-analysis methods that produce accurate and sensitive measures of the target behavior are requisite for assessment and treatment. The papers in this symposium, collectively, describe efforts toward improving measurement systems and data-analysis methods in applied research. Lesser et al. compared the accuracy and efficiency of five systems for measuring sleep disturbances in children's bedrooms. Zarcone et al. improved the precision of observation methods to detect treatment gains, beyond the common measure of frequency, by measuring the force of problem behavior. Mead and Iwata compared the extent to which sufficient interobserver-agreement scores would be obtained using a proportional reliability method with 10-s versus 1-min intervals. Roberts and Bourret compared the strengths and weaknesses of three methods for quantifying the relation between two events during descriptive assessments. We are fortunate to have Dr. Brian Iwata serve as the discussant for this set of papers, given his exceptional scholarship in this area.
|Keyword(s): assessment, data analysis, interobserver agreement, measurement
A Comparison of the Accuracy and Efficiency of Measurement Systems to Score Sleep Disturbances Exhibited by Children with an Autism Spectrum Disorder
|AARON D. LESSER (Munroe-Meyer Institute, University of Nebraska Medical Center), Kevin C. Luczynski (Munroe-Meyer Institute, University of Nebraska Medical Center), Mychal Machado (Munroe-Meyer Institute, University of Nebraska Medical Center)
Sleep disturbances affect up to 68% of children diagnosed with an autism spectrum disorder (Richdale & Schreck, 2009). The use of direct observation on a second-by-second basis produces qualitative and quantitative information on sleep disturbances, but applying this type of measurement system throughout the night may not be practical. We conducted a measurement comparison across four nights with two children to evaluate the accuracy and efficiency of actigraphy, parent diaries, motion detection, momentary time sampling at 5-min and 10-min intervals, and fast-forwarding. All data were obtained from the childrens home and were remotely transferred for analysis via the internet. The sleep measures from each measurement system were compared to a second-by-second criterion record (continuous observation). The dependent variables for accuracy included total sleep disturbance, sleep-onset latency, nighttime wakings, early wakings, and oversleeping. The dependent variables for efficiency included the number of hours to collect data. The results indicated that motion detection closely matched the criterion measure for total sleep disturbance. The most variability within and across measurement systems was observed for night wakings. These preliminary results suggest that motion-detection software is an accurate and efficient measurement system.
|Measuring the Force of Problem Behavior
|JENNIFER R. ZARCONE (Kennedy Krieger Institute), Griffin Rooker (Kennedy Krieger Institute), Mindy Christine Scheithauer (Kennedy Krieger Institute), Jonathan Dean Schmidt (Kennedy Krieger Institute), Iser Guillermo DeLeon (Kennedy Krieger Institute)
|Abstract: Treatment procedures for problem behavior often rely on measures of frequency to gauge treatment effectiveness. For the most severe behaviors, the force of the behavior may be equally relevant to evaluating the effectiveness of treatment outcomes. The goal of this study is to evaluate practical procedures for measuring the force of problem behavior during standard ABA treatment procedures. Four children who were hospitalized for the treatment of severe problem behavior participated in the study. A 3-point rating scale was developed to rate the forcefulness of behavior from 1 (low force) to 3 (high force). Both frequency and force of behavior was measured for all participants during baseline and treatment using differential reinforcement (DRA) or noncontingent reinforcement (NCR). Results showed that for all participants, treatment was effective at reducing the occurrence of problem behaviors. When DRA was used however, the frequency of the target behavior decreased when the DRA schedule was thinned, but the force remained high. For the participants treated with NCR, the force was initially very low during treatment, but increased when the schedule of reinforcement was thinned. These data imply that NCR may be a better treatment if reducing the force of behavior is the treatment goal.
|Interval Length Influences on Proportional Reliability
|SARAH C. MEAD (University of Florida), Brian A. Iwata (University of Florida)
|Abstract: Accuracy of measurement is a crucial component in all research but may be difficult to assess in applied research on human behavior because there is no “true standard” for observation. Consequently, reliability, or interobserver agreement, is used as an approximation to accuracy. Proportional reliability is a common method for calculating interobserver agreement for frequency measures of responding, but the resulting score can be influenced by a number of variables, including the interval length used as the basis for agreement. Although a 10-s interval typically is used as the basis for calculation, the unit of measurement for response frequency usually is a 1-min rather than a 10-s interval. We compared proportional reliability scores using the traditional 10-s interval to scores using a 1-min interval for 40 sample 10-min sessions. We considered sessions with high and low rates of responding and high and low reliability scores calculated using 10-s intervals. Our results suggest that one minute may be an acceptable interval length for calculating proportional reliability for frequency measures reported as responses per minute.
Methods for Descriptive Analysis Data Collection
|KYLIE ROBERTS (The New England Center for Children), Jason C. Bourret (The New England Center for Children)
A number of different methods are used to calculate and compare the probability of events given specific environmental variables. This investigation includes a comparison of three different methods. The first, an exhaustive contingency space analysis described by Vollmer, Borrero, Wright, Van Camp, & Lalli (2001), compares the probability of an event occurring at any time during an observation to the probability of an event given behavior. The second method, an exhaustive contingency space analysis described by Hammond (1980), compares the probability of an event given behavior to the probability of an event given the absence of behavior. The third method, a non-exhaustive contingency space analysis described by Luczynski and Hanley (2009), evaluated the probability of an event and an environmental variable by subtracting the probability of an event given the absence of an environmental variable from the probability of an event given behavior. Findings are discussed in terms of strengths and weakness across varying frequency of responding.