Association for Behavior Analysis International

This symposia will provide four papers offering different statistical procedures that may be useful for behavior analysts. There are many ways to express effect sizes between a specified active intervention and control conditions. Our first paper will provide a brief tutorial and discusses the potential utility of Number Needed to Treat as a clinically useful measure of the effectiveness of treatment in behavioral research. Our second paper will introduce a re-thinking of a regression-based procedure, the dynamic P-technique, for single subject designs. We will show its utility for applying traditionally group-type analyses (such as longitudinal mediation analyses) to single case designs. The third paper will reexamine the role of inter-observer agreement in behavior analysis in light of Skinner's (1945) critique of the "arid philosophy of 'truth by agreement'" (p. 293). In addition, this paper will describe the utility of statistical bootstrapping analysis as a direct assessment of workability in both single subject and group designs. The fourth paper, we will examine regression procedures broadly, arguing that they do not require data from multiple participants to be used properly. This paper will explore applications of regression analysis to exploring behavior-behavior relations, context-behavior relations, and behavior-behavior relations across different contexts.

There are many ways to express comparisons between a specified active intervention and a control conditions or other active intervention. Number Needed to Treat (NNT) is one way to express such comparison and was introduced by Laupacis, Sackett, and Roberts (1988) as a clinically useful measure of the effectiveness of treatment. NNT is an expression of how many people need to be treated in one intervention versus another to have one person with the desired outcome. NNT can be used to determine the clinical significance of results and can be seen asa measure of effect size. A small NNT reflects a large effect size. That is, the smaller the NNT the more people in the treatment group achieved the desired outcome. Currently, NNT is nearly extensively used in medical and psychopharmacological randomized trials and observational studies. This paper provides a brief tutorial and discusses the potential utility of NNT in behavioral research.

The reluctance to use statistical procedures in behavior analytic research has perpetuated for at least two substantive reasons: First, the popular statistical procedures employed in the group designs common within mainstream clinical psychology are ill-suited for the analyses behavior analysts are primarily interested in intra-subject change. Second, development and dissemination of adequate statistical procedures for examining intra-individual change has lagged tepidly behind large-N procedures. One of the many undesirable consequences of this analytic divide between mainstream clinical psychology and behavior analysis is uncertainty about applying group models for change to single cases and vice versa. This paper will introduce a re-thinking of an old regression-based procedure, the dynamic P-technique (see Nelson, Aylward, & Rausch, 2011). When extended repeated measures are feasible, as in daily diary tracking, substantive researchers can use the dynamic-P technique to construct single subject regression models, such as longitudinal panel models, longitudinal factor analysis, and longitudinal mediation analyses. In this paper, we will argue that the dynamic-P technique may be a feasible way to bridge the gap between group and single case models.

Inter-observer agreement (IOA) is considered an essential component of single subject designs that employ non-automated measurements of behavior. While many different forms of IOA have been developed, all share the common goal of assessing formal correspondence between independent observers. That is, whether observers agree about the quality of one or more topographical dimensions of a behavior. While topographical agreement may be sufficient to demonstrate reliability of measurement, this paper will argue that it is not necessary. Skinner (1945), in a commentary accompanying his seminal paper on operationalism, decried the "arid philosophy of 'truth by agreement'" (p. 293) and asserted that workability is paramount to agreement. This paper will explore Skinner's relatively unknown position on this topic with an emphasis on the philosophical assumptions that underlie workability as a scientific truth criterion. In addition, this paper will describe the utility of statistical bootstrapping analysis as a direct assessment of workability in both single subject and group designs. Finally, this paper will propose areas of scientific inquiry that, while largely lacking with regard to traditional IOA, may be amenable to study using workability as the criterion for experimental control.

Many behavior analysts have rejected statistical analyses like regression as irrelevant for understanding individual-level functioning. These approaches often group individual data points into distributions that are then compared with distributions from other groups or of other behaviors. Traditional statistical techniques offer limited attention to relationships 1) between behaviors or 2) between particular contexts and behaviors. However, statistical analyses do not require data from multiple participants to be used properly. Regression analysis may be a particularly powerful approach to statistical analysis of data collected from a range of single-subject research designs. In this approach, regression lines represent some aspect of an individual's repertoire, instead of multiple repertoires collapsed into one pair of distributions. This may be timely as an increasing number of behavior analysts are becoming interested in the relations among private events and overt behaviors in different contexts, and calling for analyses that include these relations. This paper will explore applications of regression analysis to exploring behavior-behavior relations, context-behavior relations, and behavior-behavior relations across different contexts. Data collection methodologies will be discussed along with relevant statistical assumptions.