Research Article

Efficacy of Differential Negative Reinforcement of Alternative Behavior to Improve Reading Comprehension

Nicole Goehring and John W Maag*

Special Education & Communication Disorders, University of Nebraska-Lincoln, Lincoln, United States

Submission: August 29, 2020; Published: October 06, 2020

*Corresponding author: John W Maag, University of Nebraska-Lincoln, Lincoln, Nebraska, United States

How to cite this article: Nicole Goehring and John W Maag*. Efficacy of Differential Negative Reinforcement of Alternative Behavior to Improve Reading Comprehension. Psychol Behav Sci Int J. 2021; 15(4): 555917. DOI: 10.19080/PBSIJ.2020.15.555917

Abstract

A multiple baseline design across participants was used to determine if differential negative reinforcement of alternative (DNRA) behaviors was effective in improving reading comprehension accuracy for three students at risk for displaying challenging behaviors to escape academic tasks. Reading comprehension was assessed using Maze assessments consists of stories with every fourth or fifth word omitted and replaced with a three-item word bank from which participants must provide the correct missing word that corresponds to an answer key. During baseline participants were given two Maze sheets to complete. During DNRA, if participants reached a predetermined criterion on the first sheet, they would not have to complete the second worksheet. Results indicated that all participants improved their Maze scores on the first sheet and could escape doing the second sheet during 89% of their DNRA sessions. Results are discussed in terms of implications for practice and areas for future research.

Keywords: Differential Negative Reinforcement of Alternative; Behaviors; Reading comprehension; Maze scores; Challenging behaviors; Individual Education Plan.

Introduction

Literacy is one of the most important academic skills students can acquire. Students abilities in literacy can be predictive of successfully living in society [1]. However, several years ago, the National Center for Education Statistics [2] determined that only about one-third of fourth-grade and eighth-grade students have acquired the skills to read at a proficient level. Two fundamental cognitive processes that are required for skilled reading are word recognition and reading comprehension [3]. Furthermore, for successful reading comprehension to take place, Hoover and Gough [4] posit that both word identification and oral language comprehension are necessary. These two skills—particularly reading comprehension are very important especially as children move from third grade into fourth grade. The reason is because the third or fourth grade curriculum makes the transition from learning to read to reading to learn [5,6].

Although students may struggle with reading comprehension at any age or grade level, this transition dramatically increases the difficulty level of the assignments, and in some cases so does the resistance to complete assignments [7]. Consequently, students struggling academically are at risk for displaying a variety of behavior problems [8]. These students who display challenging behaviors often do so to escape something unpleasant, such as a difficult assignment or to obtain some sort of positive reinforcement such as peer and/or adult attention [9,10]. Schmidt et al. [11] found specifically that some children’s problem behaviors were maintained by escape from academic task demands. For other students reading has become so unpleasant that even when they have the necessary requisite skills to complete a reading comprehension task, they nevertheless refuse to do so [12]. Without intervention, students who misbehave to escape tasks or activities lose academic engaged time, particularly if the content is reading comprehension because skills in this area are important for students to learn other aspects of the curriculum [13]. An effective intervention for students who misbehave to escape an academic task is differential negative reinforcement of alternative (DNRA) behavior [14]. In this approach, students are given a break (or shorter assignment) from a task contingent upon following directions quicker [15]. For example, a typical math assignment for a student could be two pages with 15 problems on each page. Using DNRA, the student would be told that if she reaches a certain criterion on the first page that she would not have to complete the second page. Several other researchers have conducted several studies using DNRA to decrease problem behaviors associated with escaping an aversive task [15-20].

There are two specific similarities between these six studies. First, participants typically had moderate to severe levels of intellectual disability (e.g., Down syndrome), autism, or a feeding disorder. Second, the tasks were non-academic in nature. For example, Golonka et al. [16] and Marcus and Vollmer [15] only indicated the dependent variable as being increasing compliance to “task demands” without physical prompts. Piazza et al. [17] used DNRA to decrease destructive behaviors by following instructions while Roberts et al. [18] and Vollmer et al. [20] instructed participants to stop engaging in self-injurious behaviors. Vaz et al. [19] increased directions for a boy to selffeed by taking one bite of a target food and five bites of another food. Another issue with these studies is the extreme range of participants’ ages. The youngest participant was four years old [18] while the oldest participant was 30 years of age [16]. Further, most DNRA approaches consisted of giving participants a break contingent upon engaging in appropriate behavior. This approach may work well for individuals with severe disabilities displaying destructive or self-injurious behaviors but would not work well for students with normal intelligence during classroom activities such as completing assigned work because it would reduce academic engaged time.

To date, there is only one study examining the effectiveness of DNRA for students with high incidence disabilities or those at risk during an academic task. Holtz and Daly [21] examined the effects of DNRA on the quantity of high school students’ writing. There were six male participants, two of whom received special education services, in this study: two in tenth grade, one in eleventh grade, and three in twelfth grade. Participants were given story starters and told to write about the given topic. Dependent variables were number of attempted revisions, number of correct revisions, and number of attempted unique revisions. Participants were told that if they reached or exceeded a certain concealed criterion number of revisions (number on a sealed notecard) they did not have to complete another writing task. Results showed that adding DNRA to the instruction package increased the number of attempted revisions, the amount of correct revisions made, and the number unique revisions. The present study was conducted due to the lack of DNRA studies with high incidence disabilities or those at-risk that focused on academic tasks. Therefore, the purpose of the present study was to investigate the effectiveness of DNRA as a motivation tool to increase the accuracy of students’ reading comprehension skills as assessed using the Maze technique. The present study and the Holtz and Daly [21] study are the is only ones that used escape from a second task, versus taking a break— the latter of which negatively impacts academic engaged time.

Method

A multiple baseline design across participants was used in the present study because of its several positive features. Unlike a reversal design, which can have carryover effects from the repeated introduction of intervention, multiple baseline designs do not require treatment withdrawal. Another problem with a reversal design in the present study is that once treatment was implemented and participants knew that reduced workload was again forthcoming, they may decide to work harder in subsequent baseline conditions to more quickly being exposed to the DNRA contingency. Multiple-baseline designs also allow for gradual application of the treatment, as well as allowing for easier determination of experimental control by permitting application to one behavior/participant/setting at a time. With a multiple baseline design, experimental control is demonstrated when performance changes in terms of level and/or trend with the introduction of treatment and when the data points in baseline remain stable across participants.

Participants

Three elementary-school children participated in the study. Betty (female, age 9, reading one grade level below), Max (male, age 9, reading one grade level below), and Jack (male, age 9, reading two grade levels above) were recruited from a reading tutoring center. Students were recommended to the researchers based on parent and tutor comments of disliking reading and/or reading comprehension and engaging in escape behaviors (e.g., head on desk, walking around the room). Students were excluded if they had a diagnosis of Attention Deficit Hyperactivity Disorder so that inattention would not confound treatment results. Eligible student families were then contacted to determine interest.

Setting

All sessions were conducted in the library area of a university Reading Center. This area contained one table with three chairs. Three sides of the area were designated by shelves containing books and materials for tutoring. The fourth side opened up for a hallway, which is then followed by a fourth wall containing books. Sessions were conducted privately to avoid as many distractions as possible, but parents were able to wait outside, if desired, in provided seating. All parts of the study involving participants took place in this room. Participants had scheduled sessions at separate times from one another to limit confounds as well as to limit distractions. All participants took part in two sessions each day. These sessions were separated by various activities, including reading, homework, or reading center tutoring sessions.

Measures

Screening: This study used results of prior assessments conducted by tutors in the Reading Center that had been documented in the students’ files. The information used included the child’s scores in the area of reading assessment, as well as the child’s instructional reading level as determined by a Developmental Reading Assessment. These scores were used to determine the level required in the Maze assessments.

Dependent variables: Reading comprehension accuracy was measured using a Maze comprehension assessment, created by the DIBELS curriculum. These assessments are formally called “Daze” (when created by DIBELS) and were pre-leveled to match each participant’s reading level. The researcher chose the Daze assessments because they were readily available and pre-leveled to match the child’s instructional level. They also are formatted to yield continuous scores to be graphed versus answering reading comprehension questions either in writing or verbally over passages read. These assessments were also researchbased and tested [22,23]. A sample Daze passage can be found in Figure 1. Each Daze assessment involves one grade level passage, either fiction or nonfiction, and certain words throughout the assessment are removed and replaced with a three-item word bank from which the child chooses. The omitted words vary in difficulty, from being vocabulary based, to the correct tense of a verb. The assessments varied in length from 44 to 69 opportunities to respond, depending on the reading level of each participant and can be accessed by creating a free account on the DIBELS Next website (dibels.org) and downloading the materials. The researcher measured the participants score on each assessment and used the score as the dependent variable.

General procedures: IRB approval was obtained before participant recruitment. An employee of the reading center made the first contact through a phone call to parents/guardians in order to notify parents of the potential study and determine interest. Tutors and reading center supervisors nominated students who possessed the requisite skills but hurried through the work without regard to accuracy as a way to escape the task as quickly as possible. A researcher then e-mailed the three families who indicated interest to set up a meeting time for consent and assent. Families gave signed consent through individual meetings with one of the researchers. Two participants provided assent in the family meeting. The third participant provided assent prior to the first session. To encourage participants to remain in the study, a noncontingent reinforcer was provided. Each child would be able to pick a prize from a prize box at the start of each session as a reinforcer for coming to the sessions. By providing this reinforcer at the beginning of the session, the reinforcer for participation would not confound subsequent performance on Daze assessments.

Baseline: During baseline, participants were given two Daze assessments in succession with no exceptions per session. Participants had unlimited time on each Daze, which they read aloud. Due to the subsequent DNRA condition, only the score on the first Daze assessment was recorded and graphed. The researcher followed a script to ensure each assessment was implemented consistently every time. Participants were provided with instructions for completing each Daze assessment. They could ask questions regarding any unclear instructions but were prohibited from asking questions regarding the content of each Daze assessment. The procedure for scoring each Daze assessment was repeated for the second assessment during each session. Due to the multiple baseline design, each participant had a different length baseline. The baseline was not dependent on ability level or success rate of each participant, only based on a visual assessment of stability and when a functional relation could be visually analyzed from the data.

Differential Negative Reinforcement of Alternative (DNRA) Behavior: The DNRA condition consisted of providing participants with negative reinforcement after increasing their reading comprehension accuracy on the Daze assessments. After each participant completed baseline assessments, their mean scores were calculated. The score was then multiplied by 1.33, and the researcher determined and recorded the total as well as the four consecutive whole numbers for a total of five possible scores. For example, if a student had a mean score of 60%, the scores recorded would be 80, 81, 82, 83, 84. This approach is commonly used in changing criterion designs by multiplying baseline average by 1.3 to 1.5 [24].

When participants arrived for their first session in the intervention phase, assessment instructions were given to them once again, but this time they were informed that they had the ability to escape the second Daze assessment if they increased their score on the first assessment. Participants were reminded of their average score on baseline data as well as telling them that a higher score on the first assessment would be necessary to escape the second one. Immediately after a participant completed the first Daze assessment, the score was calculated (correct answers divided by total answers) and score announced to the participant. The participant then drew a number from the bowl and checked if his or her score was at least as high as that number. This approach was to ensure indistinguishable contingencies for performance. If the score met the criterion, the session was terminated. If the score on the first assessment did not meet the criterion, then the participant received the second assessment and the procedures repeated. As in the baseline condition, if a participant could not escape the second Daze assessment, only the score on the first one was recorded and graphed.

Interobserver Agreement (IOA)

The primary researcher conducted all sessions. She provided a graduate assistant (GA) with an identical answer key, and the GA scored each assessment independently from the primary researcher. During these sessions, IOA was 100%. For each response in which there was an agreement, the primary researcher awarded one point. For each response in which there was a disagreement, 0 points were awarded. The number of points awarded were divided by total possible points to determine IOA.

Fidelity

The primary researcher created checklists to assess implementation fidelity during baseline and intervention phases. A graduate assistant checked fidelity in 6 of 18 sessions (sessions 1, 4, 7, 10, 14, and 18) so that both phases were observed multiple times for each participant. Fidelity for all sessions across all participants was 100%.

Data Analysis

Data were analyzed primarily through visual inspection of graphed trends, variability, and immediacy of changes in trends from one phase to another. The primary researcher decided that a minimum of five baseline sessions must be completed before intervention was introduced, provided a stable trend. A secondary analysis was conducted by computing effect sizes using improvement rate difference (IRD) and Tau-U. IRD was computed because it provides an effect size similar to the risk difference used in medical treatment research, which has a proven track record in hundreds of studies [25]. Tau-U values were computed because it controls for monotonic trend (i.e., increasing trends during baseline). The IRD and Tau-U effect sizes were calculated using the www.singlecaseresearch.org/calculators.

Results

Results from the comprehension accuracy data collected are displayed in Figure 2. Visual inspection of this graph reveals all participants improved their scores from baseline to intervention. Across all three participants, DNRA was earned 25 out of 28 intervention sessions, for a total success rate of 89.3%. Mean scores with ranges for each participant in baseline and intervention as well as effect sizes appear in Table 1.

Jack

As the first participant, Jack had the shortest baseline of five sessions. During baseline, Jack scored an average of 71.3% on the Daze assessments with no outliers, creating a very stable baseline. During the intervention phase, Jack’s score was higher than the score on the criterion drawn and DNRA was earned in 13 out of 13 sessions, for a total of 100%. Jack increased his average comprehension scores to 97.4% and his intervention phase data showed a slight ascending trend.

Max

During his nine baseline sessions, Max scored an average of 45.7% on his Daze assessments with outliers of 68.8% and 28.6% in his first two sessions, respectively. Baseline data indicated a descending trend. He earned the DNRA during the intervention phase for eight out of nine sessions, or 88.9% of the sessions that showed a slight ascending trend. Max increased his average percentage to 67.1%.

Betty

Betty had the longest baseline and scored an average of 46.6% on Daze assessments across her 12 sessions. She was able to escape the second assessment 66.7% of the time during her six DNRA intervention sessions. Betty also improved her overall comprehension accuracy from 46.6% to 60.3% over the entirety of the study. Betty’s data remained relatively stable during baseline, and her two outliers occurred in consecutive intervention sessions. During session 15, Betty was distracted by a preferred object she had brought from home. When this item was removed, Betty became upset and displayed inappropriate behaviors such as throwing items, ripping papers, and scribbling over the assessments. Session 16 took place following a break, and Betty re-escalated when work was presented to her, leading to two outlier scores in her intervention phase. Even with the two outlier scores, Betty’s overall trend in the intervention phase was ascending.

Discussion

The purpose of the present study was to determine if a relatively new approach to DNRA would increase reading comprehension accuracy. Results of the study indicated that, overall, all three participants improved their comprehension accuracy on Daze passages when DNRA was implemented. Although there was some variability in the data, visual inspection and effect sizes still indicated substantial improvements. This study extends previous research on DNRA to use with reading comprehension for children with high incidence disabilities or those at risk. Previous research focused primarily on children with autism or moderate to severe intellectual disabilities and typically gave participants a break from a task for improved performance. Current results also represented one of the few DNRA approaches of removing a second assignment contingent on an improved score on the first one rather than just giving a child a break but then having to return to the task. Finally, the current study was the only one to date to use DNRA to motivate escape-prone children to improve reading comprehension scores. Results will be discussed in terms of their relation to previous research, implications for practice, and areas for future research.

Extending the DNRA Research

Previous DNRA research used the procedure of offering children a break part way through a task contingent on improved performance but then requiring them to return to complete the activity [15-17]. This approach meets the function of the behavior but subsequently requires the student returns to the aversive activity after a break, which may reintroduce inappropriate escape behaviors. The DNRA approach used in the present study was similar to the one used by Holtz and Daly [25] and Warzak et al. [14] that offered participants the opportunity to terminate a session by achieving a certain designated level of success on the first half of the activity or task, but there were several differences. First, Holtz and Daly focused on improving quality of writing assignments for six high school students with DNRA as part of a larger instructional package. Warzak and his colleagues tried to improve reading accuracy of a 10-year-old boy with functional alexia (visual disorder). The boy enjoyed reading but complained that the letters were “blurry” and “moving back and forth.” The dependent variable was ten lists each of which consisted of 10 four-letter words and the study took place in a hospital. Baseline sessions were 15 to 20 minutes long while treatment sessions lasted between 45 minutes and two hours. These conditions were clearly different from those in the present participants presenting escape-driven behaviors.

Implications for Practice

By nature, schools place demands and expectations on students. For students with behavior problems or those at risk, those demands may be perceived as aversive in some way, leading to misbehavior as a way to escape the academic tasks/ activities. In terms of students with disabilities, the Individual with Disabilities Education Act (IDEA) requires the Individual Education Plan (IEP) to include a behavior intervention plan (BIP) for students displaying challenging behaviors, regardless of the disability [8]. Therefore, by logical extrapolation, any students who display inappropriate behaviors severe enough to interfere with learning should have at least a tier 2, but several researchers demonstrated greater reductions in challenging behaviors during function-based versus non-function-based interventions [9,26,27]. Perhaps the most important implication for teachers using DNRA is that it is a simple and quick intervention to improve escape-minded students’ scores on worksheet-type assignments. Further, it takes an unpleasant task for the student and makes it a little more novel by including the indistinguishable contingency. Students get to pull one piece of folded paper out of many out of a bag and read the number to see if they “won” and get out of completing the remainder of the assignment or task. Hence, DNRA is a high social validity [28] intervention because it is easy to learn and implement, takes little time and effort, does not interrupt the flow of class activities, and does not require intensive levels of teacher consultation. DNRA is the type of intervention that led to the Institute of Educational Sciences [29] forming a group to determine how evidence-based practices could be translated into simple and vibrant techniques that would lead to better behavioral outcomes for all students. Mooney and Ryan [30] described interventions for practitioners meeting these goals as “low-intensity” because they are simple, clear, and easy to implement—those with high social validity.

Areas for Future Research

One area for future research would be to extend the current DNRA approach to additional academic content and lessons to improve performance. Research should also examine the effectiveness of the DNRA to reduce any socially inappropriate escape behaviors. Further, research should examine the level with which this intervention can be faded while still maintaining improved academic performance and appropriate behaviors. Perhaps adding a differential reinforcement of alternative behavior (DRA) for also positively reinforcing appropriate behaviors would help fade the DNRA component, but this suggestion requires additional research to corroborate. Finally, participants in the present study were fourth graders, all coming from a low-middle socioeconomic background and struggled in at least one area of reading (mainly fluency). Future research could focus on different populations. While reading comprehension was the content medium in which the effectiveness of the intervention was tested, a similar intervention technique could be replicated in other content or social areas.

Conclusion

The present study demonstrated that a brief, quick, and easy intervention can increase the reading comprehension for students who displayed escape-function inappropriate behaviors. All participants improved their reading comprehension scores on Daze assessments. This is the first study to use DNRA for reading comprehension while Holtz and Daly [25] used it to increase quality of writing on essay tasks. Consequently, the use of DNRA— having shown success in the area of reading comprehension and writing—should hold promise for use in other content areas such as spelling, math, and reading fluency. Finally, DNRA represents a low-intensity intervention which was not previously covered by the Institute of Educational Sciences.

References

1. Rinaldi L, Sells D, McLaughlin TF (1997) The effects of reading racetracks on the sight word acquisition and fluency of elementary students. Journal of Behavioral Education 7(2): 219-
2. National Center for Education Statistics (2015) National assessment of educational progress.
3. Spector JE (2010) Sight word instruction for students with autism: An evaluation of the evidence base. Journal of Autism and Developmental Disorders 41(10): 1411–
4. Hoover WA, Gough PB (1990) The simple view of reading. Reading and Writing 2(2): 127-
5. Harlaar N, Dale PS, Plomin R (2007) From learning to read to reading to learn: Substantial and stable genetic influence. Child Development 78(1); 116-131.
6. Tong F, Irby BJ, Lara-Alecio R, Koch J (2014) Integrating literacy and science for English language learners: From learning-to-read to reading-to-learn. Journal of Educational Research 107(5): 410–
7. Wehby JH, Hollahan MS (2000) Effects of high-probability requests on the latency to initiate academic tasks. Journal of Applied Behavior Analysis 33(2): 259-
8. Kauffman JM, Landrum TJ (2008) Characteristics of emotional and behavioral disorders of children and youth (9^th edtn) Pearson.
9. Ingram K, Lewis Palmer T, Sugai G (2005) Function-based intervention planning: Comparing the effectiveness of FBA function-based and non-function-based intervention plans. Journal of Positive Behavior Interventions 7(4): 224-
10. Payne LD, Scott TM, Conroy M (2007) School-based examination of the efficacy of function-based intervention. Behavioral Disorders 32: 158-174.
11. Schmidt JD, Shanholtzer A, Mezhoudi N, Scherbak B, Kahng S (2014) The utility of a brief experimental analysis for problem behavior maintained by escape from demands. Education & Treatment of Children 37(2): 229–
12. Learned JE (2016) “The behavior kids”: Examining the conflation of youth reading difficulty and behavior problem positioning among school institutional contexts. American Educational Research Journal. 53(5): 1271-
13. Peckman T, McCalla G (2012) Mining student behavior patterns in reading comprehension tasks [Paper presentation]. The 5th  International Conference on Education Data Mining (EDM), Chania, Greece.
14. Warzak WJ, Kewman DG, Steffans V, Johnson E (1987) Behavioral rehabilitation of functional alexia. Journal of Behavior Therapy and Experimental Psychiatry 18: 171–
15. Marcus BA, Vollmer TR (1995) Effects of differential negative reinforcement on disruption and compliance. Journal of Applied Behavior Analysis 28(2): 229–
16. Golonka Z, Wacker D, Berg W, Derby KM, Harding J, Peck S (2000) Effects of escape to alone versus escape to enriched environments on adaptive and aberrant behavior. Journal of Applied Behavior Analysis 33(2): 243–
17. Piazza CC, Moss DR, Fisher WW (1996) Differential reinforcement of alternative behavior and demand fading in the treatment of escape maintained destructive behavior. Journal of Applied Behavior Analysis 29(4): 569–
18. Roberts ML, Mace EC, Daggett JA (1995) Preliminary comparison of two negative reinforcement schedules to reduce self-injury. Journal of Applied Behavior Analysis 28: 579-580.
19. Vaz PCM, Volkert VM, Piazza CC (2011) Using negative reinforcement to increase self-feeding in a child with food selectivity. Journal of Applied Behavior Analysis 44: 915-920.
20. Vollmer TR, Roane HS, Ringdahl JE, Marcus BA (1999) Evaluating treatment challenges with differential reinforcement of alternative behavior. Journal of Applied Behavior Analysis 32: 9-23
21. Good RH (III) & Kaminski, R (2011) Dibels next assessment manual. Sopris West, Dallas, Australia.
22. Center on Teaching and Learning (2012) 2012 - 2013 DIBELS Data System Update Part I: DIBELS Next Composite Score(Technical Brief No. 1202) University of Oregon, Eugene, Oregon, United States.
23. Maag JW (2018) Behavior management: From theoretical implications to practical applications (3^rd) Cengage Learning.
24. Parker RI, Vannest KJ, Brown L (2009) The improvement rate difference for single-case research. Exceptional Children 75(2): 135-150.
25. Holtz JW, Daly EJ III (2019) An evaluation of an instructional and motivational treatment package on writing revisions.Contemporary School Psychol Advance online publication.
26. Ellingson SA, Miltenberger RG, Stricker J, Galensky TL, Garlinghouse (2000) Functional assessment and intervention for challenging behaviors in the classroom by general classroom teachers. Journal of Positive Behavior Interventions 2: 85-97.
27. Filter KJ, Horner RH (2009) Function-based academic interventions for problem behavior. Education and Treatment of Children 32: 1-19.
28. Wolf MM (1978) Social validity: The case for subjective measurements, or how behavior analysis is finding its heart. Journal of Applied Behavior Analysis 11: 203-214.
29. Institute of Education Sciences (2014) Practitioner perspectives on emerging research needs and improving relevance in education research—Technical working group summary.
30. Mooney P, Ryan JB (2018) Supporting professionals in becoming skilled classroom managers of behavior. Beyond Behavior 27: 127.