Disaggregation of Data in a PBIS/RtI Framework
The use of data within a Positive Behavior Interventions and Supports (PBIS) or Response to Intervention (RtI) framework is as essential to the development and sustainability of a healthy PBIS system as air and water is to the human body. If teams are not looking for patterns and trends in their data and are, rather, plodding along and making system and practice decisions because it “seems” right, then they are perpetuating inefficiency and ineffectiveness that a response to intervention framework seeks to correct. In this article the use of precision statements, action planning based on precision statements and data sets typically disaggregated in a PBIS framework will be reviewed.
School teams, as they start to use data, typically experience a phenomenon called “paralysis by analysis”. That is, they have an abundance of data but generally are unsure where to start with the action planning. Universal (Tier I) teams are trained to look at their fidelity and outcome data to identify and plan around system needs. Generally, at a universal level, this means looking for large data patterns and then creating a precision statement that allows the team to create an action plan addressing the who, what, where, when and why. It also means considering subgroups of data.
In other words, when looking at the initial data, a school team may see that they have a significant number of Office Discipline Referrals on the playground or during passing time in a middle school. But that doesn’t tell the team much besides the location. However, if the team looks deeper into the data, they would be able to create a precision statement such as; “during transition time between third and fourth hours, we have had 35 ODR for Disrespect. Of the 35 ODR for Disrespect, 27 of those were from 7th graders with the ODR given by the 8th grade teachers monitoring that hallway. The purpose of the behavior appeared to be to get peer attention”.
In the previous example, the precision statement would allow the team to develop an action plan that would address things to do different with the students (create a cool tool and teach it to the 7th grade students and target acknowledgement system on this new skill), as well as what to do different with the staff (at next staff meeting conduct a fluency check on what disrespect is defined as, revisit what is major behavior and what is minor behavior, etc.). It also allows the team to set a measure for accountability (decrease the ODR for 7th grade disrespect in the hallway by 35% within the next two weeks).
This example is basic disaggregation of data or the creation of a precision statement. However teams area also trained to disaggregate their Big 5 (average number of referrals per day per month, by behavior, by date, by time, by location and by student) by ethnicity and disability status at a minimum. There are several reasons for this.
On a national and state level, students of color and students with disabilities tend to have much higher rates of disciplinary contact than white students, AND, when they have those disciplinary contacts the consequences tend to be more severe. (Skiba, Russ. 2006. Race is not neutral.) Generally, because schools have tended to look at big picture data, they have missed the smaller patterns that are driving the bigger trends. When considering what white students are referred for, they tend to be objective behaviors such as fighting, tobacco use, and skipping. Students of color tend to get ODR for subjective behaviors like disrespect, harassment and loitering.
One of the easiest and most startling data points comes in the calculation of “risk ratio” which is applicable for reviewing data subgroups. To calculate risk, you identify the percent of students within each ethnicity group or disability status with referral and divide their percentage by the percent of students within the control group (i.e. white ethnicity or students without disability). The closer the score is to 1.0 (the score for the control group) the more equal the risk for the group you are looking at.
For example: In your building 47% of your students enrolled as Asian students received ODR and you are trying to determine the risk of them getting ODR compared to white students who are the largest enrolled group (of whom 65% received ODR), you would divide 47% by 65% for a risk ratio of .72. In other words, in this example students enrolled by their parents as Asian have a lower risk of getting ODR (.72) when compared to students enrolled as White who have a risk ratio of 1.
To this end, if schools do not disaggregate their data they are missing the opportunity to make their systems more responsive. When teams see large data patterns at a universal level, that is, large numbers of students or referrals, it is generally an indication of SYSTEM needs, that is, things that the staff need to address.
When considering the national and state trend of African American students having higher ODR contact and knowing that the data trend nationally and in Wisconsin is for subjective behaviors, a school calculating their risk ratios and finding a higher risk for Black students would then key in on what the majority of ODR was for among the African-American students and to address it within the system planning (fluency checks of behavior definitions, increasing family engagement, staff development, etc.).
Unfortunately, without disaggregation of data, it becomes easy to schools to overlook that system need or focus solely on the student or not see the bigger trend. The use of disaggregated data is critical in making systems responsive to ALL students. It helps to identify the issues a system must address and helps the team plan their approach.