Teach By Design
Data Integrity
Data-based decision making
Aug 12, 2019

Consistent Process + Reliable Data = Better Results

Your school's systems work together to make sure you’re collecting the right information in the right way at the right time. When it comes to improving student outcomes, everything starts with your discipline systems.

No items found.
Apple Podcast Button

I volunteered at last year’s environmental studies commencement ceremony here at the University of Oregon. One of the department’s traditions is for each student to say what they plan to do after graduation. I listened as one by one, graduates rattled off job offers and newly acquired research funding and positions with environmental conservancies. When I graduated, I had a degree in journalism and an open room at my parent’s house. I moved home after graduation and spent the summer with my family, which included going to my brother’s soccer tournaments with his club team.

There was a kid named Cody on that team. We sat with his parents during most games. One day, as I talked about my lack of a job and my ever decreasing bank account, Cody’s mom mentioned they needed a temp in her department at the University of Oregon. I sent my resume to the business manager the next day and they hired me over the phone.

It’s been 14 years, and I’m still here…thanks to my brother playing on the same soccer team as Anne Todd’s son.

GIF of the words Believe in Destiny flashing in neon lights

Do you ever think about the random string of events that led you to where you are today? Like how you juuuuuust missed that 3-car fender bender because you couldn't find your keys so you left the house five minutes late. The way a small choice ripples to a larger impact on our lives is known as the butterfly effect. Science will tell you it's all part of chaos theory and using the theory to create order to our lives is a misuse. However, if outcomes are just a series of choices strung together, we should take a closer look at the choices we make to see if we can't create the outcomes we want.

Origin of a Referral

We're big fans of the office discipline referral (ODR) around here. An ODR, in our eyes, isn't a punishment. (It turns out, it's really hard to punish someone with a piece of paper.) An ODR tells a small story about a student's behavior that someone saw and how that person reacted to it. It's a piece of data. According to research, it's a good measure for predicting teacher perceptions, student perceptions, chronic behavior issues, and academic outcomes.[1] In nearly every study where ODRs are included as a data source, you'll find this sentence…or something close to it:

"…ensure that all school's existing referral systems (are) sufficient to support meaningful and usable referral data…"

Your various systems work together to make sure you’re collecting the right information in the right way at the right time. When it comes to improving student outcomes, that butterfly effect starts with your discipline systems. Here’s how…and we’ll throw in a strategy for intervening earlier with students needing additional support, just for good measure.

ODR infographics v4-03.png

Using Consistent Procedures…

Every school has a way of handling student problem behavior. That way boils down to four parts working together to create a strong discipline procedure:

  1. A consistent referral process
  2. Definitions for problem behaviors
  3. A referral form to document the event
  4. Proactive strategies for responding to problem behaviors

When every referring adult agrees to these four components, you can be sure your data will be consistent. Our example starts with the procedures schools use with the School-Wide Information System (SWIS). All schools using SWIS agree to use a common referral form with defined categories and labels. Each school has a procedure for when to handle behaviors in the classroom and when to send a student to the office. The referral process also defines what happens next. If nothing else, SWIS schools have a documented, consistent way to manage student behavior. Each referral can be compared reliably to every other referral collected at the school; and if the sample is large enough, we can compare each referral to all other referrals collected nationally in the SWIS database.

ODR infographics v4-04.png

Leads to Reliable Data…

In the same schools use their SWIS data to look for patterns, researchers also wanted to see what trends they could find in the national database.[2] As you might expect, there are common themes all over the place. The first that stands out is the behaviors most likely to get an elementary or a middle school student referred – fighting and defiance, respectively. Taken at face value, it's not so interesting, but keep going with me and you'll see how it fits.

In addition to research questions, PBISApps also comes out with national trend data based on the referrals entered in SWIS during the previous school year. For example, in 2017-18, the median average number of referrals written per day in elementary schools was about 1 referral for 500 students. During that same school year, about 8% of elementary schoolers, 13% of middle schoolers, and 13% of high schoolers received 2 or more referrals.

Still with me?

Students with more than two referrals typically need additional support for their behavior; these students often don't appear on anyone's radar until late in the year after they've accumulated their third, fourth, or fifth referral. If schools could intervene earlier, they'd have the ability to change the trajectory of a whole group of students' entire year for the better.

ODR infographics v4-05.png

So Teams Can Anticipate Problems…

Enter two studies – one in 2014, one in 2010 – aimed at determining how early we can predict which students will be referred to the office more than twice during the year[3],[4]. Turns out, we can predict that, and we can do it as early as October. In fact, 50% of elementary students and 71% of middle school students who ended the year with six or more referrals had at least two referrals by the end of October. (Whew, that was a mouthful.) Also, if students had at least one referral by October for one of two behaviors, their odds of accumulating more than six referrals by the end of the year went up. Can you guess which two behaviors they were?

Fighting and defiance: The most referred behaviors in elementary and middle school.

This means, you have an opportunity to break the cycle and help students build a new path.

And Improve Student Outcomes

Start by finding out which students have one or more warning flags by the end of October:

  • Two or more referrals
  • At least one referral for fighting
  • At least one referral for defiance

Save these names and check in on them again at the end of November. If any student continued to rack up referrals, rally your Tier 2 and Tier 3 teams around them and see what support they might need.

Do you see what we just did? All of these trends and patterns and predictions are all possible because of where we started: consistent procedures. The agreements you make about your discipline procedures directly affect your ability to support students to achieve better outcomes. With all our talk about using data to make decisions, we should spend a little more time talking about the systems surrounding those data – and that's just what we'll do over the next three months. Join us for our upcoming series as we explore creating efficient referral processes, defining subjective behaviors, and ensuring equitable responses to problem behaviors.

Click to Download Infographic

1. Irvin, L., Tobin, T., Sprague, J., Sugai, G., & Vincent, C. (2004). Validity of Office Discipline Referral Measures as Indices of School-Wide Behavioral Status and Effects of School-Wide Behavioral Interventions. Journal Of Positive Behavior Interventions, 6(3), 131-147. doi: 10.1177/10983007040060030201
2. Spaulding, S., Irvin, L., Horner, R., May, S., Emeldi, M., Tobin, T., & Sugai, G. (2008). Schoolwide Social-Behavioral Climate, Student Problem Behavior, and Related Administrative Decisions. Journal Of Positive Behavior Interventions, 12(2), 69-85. doi: 10.1177/1098300708329011
3. Predy, L., McIntosh, K., & Frank, J. (2014). Utility of Number and Type of Office Discipline Referrals in Predicting Chronic Problem Behavior in Middle Schools. School Psychology Review, 43(4), 472-489.
4. McIntosh, K., Frank, J., & Spaulding, S. A. (2010). Establishing Research-Based Trajectories of Office Discipline Referrals for Individual Students. School Psychology Review, 39, 380-394.

Download Transcript

Megan Cave


Megan Cave

Megan Cave is a member of the PBISApps Marketing and Communication team. She is the writer behind the user manuals, scripted video tutorials, and news articles for PBISApps. She also writes a monthly article for Teach by Design and contributes to its accompanying Expert Instruction podcast episode. Megan has completed four half marathons – three of which happened unintentionally – and in all likelihood, will run another in the future.

No items found.