Using Data
| With OSEP support, researchers are investigating how data-based decision making enhances results when developing schoolwide positive behavioral support systems, using high stakes assessment scores for school improvement, and integrating curriculum-based measurement into the instructional program. DATA INFORM SCHOOLWIDE POSITIVE BEHAVIOR SUPPORT PROGRAMS "Data-based decision making is critical to addressing schoolwide discipline and safety issues effectively," says George Sugai. "To sustain the use of a systematic approach to positive behavioral support, procedures must be in place to enable informed and accurate decisions about whether adequate gains are being achieved and what actions should occur next." In effective systems of behavior support, student behavior is monitored continuously, and data are used by staff as a basis for decisions. For example, Sugai and his colleagues have developed a system that uses scientifically derived information about office referrals along with other data to evaluate the effectiveness of a comprehensive schoolwide discipline and violence prevention program. Sugai's work is one of the first attempts to link a systematic analysis of office discipline referral data to inform discipline program reform efforts. "Research suggests that office referrals provide a useful index to assess school discipline needs and monitor intervention effects," Sugai points out. "We found that school-based teams using office referral data have successfully established and maintained schoolwide discipline systems that resulted in a 50 to 60 percent reduction in rates of office discipline referrals." Sugai goes on to say that schools should use data to identify their specific needs and to determine if selected programs and interventions match their needs. When creating a data-based decision making system, Sugai suggests that educators consider the guidelines found in the sidebar, Guidelines for Creating a Data-Based Decision Making System. Flossmoor Elementary School District 161, IL, has implemented positive behavior support programs districtwide and uses systematic data to target their efforts. "Teams in our district regularly review student data," Judith Green, assistant superintendent, tells us. "By using data, we know what to target, and how and when to intervene." Since implementing the approach, the district has seen a significant improvement in behavior. Teachers complete referral forms when students violate discipline guidelines. A secretary enters the information into a database. Teams in each school meet monthly to review the data and make decisions. To assist their efforts, each school has access to the School-Wide Information System (SWIS) [http://www.SWIS.org], a web-based information system.
"Having a database with information is helpful in focusing attention on where and when the problem is occurring," Green points out. "For example, after reviewing the data from the elementary schools, we found that many infractions were happening during recess. This enabled us to target that setting for intervention." Statewide Positive Behavioral Support Initiative Features Data-based Decision Making Hawaii has launched a statewide initiative to prepare all schools to use Sugai's positive behavioral support model. According to Jean Nakasato, educational specialist in the Hawaii Department of Education, a major element of the positive behavioral support system is data-based decision making. "Our goal is for everyone to be data smart, to be able to analyze and interpret behavioral data," Nakasato tells us. "We need to know if we are meeting student needs and data help us do this." To this end, the use of data is built into the training process. School-based teams bring data (e.g., about office referrals) to the state-sponsored training sessions where they learn how to use it. In teams, participants use the data to self-assess and problem solve. The process results in action plans that participants are expected to implement in their schools.
USING LARGE-SCALE ASSESSMENT DATA FOR SCHOOL IMPROVEMENT "The benefits of having students with disabilities participate in state and districtwide assessments will never be fully achieved if educators do not actually look at, explore, and use the data for making decisions about educational programs," says Martha Thurlow, director of the OSEP-funded National Center on Educational Outcomes (NCEO). "Those schools, districts, and states making the most progress are going to be the ones that thoroughly and carefully use their assessment results." Since 1998, Thurlow and her colleagues have been studying how states and localities are implementing the IDEA requirement that students with disabilities participate in state- and districtwide assessments, with accommodations as necessary. Of particular interest is the IDEA provision requiring the public reporting of scores of students with disabilities with the same frequency as for other students.
"Reporting the scores of students with disabilities is one element of a truly inclusive accountability system" Thurlow says. "Once scores are reported, they should be used to make programmatic and instructional decisions." To understand and use state and/or district assessment data, Thurlow suggests that educators first reflect on why it is important to look at data and how knowing such information may prove relevant for evaluating whether decision making is producing the expected results. From here, educators can decide where they might want to start their efforts and how far into the data they want to dig. According to Thurlow, this step involves knowing the data elements that are used to generate district or state reports. "A thorough understanding of the information provided also enables us get more out of the data that are given," Thurlow adds. "It allows us to explore trends in performance, for example, changes in test scores across grades within a given year, changes in the scores of students within specific grades across years, and performance of the same students as they progress across grades. It also is important to keep track of mobility in and out of special education, and to look at data in varied ways," Thurlow says. "This is especially true when trying to reach conclusions about improvements in the performance of students with disabilities."
A District Looks at Statewide Data for School Improvement Judy Elliott, assistant superintendent in the Long Beach Unified School District, California, has worked with Thurlow for a number of years. Her district looks closely at statewide data when making school improvement policy decisions. "Our district's standards are aligned with the state assessment so we are very interested in our students' results," Elliott reports. Every principal in the district receives a schoolwide profile of trend lines, participation rates, and test scores. The profile shows parallel data for students with disabilities. "Data help us determine where students are making gains and where they need additional support." According to Elliott, at first there was some concern around getting data for students who were not typically counted. IEPs were revamped to include information about the particular assessment accommodations that were being used during instruction and classroom testing. "We used this information to help us plan the best use of accommodations on the statewide assessment," Elliott explains. To support principals in using the data, the district paid attention to information dissemination and relationship building. "We made a conscious effort to help everyone understand that the reporting requirements in IDEA and NCLB were intended to help all children achieve better results," Elliott adds. CURRICULUM-BASED MEASUREMENT Fact: When teachers use curriculum-based measurement (CBM)-a form of systematic progress monitoring-to track their students' progress in reading, mathematics, or spelling, they are better able to identify students in need of additional or different forms of instruction, they design stronger instructional programs, and their students make greater gains. Fact: More than 200 empirical studies published in peer-review journals provide evidence of CBM's reliability and validity for assessing the development of competence in reading, spelling, and mathematics and document CBM's capacity to help teachers improve student outcomes. "CBM is a powerful tool that can help teachers monitor their students' academic progress and design more effective instructional programs," says Lynn Fuchs, researcher at Vanderbilt University. "Using CBM, teachers can infuse ready-made accountability into their instructional design and can streamline the individualized education program (IEP) system to make the IEP into a living document that guides and enhances instructional decision making on a day-to-day basis." Each CBM test assesses all the different skills covered in the annual curriculum. CBM samples the many curricular skills in such a way that each weekly test is an alternate form. Scores earned at different times during the school year can be compared to determine whether a student's competence is increasing. "The overall CBM score can be used in three ways," Fuchs explains. "Teachers use CBM scores in universal screening to identify students in need of additional or different forms of instruction, to monitor students' development of academic competence, and to improve instructional programs." Fuchs adds that CBM skills profiles also are used to identify the skills in the annual curriculum in which students require additional instruction and to identify the students who are experiencing problems with maintaining skills after initial mastery was demonstrated.
Helping teachers use CBM also has been a focus of research for Lynn and fellow researcher Douglas Fuchs. Their work has addressed how CBM users can keep up with the mechanical tasks of measurement. "Technology can dramatically reduce the need for teachers to conduct the mechanical tasks associated with measurement, such as test administration, test scoring, graphing, and data analysis," Fuchs tells us. "Data collection software used in combination with data management software can entirely eliminate most teachers' time in such tasks." [For information on the use of technology, see the sidebar, Computer Administered and Scored CBM Resources.] Although CBM has been commonplace in special education classrooms for years, it was not until the early 1990s that the Fuchses and their colleagues began integrating CBM into general education classrooms. "General education teachers liked CBM," Fuchs reports, "however, we needed to refocus the attention on classwide reports and skill summaries rather than individual student graphs." In addition, Fuchs found that many teachers benefited more when they received assistance in using the CBM data for decision making. "Even though analyzing the data is helpful, in order to intervene, most teachers prefer to be given new routine ways to differentiate instruction." To help teachers break beyond their standard instructional routines and identify alternative teaching procedures, Fuchs and her colleagues have developed full-scale instructional approaches, as well as expert systems that provide instructional modifications. District Uses CBM Data To Improve Student Achievement We often think about CBM as a classroom intervention. But, its use school- and districtwide is growing. The Pittsfield (MA) Public Schools are a good example. In 1997, Michael Meyers, the special education director, was introduced to Mark Shinn, University of Oregon researcher who has received OSEP support for his work in CBM over the years. Shinn's model utilizes school psychologists in key roles in collecting and analyzing the CBM data. Meyers used Shinn's work as a jumping off point to implement CBM throughout the district. "What started as a special education initiative to identify students needing help with reading has now expanded to include every child K-3 and a plan to include all students through Grade 8," Meyers reports. CBM data are used to verify what is working and not working, and to help guide educators in selecting an appropriate curriculum. "Students are assessed every fall, winter, and spring," Meyers says. "With this monitoring, we are able to identify which students are not meeting benchmarks. With CBM we make sure every child is making progress-and if not, we intervene." Initially, school psychologists were trained in CBM, but now, all special education teachers also are trained. One of the challenges Meyers and his colleagues faced was how to manage the data. "We found that once we went with a web-based data management program, it freed the psychologists to spend more time with teachers-helping them interpret the data and identify interventions." |
Next: Views From the States
Back
to this Issue's Contents ERIC
Clearinghouse on Disabilities and Gifted EducationERIC/OSEP Special Project Page CEC Home Page