Does Mass. Exclude Low-Achievers from National Education Tests?
Last week, in reporting the national test results on how our public schools are doing, I noted that while
It is wonderful that Massachusetts has maintained its lead nationally, … [o]ur students are no longer improving at the rate they were and in fact their performance has largely flatlined.
On the scaled scores for the Commonwealth, the loss of momentum is very clear with no change on the 4th and 8th grade math scores, and a slight increase on the 4th and 8th grade reading scores (which amount to scores that are statistically unchanged).
Let me share an additional reasons to be concerned and it starts with a solid piece from Nirvi Shah of EdWeek entitled “How Many Students With Disabilities Take the NAEP?”
While many students with disabilities are included in state exams in reading, math, and other subjects, in 2005, a Government Accountability Office report found that they are more likely to be excluded from the National Assessment of Educational Progress, often called the Nation’s Report Card.
Even before the GAO report, there were studies and questions about whether students with disabilities participated in the NAEP.
We are all accustomed to the MCAS, and therefore know how students are tested. But many readers are absolutely right to ask those of us who use performance statistics from the NAEP, the Trends in Math and Science Study (TIMSS), and the OECD’s Programme for International Student Assessment (PISA): How are students selected and is there any bias against underperforming or challenged students?
Sirvi goes on
A more recent study, done at least in part in response to the GAO report, takes another look at how many students with disabilities are included in NAEP and why others are not. The report, from the National Center for Education Statistics, which administers the NAEP, notes that a “student with disabilities is assumed to be able to participate in NAEP if he or she participated in the state assessment in the selected subject and can participate with accommodations allowed by NAEP.”
But reality hasn’t matched that ideal. For example, the study notes that in the 2009 4th grade math version of NAEP, 85.4 percent of students with disabilities took the test. On the 8th grade math test, 78.5 percent of students with disabilities were tested…
The NCES found that several factors affected students’ inclusion on the NAEP, including the type of disability they have, the severity of their disability, and whether an accommodation used in a state test was allowed on the NAEP. And while the percentage of students with disabilities included on the NAEP varies from state to state, a larger inclusion rate in one state doesn’t mean that state is more inclusive than another, the report says, because students aren’t spread uniformly through the country…
It also researched how those in the states administering the NAEP decide whether students should participate, put in place a specific process to determine if a student could take NAEP tests without the accommodations they use on state tests, and improved the training for NAEP administrators and staff to clarify the criteria for including students.
No reason for alarmism, but there are some indications in the “exclusion” data that suggest that the Commonwealth has been pretty generous in excluding or granting accommodations to students with disabilities and/or English language learners. For example, in 2009 we often excluded or allowed accommodations for a higher percentage of students with disabilities and ELL students than the nation as a whole.
There’s data on 4th- and 8th-grade students and the percentage of students excluded/accommodated on the reading NAEP here and on the math NAEP here. Just as an example, the 4th-grade math test data shows the percentages for Massachusetts and the nation as follows:
• US: Excluded 2, Assessed with accommodations 11
• MA: Excluded 4, Assessed with accommodations 13
What is the justification for the higher level of exclusions and accommodations?
Drill down to cities and the picture is interesting and one that would benefit from clarity from state and district officials as to why our numbers are what they are. Here is data for 8th-grade math, representing exclusions and accommodations in Larger US Cities as a whole, in Houston and in Boston in 2009:
• US Cities: Excluded 1, Assessed with accommodations 4
• Houston: Excluded 2, Assessed with accommodations 3
• Boston: Excluded 4, Assessed with accommodations 5
So, Boston is excluding a percentage of students four times higher than a representative basket of larger US cities. And if you look at the data from 2003-2009, Boston has consistently excluded higher percentages of students. This is a conversation worth looking into. What’s the view of state leaders?
Crossposted at Boston.com’s Rock the Schoolhouse. Follow me on twitter at @jimstergios, or visit Pioneer’s website.