Study Finds Student Growth Percentile Is Unreliable, Limits Access to Charter Public Schools

Share on Facebook
Share on Twitter
Share on

High degree of error, especially in small school districts, leads to small districts replacing larger ones in bottom 10 percent

BOSTON – The Student Growth Percentile (SGP) that the Massachusetts Department of Elementary and Secondary Education (DESE) now uses as 25 percent of the formula for determining school district rankings has a high degree of error. While the SGP may have a role to play as part of discussions around holding districts accountable for performance, it should not be used for high-stakes policy decisions, including which districts are eligible for an increase in the charter public school cap, according to a new study published by Pioneer Institute.

A 2010 state law requires DESE to rank school district performance.  Districts in the bottom 10 percent statewide are subject to an increase in the amount of tuition that can be paid to charter schools from 9 percent to 18 percent of net school spending (NSS).  Prior to 2015, rankings were based solely on the percentage of students scoring proficient on MCAS ELA, math and science tests.

“Using absolute proficiency as the only measure to determine eligibility for an increase in the charter school cap better aligns with the 2010 law,” said Jamie Gass, director of PioneerEducation.  “The law’s intent was to provide students and families in struggling districts with different public school options.”

SGP compares changes in an individual’s MCAS score to those of students who scored similarly in previous years.

But SGP scores suffer from wide margins of error.  Several recent studies report that a 95 percent confidence interval for the scores is a spread of 50 points.

“If one school received an SGP of 30 while another got a score of 70, researchers couldn’t be confident that the school with the higher score actually helped its students progress more than the one with the lower score,” said Dr. Cara Candal, who co-authored “An Analysis of How Massachusetts’ ‘Student Growth’ Model Limits Access to Charter Public Schools” with Dr. Bruce Bean and Rebekah Paxton.

When district enrollment is higher, SGP scores cluster around a “typical mean,” which may not reflect the diversity of real SGP scores within a district. In districts where enrollment is comparatively low, SGP is more volatile. As a result of the spurious relationship between SGP and district enrollment, some low-performing large districts where charter demand is high are exiting the bottom 10 percent and being replaced by higher performing small districts.

[ytp_video source=”5hLm2HUBppc”]

In the 2018-19 school year, Lynn moved out of the bottom 10 percent due to an increase in its SGP score, even though it remained solidly in the lowest 10 percent in terms of absolute proficiency.  The change resulted in the charter cap falling from 18 percent to 9 percent of NSS, and meant none of the 1,920 unique students on charter waitlists would be able to gain access to charter schools, which generally outperform the district.

In 2018, one charter school application from Lynn “substantially met the criteria for approval,” according to then-Acting Commissioner of Elementary and Secondary Education Jeff Wulfson.  But he was unable to recommend the school for approval by the Board of Elementary and Secondary Education (BESE) “because of upcoming changes in the NSS cap for Lynn.”

This increased volatility among districts entering and exiting the bottom 10 percent, and therefore BESE’s ability to authorize more charters, discourages operators from applying for a charter in high-demand areas that are close to exiting the lowest 10 percent.

Three quarters of the school districts entering the lowest 10 percent under SGP have enrollments below 2,500.  They include districts like Hawlemont, Easthampton, Greenfield, Palmer, and Leicester that are among the smallest in Massachusetts.  The result is an overall reduction in the number of available charter school seats.

About the Authors

Dr. Cara Stillings Candal is Director of Educational Opportunity at the Foundation for Excellence in Education and a senior fellow at Pioneer Institute. She was formerly research assistant professor and lecturer at the Boston University School of Education and a founding team member of the National Academy of Advanced Teacher Education. Candal holds a B.A. in English literature from Indiana University at Bloomington, an M.A. in social science from the University of Chicago, and a doctorate in education policy from Boston University. She is the author/editor of several books, including The Fight for the Best Charter Public Schools in the Nation.

Dr. Bruce Bean has been involved in various forms of data analysis for over forty years, creating custom software for many uses. Working as an accountant, he created a complete custom software general ledger package and other software programs. As the founder of his own economic statistics software company, he created custom software that contained over 14,000 economic and financial time series and was used nationwide. For the past twenty-one years, he has been the data manager of The Community Group in Lawrence, MA, where he has been involved in analyses of DESE and local school data both for in-house and consulting.  He earned a Ph.D. in Physics from the University of New Hampshire.

Rebekah Paxton is a Research Assistant at Pioneer Institute. She first joined Pioneer in 2017 as a Roger Perry intern, writing about various transparency issues within the Commonwealth, including fiscal policy and higher education. Since then, she has worked on research projects under PioneerPublic and PioneerOpportunity, in areas of state finance, public policy, and labor relations. She recently earned an M.A. in Political Science and a B.A. in Political Science and Economics, from Boston University, where she graduated summa cum laude. 

About Pioneer

Pioneer Institute is an independent, non-partisan, privately funded research organization that seeks to improve the quality of life in Massachusetts through civic discourse and intellectually rigorous, data-driven public policy solutions based on free market principles, individual liberty and responsibility, and the ideal of effective, limited and accountable government.

Get Updates on Our Charter Schools Initiative

Related Posts

A Conversation about Massachusetts Charter Schools: Retention Rate and Age Trends Among Public Educators

Massachusetts was home to 400 school districts in the 2020-2021…

Nina Rees on the 30th Anniversary of Charter Public Schools in America

This week on “The Learning Curve," Gerard and Cara celebrate the 30th anniversary of charter schools with Nina Rees, President and Chief Executive Officer of the National Alliance for Public Charter Schools.

The Washington Post’s Jay Mathews on An Optimist’s Guide to American Public Education

This week on “The Learning Curve," Gerard and Cara talk with Jay Mathews, an education columnist for The Washington Post and author of the recent book, An Optimist's Guide to American Public Education. Jay describes the three key trends in K-12 schooling that he views as cause for hope.

Best-Selling, Netflix Author Loung Ung On Surviving Pol Pot’s Killing Fields

This week on “The Learning Curve," Gerard and Cara talk with Loung Ung, a human-rights activist; the author of the bestselling books First They Killed My Father: A Daughter of Cambodia Remembers, Lucky Child, and Lulu in the Sky; and a co-screenwriter of the 2017 Netflix Original Movie, First They Killed My Father. Ms. Ung shares her experiences living through genocide under Pol Pot and the Khmer Rouge from 1975 to 1979, which resulted in the deaths of nearly a quarter of Cambodia's population. 

UGA Prof. Valerie Boyd on Zora Neale Hurston, the Harlem Renaissance, & Black History Month

This week on “The Learning Curve," Cara and Gerard celebrate Black History Month with Professor Valerie Boyd, the Charlayne Hunter-Gault Distinguished Writer in Residence and Associate Professor of Journalism at the University of Georgia, and the definitive biographer of Zora Neale Hurston. Boyd discusses why Hurston is such an important novelist and cultural figure, and the influence of Hurston’s 1937 classic novel, Their Eyes Were Watching God, on American literature.

Eva Moskowitz of Success Academy on Charter Schools, Achievement Gaps, & COVID-19 Learning Loss

This week on “The Learning Curve," Cara and Gerard kick off the new year with Eva Moskowitz, CEO & Founder of Success Academy Charter Schools, a network of 47 schools enrolling 20,000 K-12 students in New York City. Eva shares her own education path, and how it influences her leadership and philosophy.

SABIS® President Carl Bistany on International Education, Charter Public Schools, & At-Risk Students

This week on “The Learning Curve,” Cara and Gerard are joined by Carl Bistany, the president of SABIS® Educational Systems, an education company founded over 130 years ago that serves young women in the Middle East, and poor and minority students in the U.S.

Wall Street Journal Columnist Jason Riley on the 2020 Election, School Choice, & Race in America

This week on “The Learning Curve,” Cara and Gerard are joined by Jason Riley, a senior fellow at the Manhattan Institute and columnist for The Wall Street Journal. Jason shares insights on the 2020 election, its implications for the next two years, and assuming Vice President Biden becomes president, how he may govern on K-12 education.

Harvard PEPG’s Prof. Paul Peterson on Charter Schools, Digital Learning, & Ed Next Polling

This week on “The Learning Curve,” Cara and Gerard are joined by Paul Peterson, the Henry Lee Shattuck Professor of Government and Director of the Program on Education Policy and Governance at Harvard University.

President of D.C.’s AppleTree Institute, Jack McCarthy on Charter Schools and Fall Reopening

This week on “The Learning Curve,” Cara and Gerard are joined by Jack McCarthy, president and CEO of AppleTree Institute for Education Innovation and board chair of AppleTree Early Learning Public Charter School. Jack shares what animated him to establish this highly innovative early childhood charter public school network that serves the most vulnerable children in Washington, D.C.

CREDO’s Macke Raymond on Charter Schools’ Quality & Growth

CREDO's Margaret “Macke” Raymond joins "The Learning Curve" this week to discuss charter school performance; the types of charters that are succeeding consistently and replicating; and the formula for quality both in instruction and policymaking.

Lance Izumi on How Charters Are Meeting Diverse Learning Needs

Happy New Year! This week on "The Learning Curve," Cara and Bob talk with Lance Izumi, Senior Director of the Center for Education at the Pacific Research Institute, about his new book, Choosing Diversity.

Dr. Howard Fuller on School Choice & Presidential Politics

Cara and Bob talk withthe the great Dr. Howard Fuller, Distinguished Professor of Education, about his passionate activism on behalf of education reform, his concerns about the lack of support among Democratic presidential candidates for charter schools & more!

Study Finds Student Growth Percentile Is Unreliable, Limits Access to Charter Public Schools

High degree of error, especially in small school districts, leads…

Pioneer Public Statement on Legislative Demise of New Bedford Charter School Deal

This past week, in a display of the state’s teacher unions'…