Shut bad schools for low performance, but don’t draw conclusions from test scores alone

(Guest Post by Michael J. Petrilli)

Editor’s note: This post is the second in an ongoing discussion between Fordham’s Michael Petrilli and the University of Arkansas’s Jay Greene that seeks to answer this question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results? The first entry [by Jay] can be found here.

The prompt for this forum promised that we would explore “areas of agreement and disagreement.” I’m pleased, Jay (and not altogether surprised), to see that we share a lot of common ground. Let me start with that, then save what I see as our major dispute (what we can learn from reading and math scores) for another post.

I’m thrilled that you dismissed the extreme position of some libertarians, who argue that society should never override the choices of parents. You write:

I…do not mean to suggest that policy makers should never close a school or shutter a program in the face of parental demand. I’m just arguing that it should take a lot more than “bad” test scores to do that.

I agree entirely, and on both counts. First let me explain why “we” should, on rare occasions, close a school that some parents prefer. And second, let me discuss what else beyond “bad” test scores we might consider when doing so.

You and others have heard me argue ad nauseam that because education is both a public and a private good, it’s only fair that both parties to the deal have a say in whether that good is, well, good enough. We both abhor a system whereby a district monopoly assigns children to schools and parents must accept whatever is handed to them. But the flip side is that we should also reject chronically low-performing schools—those that don’t prepare their young charges for success academically or otherwise—and deem them undeserving of taxpayer support. They aren’t fulfilling their public duty to help create a well-educated and self-sufficient citizenry, which is what taxpayers are giving them money to do.

Furthermore, there are real financial and political costs to letting bad schools—including schools of choice—fester. We see this in many cities of the industrial Midwest—Detroit, Cleveland, and Dayton come to mind—where too many schools are chasing too few students. Perhaps the marketplace forces of “creative destruction” will eventually take hold and the weakest schools will disappear, allowing the remaining ones sufficient enrollment to ensure their financial sustainability and a higher level of program quality. But that process is taking an awfully long time, particularly when we’re talking about disadvantaged children who have no time to waste. The charter sectors in these cities would be stronger—academically, financially, and politically—if authorizers stepped in to close the worst schools. But some libertarians see that as paternalistic government intrusion. I think they are misguided; I hope that you agree.

Now to your second point, that “it should take a lot more than ‘bad’ test scores” to “close a school or shutter a program in the face of parental demand.” Hear, hear! This is the genius of effective charter school authorizers that look at a school’s big picture as well as its scores. Fordham’s Dayton office strives hard (and with fair success) to be that kind of authorizer. We certainly look at test scores—especially individual student progress over time, a.k.a. “value added.” But we also examine lots of other indicators of school quality, operational efficiency, and financial sustainability. (See our current accountability framework in the appendix here.) And most importantly, we know our schools intimately. We attend their board meetings, conduct site visits frequently, and get up close and personal.

So when we consider the painful step of closing a school (which we’ve had to do a handful of times), we’re hardly just sitting in our offices “looking at spreadsheets of test scores.” The same goes for other leading authorizers nationwide.

Not that it’s easy to identify measures beyond reading and math scores that are valid and reliable indicators of school success. I share your enthusiasm for character education, non-cognitive skills, high school graduation rates, and long-term outcomes such as college completion and labor market earnings. And I’d love to see states maintain regular testing in history, geography, science, and more. Whenever we can use those scores, we absolutely do. But as the early debate around the Every Student Succeeds Act illustrates, measures of character and non-cognitive skills don’t appear ready for prime time, and they may never be appropriate for high-stakes decisions. High school graduation rates, meanwhile, are perhaps the phoniest numbers in education. Long-term outcomes are just that—long-term, making them both difficult to tie to a school (especially and elementary or middle school) and not very helpful for making decisions in the here and now. And there’s no political appetite for more testing; if anything, everyone wants less. (Let me know if you disagree with my analysis.)

So where does that leave us? As far as I can tell, facing a trade-off, which is the normal state of affairs in public policy. We can either use reading and math gains as imperfect indicators of effectiveness while working to build better measures—buttressed by school visits and the like—or we can succumb to “analysis paralysis” and do nothing.

I know which one I prefer. What about you?

(Also posted at Flypaper)

Advertisements

5 Responses to Shut bad schools for low performance, but don’t draw conclusions from test scores alone

  1. sstotsky says:

    The argument seems to be about closing down schools most of whose students are failing on federal or state-mandated tests, even if the parents want these schools to stay open. Who is making the call here? And why are teachers and administrators being held accountable for students’ scores, and only them?

    Could you also discuss the right of parents and others to eliminate these tests and the standards they are based on? The attempt to exercise that right is turning into a huge uphill battle (countering Gates et al money) all over the country, and it seems to me that this is a far more important question.

    http://www.educationviews.org/informational-meeting-common-core-ballot-question-ma/

  2. Jason Bedrick says:

    Once again, Mike fails the Ideological Turing Test with this statement:

    “I’m thrilled that you dismissed the extreme position of some libertarians, who argue that society should never override the choices of parents.”

    On Twitter he clarified (as if it weren’t obvious) that he was referring to me, Neal McCluskey, Lindsey Burke, et al. And yet, nowhere has any of us made such a case. Why does Mike find it necessary to argue with straw men?

    Jay wrote:

    “I…do not mean to suggest that policy makers should never close a school or shutter a program in the face of parental demand. I’m just arguing that it should take a lot more than “bad” test scores to do that.”

    I agree wholeheartedly with that statement. Mike claims to agree with it as well, although he holds that policymakers “should, on rare occasions, close a school that some parents prefer.”

    I’m glad he thinks such closures should be “rare” though I wonder if we have the same understanding of the word.

    Anyway, Mike agrees that “not by test scores alone” should a school be closed. So what other criteria would he examine? He writes:

    “But we also examine lots of other indicators of school quality, operational efficiency, and financial sustainability.”

    He says “lots” but cites only two examples, neither of which make any sense. If the school isn’t financially sustainable, then do we really need the authorizer to shut it down? And what is it exactly about “operational efficiency” problems that affect students? Are we concerned about the “efficiency” or student outcomes? Which student outcomes? Measured how? Mike doesn’t tell us. But don’t worry, they’re “hardly just sitting in our offices ‘looking at spreadsheets of test scores.'”

    Or are they? Mike links to a handy report and directs us to see the criteria in the appendix, which starts on page 27. So what do we have?

    The “Primary Academic Indicators” include: Value added test scores, graduation rates (which Mike calls “the phoniest numbers in education”), K-3 Literacy Improvement (test scores?), and Performance vs. Local Market/Statewide Charters (test scores?).

    The “Secondary Academic Indicators” include a bunch more value added metrics (test scores), college admissions test participation rate, dual enrollment credits, industry credentials, honors diplomas awarded, AP participation rate, AP score (more test scores), IB participate rate, IB score (more tests), College/Career-Ready Assessment (another test?), and School Regularly Administers Internal Growth Assessment (more tests?).

    The metrics also include “Financial Measures of Success” and “Operations/Governance” indicators, but is that what Mike really means when talking about “bad” schools?

    FWIW, the very last metric, in the “Operations/Governance Secondary Indicators” section, is “Family Survey Results.”

    So when Mike says policymakers should close down “bad” schools based on a lot more than just test scores, what he really means is “pretty much just test scores.”

  3. Greg Forster says:

    So there are now only two questions, really:

    1) Do we want school choice programs to convert private schools into charter schools, as Mike demands? Or is there any value in private schools?

    2) How many charter authorizers actually do a good job of judging school quality – how representative is the highly idealized, romantic fantasy Mike has provided here of the way charter authorizers typically work back here on Earth Prime?

    • Jason Bedrick says:

      There are really three camps, Greg.

      1) “Education anarchists” who believe there should be absolutely no state intervention in or financing of education (including even tax credits).

      2) “Technocratic romantics” like Mike who believe bureaucrats have the necessary knowledge and wisdom to design a perfect system.

      3) “School choice realists” like us, who want to empower parents to make decisions about their own kids’ education, but are skeptical about the effectiveness of distant bureaucrats.

      Wow, this Hemisphere Fallacy game is fun!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s