Review the Charter Research, Don’t Pick the Outlier

November 2, 2011

Julian Betts and Emily Tang at the University of California at San Diego have a new systematic review of the research on charter schools.  They look at more than 30 studies that meet minimal criteria for research quality.  They find that charters have statistically significant positive effects on math and reading achievement in elementary grades and on math in middle school.  There are no significant effects for reading in middle school or for high school student achievement.  The size of the effects are modest, ranging between 2% and 6% of a standard deviation.  (See Table 2)

It’s important to step back and review an entire literature, rather than focus on a single study.  It is sensible to focus on higher quality research, since results are highly sensitive to research design.  But it is completely inappropriate and misleading to pick a single study while ignoring all others of equal or higher quality simply because that one study produces the result you like.

Of course, highlighting the one study she likes is Diane Ravtich’s stock in trade.  All we hear from Ravitch and her Army of Angry Teachers is about the CREDO study.  That’s one study — and not a high quality one.  And even then Ravitch distorts what CREDO finds.

But Betts and Tang’s review includes CREDO and dozens of other studies.  When we look at the full set of research, we see some significant and positive results.  And in Table 4, Betts and Tang show us that if you exclude the CREDO study, the positive effects for charters get stronger so that charters significantly improve math achievement across all grades.

Of course, you shouldn’t exclude that one study, but it is informative that the one study that Ravitch and her Army of Angry Teachers hold up as proof of their view that charters don’t work is clearly an outlier from the full set of research.  And if we focus on the highest quality random-assignment studies of charter schools, the positive results are even stronger.

I wonder if Diane does this in her historical research.  Does she pick the one quotation or document that supports her argument while misleading readers about the entire set of information?  It’s harder to catch Ravitch in this sort of deceptive scholarship in historical work, but in quantitative empirical research, it is the essence of what she does.

(edited for typos)


Peterson and West on the NAACP and Charters

August 3, 2010

Paul Peterson and Marty West have a great piece in today’s WSJ showing how increasingly popular charter schools are among African-Americans.  Despite that fact, the NAACP continues to oppose charters.

Given that 64% of African-Americans surveyed stated that they supported the formation of charter schools (up from 49% last year), Peterson and West remark that: “It’s time civil-rights groups listened to their communities.”

Unfortunately, Peterson and West tell us, the NAACP has picked their political allies in the teacher unions over their constituents:

By casting their lot firmly with teachers unions, the leadership of the NAACP and the Urban League hope to preserve their power and safeguard their traditional sources of financial support. Not only is this is a cynical strategy, it ignores where African-Americans and Hispanics are on the issue. Thankfully, the Obama administration is paying attention to the needs of low-income, minority communities and not to their purported leaders.

You can read more about the survey over at Education Next.


Charter Chatter

February 10, 2010

Readers of JPGB have already seen the working paper, but Education Next now has the peer-reviewed and published version of Booker, Sass, Gill, and Zimmer’s study of the effect of charter high school on graduation and college attendance.  Since you are way ahead of the curve you already knew that attending a charter high school increases the probability that a student will graduate high school and go to college.

The study is so clever because it focuses on students who attended charter middle schools.  Some went on to charter high school and some did not.  By comparing the two groups Booker, et al reduce the selection bias of choice, since all of the students chose charter schools at least for middle school.  But there may still be some selection bias in who chooses to continue in charter high school, so Booker, et al address that with a neato instrumental variable.  Some students don’t go on to charter high school because there isn’t one available nearby.  Their analysis predicts whether students continue to a charter high school based on the availability of nearby charter options.

Check out the highly readable Ed Next article for yourself.  Also watch the podcast interview with Brian Gill.


UCLA Civil Rights Project Gets It Wrong

February 4, 2010

My friends over at Mid-Riffs take apart the new report from Gary Orfield’s UCLA Civil Rights Project claiming that charters produce segregation:

“The report finds:

that charter schools, particularly those in the western United States are havens for white re-segregation from public schools; requirements for providing essential equity data to the federal government go unmet across the nation; and magnet schools are overlooked, in spite of showing greater levels of integration and academic achievement than charters.

It looks like, based on a quick pass through the report, their main finding is based on demographic comparisons  between charter schools and traditional public schools at the state level. This method of comparison likely leads to inaccurate conclusions due to the fact that charter schools are overwhelmingly an urban phenomenon. The correct comparison is between charters and the demographics of their immediate geographic area. We have discussed this topic as it relates to Little Rock at length here.

The Economist’s take on this report is concise, to-the-point, and spot on.

In plain English, there are a lot of black kids in charter schools. This is because charter schools tend to get set up in neighbourhoods where the public schools are terrible, such as south-eastern Washington DC or the rougher parts of New Orleans. These neighbourhoods are disproportionately African-American. Charter schools are popular with poor black parents because their other choices are so awful. There are very few charter schools in rich white suburbs with nice public schools, because there is no call for them.

The important question about charter schools is: do they give kids a better education than they would otherwise have received? The answer is yes. Nothing else matters.”


Cool Charter School Map

December 18, 2009

(Guest Post by Matthew Ladner)

Cool charter school map from U.S. News and World Report.


Coulson Schools LA Times on Charters

December 2, 2009

Andrew Coulson teaches the LA Times a thing or two about charter schools in his post on the Cato blog.  Here’s the meat of it:

Yesterday’s LA Times editorial on charter schools combined errors of fact and omission with a misrepresentation of the economic research on public school spending. First, the Times claims that KIPP charter public schools spend “significantly more per student than the public school system.” Not so, says the KIPP website. But why rely on KIPP’s testimony, when we can look at the raw data? LA’s KIPP Academy of Opportunity, for instance, spent just over $3 million in 2007-08, for 345 students, for a total per pupil expenditure of $8,917. The most recent Dept. of Ed. data for LAUSD (2006-07) put that district’s comparable figure at $13,481 (which, as Cato’s Adam Schaeffer will show in a forthcoming paper, is far below what it currently spends). Nationwide, the median school district spends 24 percent more than the median charter school, according to the National Center for Education Statistics.

Next, in summarizing the charter research, the Times’ editors omitted the most recent and sophisticated study, by Stanford professor Caroline Hoxby. It finds a significant academic advantage to charters using a randomized assignment experimental model that blows the methodological doors off most of the earlier charter research. The Times also neglects to mention Hoxby’s damning critique of the CREDO study it does cite….

There are certainly reasons to lament the performance of the charter sector, and the Times’ editors even came close to citing one of them: its inability to scale up excellence as rapidly and routinely as is the case in virtually every field outside of education. Before getting into such policy issues, however, the Times should make a greater effort to marshal the basic facts.


Mid-Riffs on Arkansas Charters

December 1, 2009

Brian Kisida and Josh McGee, who blog at Mid-Riffs, had an op-ed in the Sunday Arkansas Democrat Gazette on the State Board of Education’s rejection of all six new charter applications.  Here is the money quote:

The Board often cited the same tired reason for denying charter applicants: The proposed charter wasn’t innovative enough. Arkansas Education Commissioner Tom Kimbrell echoed this sentiment after the Board’s meeting, summarizing that he thought the Board was looking for “something different and innovative that students can’t get in a normal public school.” Likewise, a Springdale representative in attendance said that “if a charter school is going to go in, it should offer something better or do something we can’t.” Board member Brenda Gullett, at a Democratic luncheon last week, confirmed that demonstrating innovation was the standard to which she held charter applicants. Some version of this reasoning seems to show up in state and local school board discussions every time a charter school is opposed in Arkansas.

It is an undue burden to force charter applicants to demonstrate radically new techniques before they open their doors. Imagine that Taco Bueno had to get permission from Taco Bell to open a store in the same town. You might hear the same anti-competitive argument from Taco Bell: “Why should Taco Bueno be allowed to open? They’re just going to offer the same things that we do. They have tacos; we have tacos. They have burritos; we have burritos.” But of course, the whole point of choice and competition is that a competitor will offer essentially the same goods or services. If the goods or services are too different, it isn’t really competition after all. It’s up to the customer-not Taco Bell-to decide whose tacos are, in fact, “better.”

Moreover, the question regarding whether charter applicants must demonstrate innovation is a legal one. The state legislature has the power to make laws. The Board, as an arm of the executive branch, has a duty to execute the law. And nothing in Arkansas’ charter school law can reasonably be construed to empower the Board to reject charter applicants solely for not demonstrating innovation. The word “innovation” doesn’t even appear in the Arkansas Department ofEducation’s rules and regulations that govern the requirements for charter school applicants.


The Unions Have Lost Nick Kristoff

October 15, 2009

(Guest Post by Matthew Ladner)

Read it and weep K-12 reactionaries.

P.S.

Somewhere, John Rawls is smiling.


EdWize’s Racial Libel

September 28, 2009

Race Card w watermark

(Guest post by Greg Forster)

On EdWize, Jonathan Gyurko finds himself forced to acknowledge that Caroline Hoxby’s recent blockbuster study is good news for charter schools. He then starts desperately groping for any excuse he can find to neutralize the good news.

Most of his claims will be familiar to those who have seen the teachers’ unions try to spin away gold-standard empirical evidence that their positions are wrong. We’ve read all these cue cards before.

But one of his claims deserves more attention. Like many before him, Gyurko tries his hand at racial demagoguery to make parental choice seem like a scary throwback to Jim Crow:

Such a dramatically-presented conclusion is sure to feature prominently in charter advocates’ efforts to expand the number of charter schools across the city and state. And if it’s true, then why shouldn’t we? The answer actually depends on how policymakers weigh the goal of improved student achievement against other worthy goals, such as greater educational equity and meaningful diversity. And on these other objectives, nagging questions dog the charter sector.

For example, Hoxby finds that 92 percent of charter students are black or Hispanic, compared to 72 percent in district schools and concludes that “the existence of charter schools in the city therefore leaves the traditional public schools less black, more white, and more Asian.” Such racial segregation is consistent with research on charter schools in other states including North Carolina, Texas and elsewhere.

Although this statistic is likely to be a function of charter schools’ location in largely black and Hispanic neighborhoods, Hoxby also reports that fewer white students are applying to the charters; although 14 percent of residents in the charter school neighborhoods are white non-Hispanic, only 4 percent are applying.

There are two claims made here:

1) If the citywide aggregate population of all charter school students is more heavily minority than the citywide aggregate population of district school students, charters must be increasing segregation.

2) If charter school applicants who live near the charter schools are disproportionately minority, charters must be increasing segregation.

Both claims are transparently bogus.

On the first claim: citywide aggregate figures tell us nothing whatsoever about the impact charters are having on segregation, for the simple reason that citywide aggregate figures can tell us nothing whatsoever about segregation in any context, even aside from the whole charter question.

Imagine for a moment that New York is made up of 50% green children and 50% purple children. Let’s look at two scenarios:

Perfect segregation scenrio: All the green children go to fully segregated schools made up exclusively of green children, and all the purple children go to fully segregated schools made up exclusively of purple children.

Perfect integration scenario: All children attend perfectly integrated schools made up of half green children and half purple children.

Now, let’s take a look at the citywide aggregate figures we would get under these two scenarios.

Perfect segregation scenario: Citywide aggregate 50% green, 50% purple.

Perfect integration scenario: Citywide aggregate 50% green, 50% purple.

You see? Aggregate figures are intrinsically incapable of providing any information about school segregation. To find out whether schools are segregated, you must look at the individual schools.

Let’s apply that principle to the real world. Hoxby finds that the citywide aggregate population of district school students is 72% minority. But does that mean every individual school is 72% minority? Of course not. You could very well have all the white children going to perfectly segregated exclusively all-white schools, all the black children to perfectly segregated exclusively all-black schools, all the Hispanic children going to perfectly segregated exclusively all-Hispanic schools, etc., and the citywide aggregate figure would remain unchanged.

And, in fact, the reality on the ground is a lot closer to that dystopian hypothetical than it is to the utopian scenario of ideal racial balance. But Gyurko’s argument relies on the unspoken assumption that the reality on the ground in district schools is utopian.

Meanwhile, the citywide aggregate for charter schools is 92%. As with district schools, the aggregate figure tells us nothing about the actual racial balance in any individual school. Supposing for a moment that New York’s district schools are very heavily segregated – which they are – it is quite possible that the actual charter schools on the ground are better integrated than the district schools even though their aggregate population figure is disproportionately minority.

And, in fact, given that the primary cause of school segregation is housing segregation, the fact that charters can break down neighborhood barriers and draw students from other neighborhoods with different demographics makes it highly likely that they are, in fact, better integrated. That’s the reality in voucher programs, where the empirical evidence unanimously shows parent choice improves integration.

But at any rate, the data to which Gyurko appeals don’t tell us either way.

Once the essential sham behind the first claim is exposed, the second claim is much easier to refute. What counts is not how the local applicant pool differs from the local resident population, but how the final makeup of each charter school differs from the final makeup of each district school. Once the process of parents making choices is completed, are the individual charter schools more segregated? This datum tells us nothing about that.

Ironically, Gyurko’s argument on this second claim really implies that he wants charter schools to represent the racial balance of their local neighborhoods. That would imply endless racial segregation, given that neighborhoods are so racially homogeneous. Any serious attempt to break down racial segregation in schools must begin by acknowledging that schools representing their neighborhoods is the problem.

That’s why hyper-arrogant courts forced us to go through the disastrous failed experiment with forced busing. That was a terrible idea, just like anything that robs parents of their freedom. But at least those tyrannical judges understood the source of the problem correctly.

If parents want to send their children to their local neighborhood schools, they should be allowed. But anything we do that forces them to send their children to school locally is – among so many other evils – going to increase racial segregation. Assigning students to schools by ZIP code is not only educationally bankrupt, it’s racially poisonous.


More Charter Evidence

September 22, 2009

Diane Ravitch has declared that the Obama administration’s policy of expanding the number of charter schools has “no credible basis in research.”  This is just plain wrong.  And a new study coming out today from Stanford’s Caroline Hoxby demonstrates that she is even more wrong.

I’ve already noted that the highest quality studies – those that avoid bias from the self-selection of students into charter schools either with random-assignment or rigorous instrumental variable research designs — show significant academic benefits for students who attend charter schools instead of traditional public schools.  These studies examine the effect of charter schools in Massachusetts, Florida, Chicago, and New York City. 

And now add to that pile an updated study from Caroline Hoxby mentioned in today’s WSJ and NYT on New York City charter effects.  Students accepted by lottery into one of NYC’s charter schools in kindergarten and remained in a charter school through grade 8 closed the achievement gap with wealthy kids attending schools in Scarsdale entirely in math and two-thirds of the way in reading.

Critics are clinging to a study by Margaret Raymond at CREDO, which shows more mixed results.  While that study has the benefit of covering 15 states and DC, it can’t correct for the self-selection of students into charter schools like the highest quality studies linked above.  On average, students appear to be drawn to switching to charter schools because they are having trouble in their traditional public school.  Simply controlling for those students’ prior achievement and other observed demographic factors doesn’t quite correct for whatever negative factors may have caused students to switch to charters and that may continue to hinder their academic progress.  The CREDO study is as good as it can be given its approach, but I would have greater confidence in the consistent findings from several studies in different locations that do control for self-selection into charter schools.


Follow

Get every new post delivered to your Inbox.

Join 839 other followers