Hoxby Offers a Free Education

February 28, 2011

Check out this video of an hour long lecture by Stanford’s Caroline Hoxby reviewing economic insights into education policy.  They say that nothing is free, but this comes darn close to a free education.  It will only cost you an hour.

Check out the part on higher education at selective universities that begins around the 40 minute mark.

EdWize’s Racial Libel

September 28, 2009

Race Card w watermark

(Guest post by Greg Forster)

On EdWize, Jonathan Gyurko finds himself forced to acknowledge that Caroline Hoxby’s recent blockbuster study is good news for charter schools. He then starts desperately groping for any excuse he can find to neutralize the good news.

Most of his claims will be familiar to those who have seen the teachers’ unions try to spin away gold-standard empirical evidence that their positions are wrong. We’ve read all these cue cards before.

But one of his claims deserves more attention. Like many before him, Gyurko tries his hand at racial demagoguery to make parental choice seem like a scary throwback to Jim Crow:

Such a dramatically-presented conclusion is sure to feature prominently in charter advocates’ efforts to expand the number of charter schools across the city and state. And if it’s true, then why shouldn’t we? The answer actually depends on how policymakers weigh the goal of improved student achievement against other worthy goals, such as greater educational equity and meaningful diversity. And on these other objectives, nagging questions dog the charter sector.

For example, Hoxby finds that 92 percent of charter students are black or Hispanic, compared to 72 percent in district schools and concludes that “the existence of charter schools in the city therefore leaves the traditional public schools less black, more white, and more Asian.” Such racial segregation is consistent with research on charter schools in other states including North Carolina, Texas and elsewhere.

Although this statistic is likely to be a function of charter schools’ location in largely black and Hispanic neighborhoods, Hoxby also reports that fewer white students are applying to the charters; although 14 percent of residents in the charter school neighborhoods are white non-Hispanic, only 4 percent are applying.

There are two claims made here:

1) If the citywide aggregate population of all charter school students is more heavily minority than the citywide aggregate population of district school students, charters must be increasing segregation.

2) If charter school applicants who live near the charter schools are disproportionately minority, charters must be increasing segregation.

Both claims are transparently bogus.

On the first claim: citywide aggregate figures tell us nothing whatsoever about the impact charters are having on segregation, for the simple reason that citywide aggregate figures can tell us nothing whatsoever about segregation in any context, even aside from the whole charter question.

Imagine for a moment that New York is made up of 50% green children and 50% purple children. Let’s look at two scenarios:

Perfect segregation scenrio: All the green children go to fully segregated schools made up exclusively of green children, and all the purple children go to fully segregated schools made up exclusively of purple children.

Perfect integration scenario: All children attend perfectly integrated schools made up of half green children and half purple children.

Now, let’s take a look at the citywide aggregate figures we would get under these two scenarios.

Perfect segregation scenario: Citywide aggregate 50% green, 50% purple.

Perfect integration scenario: Citywide aggregate 50% green, 50% purple.

You see? Aggregate figures are intrinsically incapable of providing any information about school segregation. To find out whether schools are segregated, you must look at the individual schools.

Let’s apply that principle to the real world. Hoxby finds that the citywide aggregate population of district school students is 72% minority. But does that mean every individual school is 72% minority? Of course not. You could very well have all the white children going to perfectly segregated exclusively all-white schools, all the black children to perfectly segregated exclusively all-black schools, all the Hispanic children going to perfectly segregated exclusively all-Hispanic schools, etc., and the citywide aggregate figure would remain unchanged.

And, in fact, the reality on the ground is a lot closer to that dystopian hypothetical than it is to the utopian scenario of ideal racial balance. But Gyurko’s argument relies on the unspoken assumption that the reality on the ground in district schools is utopian.

Meanwhile, the citywide aggregate for charter schools is 92%. As with district schools, the aggregate figure tells us nothing about the actual racial balance in any individual school. Supposing for a moment that New York’s district schools are very heavily segregated – which they are – it is quite possible that the actual charter schools on the ground are better integrated than the district schools even though their aggregate population figure is disproportionately minority.

And, in fact, given that the primary cause of school segregation is housing segregation, the fact that charters can break down neighborhood barriers and draw students from other neighborhoods with different demographics makes it highly likely that they are, in fact, better integrated. That’s the reality in voucher programs, where the empirical evidence unanimously shows parent choice improves integration.

But at any rate, the data to which Gyurko appeals don’t tell us either way.

Once the essential sham behind the first claim is exposed, the second claim is much easier to refute. What counts is not how the local applicant pool differs from the local resident population, but how the final makeup of each charter school differs from the final makeup of each district school. Once the process of parents making choices is completed, are the individual charter schools more segregated? This datum tells us nothing about that.

Ironically, Gyurko’s argument on this second claim really implies that he wants charter schools to represent the racial balance of their local neighborhoods. That would imply endless racial segregation, given that neighborhoods are so racially homogeneous. Any serious attempt to break down racial segregation in schools must begin by acknowledging that schools representing their neighborhoods is the problem.

That’s why hyper-arrogant courts forced us to go through the disastrous failed experiment with forced busing. That was a terrible idea, just like anything that robs parents of their freedom. But at least those tyrannical judges understood the source of the problem correctly.

If parents want to send their children to their local neighborhood schools, they should be allowed. But anything we do that forces them to send their children to school locally is – among so many other evils – going to increase racial segregation. Assigning students to schools by ZIP code is not only educationally bankrupt, it’s racially poisonous.

More Charter Evidence

September 22, 2009

Diane Ravitch has declared that the Obama administration’s policy of expanding the number of charter schools has “no credible basis in research.”  This is just plain wrong.  And a new study coming out today from Stanford’s Caroline Hoxby demonstrates that she is even more wrong.

I’ve already noted that the highest quality studies — those that avoid bias from the self-selection of students into charter schools either with random-assignment or rigorous instrumental variable research designs — show significant academic benefits for students who attend charter schools instead of traditional public schools.  These studies examine the effect of charter schools in Massachusetts, Florida, Chicago, and New York City. 

And now add to that pile an updated study from Caroline Hoxby mentioned in today’s WSJ and NYT on New York City charter effects.  Students accepted by lottery into one of NYC’s charter schools in kindergarten and remained in a charter school through grade 8 closed the achievement gap with wealthy kids attending schools in Scarsdale entirely in math and two-thirds of the way in reading.

Critics are clinging to a study by Margaret Raymond at CREDO, which shows more mixed results.  While that study has the benefit of covering 15 states and DC, it can’t correct for the self-selection of students into charter schools like the highest quality studies linked above.  On average, students appear to be drawn to switching to charter schools because they are having trouble in their traditional public school.  Simply controlling for those students’ prior achievement and other observed demographic factors doesn’t quite correct for whatever negative factors may have caused students to switch to charters and that may continue to hinder their academic progress.  The CREDO study is as good as it can be given its approach, but I would have greater confidence in the consistent findings from several studies in different locations that do control for self-selection into charter schools.

This is the Song That Never Ends

May 20, 2009

The AFT’s Leo Casey of union cue-card check and sock-puppet fame has written a blog post for his “steakholders” once again accusing me of cherry-picking.  The last time Leo accused me of cherry-picking voucher studies I produced what I believe are comprehensive lists of random-assignment voucher participant and high-quality voucher competitive effect studies.  Given his inability to substantiate that cherry-picking charge, I’m a bit surprised to see that he is a glutton for punishment and wants to make the charge again.  I guess Leo has joined with Shari Lewis, Lambchop, and all his sock-puppet friends to make this cherry-picking charge another song that never ends.

This time the issue is whether teacher unions tend to raise costs and lower student achievenement.  Leo noticed my guest posts over at Flypaper on this and asserts: “if you think that the scholarly literature on the subject is a guide, it clearly comes down in a place quite different from that suggested by Greene.”

Leo’s claim hinges entirely on what “scholarly literature” means and whether all “studies” should be treated equally.  For example, I could claim that the scholarly literature shows that candy improves student achievement, citing as scholarly literature papers written by my 5th grade son and friends whose research design involved describing how smart they felt after eating candy.

Leo doesn’t go quite that far but he does cite a study by the AFT’s very own Howard Nelson that makes a cross-sectional comparison of test scores controlling for a handful of observed demographics.  He also cites literature reviews that consist mostly of these cross-sectional analyses controlling for observed demographics. 

The problem is that there is a serious design flaw with these studies — unobserved factors that are associated with unionization may also be associated with student achievement.  For example, wealthier communities may be more likely to produce unionization because those communities have the wealth to bear the higher costs associated with unionized teachers.  Wealth may be associated with higher student achievement, but our controls for wealth (free lunch status) may not fully or accurately capture the differences in wealth.  So, unobserved and uncontrolled factors would bias the results from these cross-sectional studies.

Caroline Hoxby’s study, upon which I base my claims, employs a vastly superior research design that addresses this problem.  I’ll let her describe the problem and how she solves it:

“The … most serious obstacle is the identification problem caused by the difficulty of differentiating between the effects of a union on a school and the characteristics of a school  that make a union more likely to exist.  Even after controlling for observable characteristics of a school district such as demographics, there are presumably unobservable school characteristics associated with unionization.  The unobservable school characteristics that promote unionization may themselves affect the education production function….  My third, and probably best, attempt to solve the identification problem combines differences-in-differences and instrumental variables estimation.”

Her instrumental variable strategy involves using changes in state laws regarding unionization to derive unbiased estimates of when schools would unionize.  The change in the state law would help predict whether a school unionizes without being associated with the academic achievement in that school.  This is a far better way to estimate the effect of unionization than simply looking at whether unionized schools have higher or lower scores, since the scores and other factors associated with school quality could themselves be causing the unionization.

I’d put much more confidence in this rigorously designed study than a dozen weakly designed cross-sectional analyses.

But even if Leo insisted upon relying on the literature reviews he cites rather than the higher quality research, he would have to accept some results that aren’t very flattering to teacher unions.  Those lit reviews find that unionization raises the cost of education by about 8% to 15%.  In addition, they find that unionization tends to hurt the academic achievement of high-achieving and low-achieving students while benefiting more typical students found in the middle of the ability distribution. 

As Leo’s authority, Eberts, Hollenbeck and Stone, put it: “While on average students fare at least as well, if not better, in unionized schools, atypical students – students well below or above average ability – do appear to fare less well because instructional settings are more standardized, less individualized in unionized schools.”

So, if Leo wants to say that unions exacerbate the achievement gap for disadvantaged minority students while driving up costs, I guess he can rely on that literature review.  I prefer to rely on Caroline Hoxby’s rigorously designed study in a top economics journal.

The Growing Charter School Consensus

January 15, 2009

A string of high quality studies is finding that students benefit academically from attending a charter school rather than a traditional public school. 

First we had a random-assignment study of Chicago charter schools by Caroline Hoxby and Jonah Rockoff that found “that students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading.” 

Then Hoxby conducted a random-assignment study of charter schools in New York City and found: “that the average effect of the charter schools on math is 0.09 standard deviations for every year that a student spends in his or her charter school. The average effect on reading is 0.04 standard deviations for every year that a student spends in his or her charter school.” 

Then Kevin Booker (Mathematica), Tim Sass (Florida State), Brian Gill (Mathematica), and Ron Zimmer (Rand)used a well-designed instrumental variable analysis to see whether charter middle-schoolers who continue to charter high schools are more likely to graduate.  They are. 

And most recently a random-assignment analysis of charter schools in Massachusetts led by Tom Kane at Harvard and Josh Angrist at MIT found that charter school students accepted by lotteries significantly outperformed their counterparts in traditional public schools, unless the charter school was operated by the teacher unions.

In light of these high quality studies, it is harder to oppose charter schools on a scholarly basis.  And with the clear support of charters from the incoming Obama administration, it is getting harder to opposed charter schools on a political basis — at least at the national level.

But don’t expect to see the teacher unions waving a white flag despite their losses in research and national politics.  They don’t need facts or the support of the US Department of Education so long as they continue to dominate local school politics. 

And that is exactly why they have focused on organizing local charter schools to neutralize the threat to their grip on local school politics.  As my colleague Marcus Winters writes today in the New York Post, the unions managed to organize two successful charter schools in New York City.  The fact that union-run charter schools in Massachusetts trailed the non-union charters in performance is not of concern to the unions.  It isn’t about student achievement; it’s about keeping their hold on power even as the facts pile up against them.

Charter School Students More Likely to Graduate High School, Attend College

November 13, 2008

The National Center for the Study of Privatization in Education at Columbia University’s Teachers College has posted a new study on charter schools by Kevin Booker (Mathematica), Tim Sass (Florida State), Brian Gill (Mathematica), and Ron Zimmer (Rand).  The researchers look at whether attending a charter high school in Chicago and Florida increases the likelihood that students would graduate high school and go on to college.  The short answer is that it does.

The paper’s abstract states:

“We find that charter high schools in Florida and in Chicago have substantial positive effects on both high school completion and college attendance. Controlling for observed student characteristics and test scores, univariate probit estimates indicate that among students who attended a charter middle school, those who went on to attend a charter high school were 7 to 15 percentage points more likely to earn a standard diploma than students who transitioned to a traditional public high school. Similarly, those attending a charter high school were 8 to 10 percentage points more likely to attend college. Using the proximity of charters and other types of high schools as exogenous instruments for charter high school attendance, we find even stronger effects in bivariate probit models of charter attendance and educational attainment. While large, our estimates are in line with previous studies of the impact of Catholic high schools on educational attainment.”

But I can already hear doubters wondering how you could compare students in charter schools to other students when the kinds of students who self-select into charter schools could be very different from those who do not. 

But never fear.  These researchers are pretty bright and they worried about this problem as well.  So they came up with three novel strategies to address the possibility of selection bias.  First they try the usual (and not entirely persuasive) technique of controlling statistically for any observed differences between the charter and non-charter students, including race/ethnicity, gender, disability status, family income, and — most importantly — 8th grade student test scores. 

But what about the unobserved (and uncontrollable) qualities of students who choose charters?  Well, their second technique to address potential selection bias is that they compare students who were all in charter schools in 8th grade.  The treatment group went on to a charter high school while the control group went to a traditional public high school.  Since both groups began as charter-choosers, the unobserved qualities of people who choose charters should be present in both groups.  As the authors describe it, “If there are unmeasured student/family characteristics that lead to the selection of charter high schools, these unmeasured characteristics ought to also lead to the choice of a charter school at the middle school level.”

But wait, they did one more thing that really nails the potential problem of selection bias.  They took advantage of the fact that not all students who attend charter middle schools live within a reasonable distance of charter high schools (especially in Florida) to create an “exogeneous” instrument for predicting whether students would attend a charter high school.  That is, they could obtain an unbiased estimate of attending charter high school based on geographic distances and then use that unbiased estimate of charter attendance to obtain an unbiased estimate of the effect of attending a charter high school on graduation and college-attendance.  If you don’t trust me that this technique works to correct for selection bias, you can trust the Nobel prize in economics, which was awarded to James Heckman at the University of Chicago for having developed this technique.

This study comes on the heels of positive results from Caroline Hoxby’s random-assignment evaluation of charter schools in New York City.  Random-assignment corrects for potential selection bias because the students accepted into the charter schools by lottery.  Only chance distinguishes the students in the treatment group (charters) from those in the control group (traditional public).  Hoxby’s analysis finds:

“What is the main result or the bottom line for the grade 3-8 tests? New York City’s charter schools raise their third through eighth graders’ math scores by 0.09 standard deviations for every year they spend in the school. Remember, these gains are in addition to whatever gains the students would have been expected to make in the traditional public schools, had they been lotteried-out. This result is statistically significant with a high level of confidence. (The p-value, shown in parentheses, is less than 0.001.) That means that we are very confident, more than 99% confident, that the effects of New York City’s charter schools on math achievement are not zero or negative…. New York City’s Charter Schools raise their third through eighth graders’ reading scores by 0.04 standard deviations for every year they spend in the school. Remember, these gains are in addition to whatever gains the students would have been expected to make in the traditional public schools, had they been lotteried-out. This result is statistically significant with a high level of confidence. (The p-value, shown in parentheses, is 0.016.) That means that we are very confident (98% confident) that the effects of New York City’s Charter Schools on reading achievement are not zero or negative…. What is a standard deviation? A standard deviation or “effect size” is a conventional way of expressing test scores that works for all tests. If students’ scores rise by one standard deviation, it is a large change in achievement. On most tests it corresponds to more than a grade’s worth of learning and more than a performance level.”

So we now have some very well-designed studies to address selection concerns and they are finding significant benefits from attending charter schools.

%d bloggers like this: