More Charter Evidence

September 22, 2009

Diane Ravitch has declared that the Obama administration’s policy of expanding the number of charter schools has “no credible basis in research.”  This is just plain wrong.  And a new study coming out today from Stanford’s Caroline Hoxby demonstrates that she is even more wrong.

I’ve already noted that the highest quality studies – those that avoid bias from the self-selection of students into charter schools either with random-assignment or rigorous instrumental variable research designs — show significant academic benefits for students who attend charter schools instead of traditional public schools.  These studies examine the effect of charter schools in Massachusetts, Florida, Chicago, and New York City. 

And now add to that pile an updated study from Caroline Hoxby mentioned in today’s WSJ and NYT on New York City charter effects.  Students accepted by lottery into one of NYC’s charter schools in kindergarten and remained in a charter school through grade 8 closed the achievement gap with wealthy kids attending schools in Scarsdale entirely in math and two-thirds of the way in reading.

Critics are clinging to a study by Margaret Raymond at CREDO, which shows more mixed results.  While that study has the benefit of covering 15 states and DC, it can’t correct for the self-selection of students into charter schools like the highest quality studies linked above.  On average, students appear to be drawn to switching to charter schools because they are having trouble in their traditional public school.  Simply controlling for those students’ prior achievement and other observed demographic factors doesn’t quite correct for whatever negative factors may have caused students to switch to charters and that may continue to hinder their academic progress.  The CREDO study is as good as it can be given its approach, but I would have greater confidence in the consistent findings from several studies in different locations that do control for self-selection into charter schools.

8 Comments | charter schools | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Post Hoc Ergo Propter Hoc

July 17, 2009

Normally solid education reporter, Rob Tomsho, has fallen for the Arne Duncan mania by writing an article in today’s WSJ that attributes expansions of charter school laws to Duncan and the $5 billion “race to the top” stimulus money.

As a graph in the print version of the article shows, charters have been expanding steadily for the last several years — all of which was before Arne Duncan and the stimulus money.  The question is whether charters are growing faster than they otherwise would have.  I doubt it.  The progress of charters doesn’t seem any faster to me now than it has been.  After we get all of the numbers in a year or two we can check to confirm my hunch.

I’m afraid that this is just another example of post hoc ergo propter hoc (after this, therefore because of this).  Duncan made some speeches and conditioned some fraction of a modest sum on policy changes.  There were some policy changes.  So people are attributing the policy changes to his speeches and tiny financial leverage.

3 Comments | charter schools | Tagged: , , | Permalink
Posted by Jay P. Greene

The Negative De-Sarcasticizer

May 14, 2009

Kevin Carey ran my post from yesterday through a “negative de-sarcasticizer”  and wants to take issue with the suggestion that D.C. vouchers were adopted democratically. 

First, I should warn Kevin that a negative de-sarcasticizer actually makes things more sarcastic.  I know because I bought one on Ebay and I use it to help make my posts as sarcastic as they are.  The negative de-sarcasticizer comes with a large, yellow label warning about the hazards of double negatives.

Second, the suggestion that DC vouchers were not democratically created because they affected DC and DC does not have a vote in Congress wouldn’t just call into question the legitimacy of DC vouchers.  All federal laws affecting DC would be undemocratic by this standard.  This would include NCLB and other federal education legislation that Kevin praises charter schools for more strictly obeying.

Third, I am glad that Kevin believes that “giving parents educational choices and opening up public education to competition and innovation will improve outcomes for students.”  And I agree with him that charters would be one way of expanding choices and competition.  But I continue to be puzzled by the argument that vouchers are bad because they are less accountable than charters.  Whatever regulation you believe is desirable for schools could be applied to vouchers as well as to charters.

Finally, I continue to be troubled by Kevin’s need to dismiss vouchers by labeling the idea as “unworkable” or “not serious.”  This is just argumentation by name-calling rather than addressing the substance of the issue.  When I hear this kind of argument it makes me want to turn my negative de-sarcasticizer up to full power.

4 Comments | charter schools, vouchers | Tagged: , , , , | Permalink
Posted by Jay P. Greene

The Growing Charter School Consensus

January 15, 2009

A string of high quality studies is finding that students benefit academically from attending a charter school rather than a traditional public school. 

First we had a random-assignment study of Chicago charter schools by Caroline Hoxby and Jonah Rockoff that found ”that students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading.” 

Then Hoxby conducted a random-assignment study of charter schools in New York City and found: “that the average effect of the charter schools on math is 0.09 standard deviations for every year that a student spends in his or her charter school. The average effect on reading is 0.04 standard deviations for every year that a student spends in his or her charter school.” 

Then Kevin Booker (Mathematica), Tim Sass (Florida State), Brian Gill (Mathematica), and Ron Zimmer (Rand)used a well-designed instrumental variable analysis to see whether charter middle-schoolers who continue to charter high schools are more likely to graduate.  They are. 

And most recently a random-assignment analysis of charter schools in Massachusetts led by Tom Kane at Harvard and Josh Angrist at MIT found that charter school students accepted by lotteries significantly outperformed their counterparts in traditional public schools, unless the charter school was operated by the teacher unions.

In light of these high quality studies, it is harder to oppose charter schools on a scholarly basis.  And with the clear support of charters from the incoming Obama administration, it is getting harder to opposed charter schools on a political basis — at least at the national level.

But don’t expect to see the teacher unions waving a white flag despite their losses in research and national politics.  They don’t need facts or the support of the US Department of Education so long as they continue to dominate local school politics. 

And that is exactly why they have focused on organizing local charter schools to neutralize the threat to their grip on local school politics.  As my colleague Marcus Winters writes today in the New York Post, the unions managed to organize two successful charter schools in New York City.  The fact that union-run charter schools in Massachusetts trailed the non-union charters in performance is not of concern to the unions.  It isn’t about student achievement; it’s about keeping their hold on power even as the facts pile up against them.

3 Comments | charter schools | Tagged: , , , , , , , , , , , | Permalink
Posted by Jay P. Greene

Charters Work, Unions Don’t

January 7, 2009

building_unions

(Guest post by Greg Forster)

On Monday the Boston Foundation released a study by researchers from Harvard, MIT and Duke, examining Boston’s charter schools and “pilot” schools using a random assignment method (HT Joanne Jacobs).

Pilot schools were created in Massachusetts in 1995 as a union-sponsored alternative to charter schools, which came to the state a year earlier. Pilot schools are owned and operated by the school district. Like charter schools, pilot schools serve students who choose to be there (though it’s easier to get into a charter school than a pilot school; see below). Like charter schools, pilot schools have some autonomy over budget, staffing, governance, curriculum, assessment, and calendar. Like charter schools, pilot schools are regularly reviewed and can be shut down for poor performance.

There are two main differences between charter schools and pilot schools. First, the teachers’ unions. Pilot schools have them, and all the shackles on effective school management that come with them. Charter schools don’t.

Second, some pilot schools are only nominally schools of choice, not real schools of choice like charter schools. Elementary and middle pilot schools – which make up a slender majority of the total – participate in the city’s so-called “choice” program for public schools, and thus have an attendance zone where students are guaranteed admission, and admit by lottery for the spaces left over.  So while on paper everyone who goes to a pilot school “chooses” to be there, some of them will be there only because the city’s so-called “choice” system has frozen them out of other schools. The students compared in the study are all lottery applicants and are thus genuinely “choice students” – they are really there by choice, not because they had no practical alternatives elsewhere. However, the elementary and middle pilot schools are not “choice schools.” (Pilot high schools do not have guaranteed attendance zones and are thus real schools of choice.)

The Boston Foundation examined two treatment groups: students who were admitted by lottery to charter schools and students who were admitted by lottery to pilot schools. The control groups are made up of students who applied to the same schools in the same lotteries, but did not recieve admission and returned to traditional public schools.

As readers of Jay P. Greene’s Blog probably know already, random assignment is the gold standard for empirical research because it ensures that the treatment and control groups are very similar. The impact of the treatment (in this case, charter and pilot schools) is isolated from unobserved variables like family background.

The results? Charter schools produce bigger academic gains than regular public schools, pilot schools don’t.

The two perennial fatal flaws of “public school choice” would both seem to be at work here. First, public school choice is always a choice among schools that all partake of the same systemic deficiencies (read: unions). Choice is not choice if it doesn’t include a real variety of options. And second, public school choice typically offers a theoretical choice but makes it impossible to exercise that choice in practice. In this particular case, if each school has a guaranteed-admission attendance zone, the practical result will be fewer open slots in each school available for choice. (Other kinds of public school choice have other ways of blocking parents from effectively using choice, such as giving districts a veto over transfers.)

Charter schools are only an imperfect improvement on “public school choice” in both of these respects. Charters have more autonomy and thus can offer more variety of choice, but not nearly as much as real freedom of choice would provide. And with charters, as with public school choice, government controls and limits the admissions process.

But charters are an improvement over the status quo, even if only a modest one, as a large body of research has consistently shown.

There are some limitations to the Boston Foundation study, as with all studies. Pilot high schools are not required to admit by lottery if they are oversubscribed, while charter schools are. (Funny how the union-sponsored alternative gets this special treatment – random admission is apparently demanded by the conscience of the community when independent operators are involved, but not for the unions.) Of the city’s pilot high schools, two admit by lottery, five do not, and one admits by lottery for some students but not others. Thus, the lottery comparison doesn’t include five of the pilot high schools. It does include three high schools and all of the elementary and middle schools.

As always, we shouldn’t allow the limitations to negate the evidence we do have. Insofar as we have evidence to address the question, more freedom consistently produces better results, and more unionization consistently doesn’t.

3 Comments | charter schools, competitive effects, research reports, teacher union | Tagged: , , , , | Permalink
Posted by Greg Forster

Ohio Charters Save Money for Public Schools and Taxpayers

November 14, 2008

(Guest post by Greg Forster)

It’s raining studies! After this one and then this one comes a study out today from Matthew Carr and Beth Lear of the Buckeye Institute. It’s a fiscal analysis of how charter schools impact the finances of regular public schools in Ohio’s “Big 8″ cities.

When a student leaves a regular public school for a charter school (or a private school for that matter), the district loses the state revenue stream associated with that student, but it gains on the local revenue side because local revenues don’t go down, allowing the district to take that student’s share of local funds and redirect it to funding the education of the students who remain behind. The net fiscal impact depends on which is bigger, the state revenue stream per student or the local property taxes per student.

Carr and Lear find that in Ohio’s Big 8, the regular public schools are fiscal winners when students leave for charter schools. The biggest savings are in Cincinnati, where the net gain is $4,030 per student; the lowest is in Canton, where the net gain is $918 per student.

Charters in Ohio’s Big 8 also keep overall educational costs down by providing a better education (as Carr’s previous work in Ohio has shown) for less money per student.

Leave a Comment » | charter schools, research reports, school funding | Tagged: , , , , , | Permalink
Posted by Greg Forster

Charter School Students More Likely to Graduate High School, Attend College

November 13, 2008

The National Center for the Study of Privatization in Education at Columbia University’s Teachers College has posted a new study on charter schools by Kevin Booker (Mathematica), Tim Sass (Florida State), Brian Gill (Mathematica), and Ron Zimmer (Rand).  The researchers look at whether attending a charter high school in Chicago and Florida increases the likelihood that students would graduate high school and go on to college.  The short answer is that it does.

The paper’s abstract states:

“We find that charter high schools in Florida and in Chicago have substantial positive effects on both high school completion and college attendance. Controlling for observed student characteristics and test scores, univariate probit estimates indicate that among students who attended a charter middle school, those who went on to attend a charter high school were 7 to 15 percentage points more likely to earn a standard diploma than students who transitioned to a traditional public high school. Similarly, those attending a charter high school were 8 to 10 percentage points more likely to attend college. Using the proximity of charters and other types of high schools as exogenous instruments for charter high school attendance, we find even stronger effects in bivariate probit models of charter attendance and educational attainment. While large, our estimates are in line with previous studies of the impact of Catholic high schools on educational attainment.”

But I can already hear doubters wondering how you could compare students in charter schools to other students when the kinds of students who self-select into charter schools could be very different from those who do not. 

But never fear.  These researchers are pretty bright and they worried about this problem as well.  So they came up with three novel strategies to address the possibility of selection bias.  First they try the usual (and not entirely persuasive) technique of controlling statistically for any observed differences between the charter and non-charter students, including race/ethnicity, gender, disability status, family income, and — most importantly — 8th grade student test scores. 

But what about the unobserved (and uncontrollable) qualities of students who choose charters?  Well, their second technique to address potential selection bias is that they compare students who were all in charter schools in 8th grade.  The treatment group went on to a charter high school while the control group went to a traditional public high school.  Since both groups began as charter-choosers, the unobserved qualities of people who choose charters should be present in both groups.  As the authors describe it, “If there are unmeasured student/family characteristics that lead to the selection of charter high schools, these unmeasured characteristics ought to also lead to the choice of a charter school at the middle school level.”

But wait, they did one more thing that really nails the potential problem of selection bias.  They took advantage of the fact that not all students who attend charter middle schools live within a reasonable distance of charter high schools (especially in Florida) to create an “exogeneous” instrument for predicting whether students would attend a charter high school.  That is, they could obtain an unbiased estimate of attending charter high school based on geographic distances and then use that unbiased estimate of charter attendance to obtain an unbiased estimate of the effect of attending a charter high school on graduation and college-attendance.  If you don’t trust me that this technique works to correct for selection bias, you can trust the Nobel prize in economics, which was awarded to James Heckman at the University of Chicago for having developed this technique.

This study comes on the heels of positive results from Caroline Hoxby’s random-assignment evaluation of charter schools in New York City.  Random-assignment corrects for potential selection bias because the students accepted into the charter schools by lottery.  Only chance distinguishes the students in the treatment group (charters) from those in the control group (traditional public).  Hoxby’s analysis finds:

“What is the main result or the bottom line for the grade 3-8 tests? New York City’s charter schools raise their third through eighth graders’ math scores by 0.09 standard deviations for every year they spend in the school. Remember, these gains are in addition to whatever gains the students would have been expected to make in the traditional public schools, had they been lotteried-out. This result is statistically significant with a high level of confidence. (The p-value, shown in parentheses, is less than 0.001.) That means that we are very confident, more than 99% confident, that the effects of New York City’s charter schools on math achievement are not zero or negative…. New York City’s Charter Schools raise their third through eighth graders’ reading scores by 0.04 standard deviations for every year they spend in the school. Remember, these gains are in addition to whatever gains the students would have been expected to make in the traditional public schools, had they been lotteried-out. This result is statistically significant with a high level of confidence. (The p-value, shown in parentheses, is 0.016.) That means that we are very confident (98% confident) that the effects of New York City’s Charter Schools on reading achievement are not zero or negative…. What is a standard deviation? A standard deviation or “effect size” is a conventional way of expressing test scores that works for all tests. If students’ scores rise by one standard deviation, it is a large change in achievement. On most tests it corresponds to more than a grade’s worth of learning and more than a performance level.”

So we now have some very well-designed studies to address selection concerns and they are finding significant benefits from attending charter schools.

3 Comments | charter schools | Tagged: , , , , , , , , | Permalink
Posted by Jay P. Greene

Want to Pass A Local School Tax Increase? Open Charters

November 3, 2008

Here’s a neat piece of research posted at Heny Levin’s National Center for the Study of Privatization in Education.  The study is actually by Arnold Shober and it examines whether the presence of charter schools in a district affects the likelihood that voters will support a local school tax increase. 

It has been getting more and more difficult to obtain local support for school tax increases.  But, Shober wonders, might it be easier to pass a school tax referenda in communities that have more options paid by tax dollars?  Maybe people more satisfied with the quality and diversity of publicly-financed schools, including charter options, are more willing to provide extra tax dollars for all schools.

As it turns out, Shober finds that they do.  He analyzed data from 1,111 school tax referenda in Wisconsin between 1998 and 2005.  He concludes:

“Adding one charter school to the district that has none increases the likelihood of passage 4.1 percent; increasing the number of charter schools from 0 to 8 (the maximum for these data) increases the likelihood of passage 30.2 percent second only to the effect of a college-educated electorate (below). This suggests that charter schools do have some bearing on how votes perceive a school district’s responsiveness to active-parent demands. Indeed, authorizing charter schools is the only variable in this analysis that a school district’s administration could directly manipulate (save the actual ballot request).”

It seems that restricting families’ options and forcing them to attend dirstrict schools whether those schools serve their kids well or not is not the best strategy to get those same families to cough up more dough for the public school system.  People are more likely to be supportive of a public school system that helps them find schools that work for their kids — even if those schools are charters.

2 Comments | charter schools, competitive effects | Tagged: , , , , | Permalink
Posted by Jay P. Greene

The Establishment Mindset

June 16, 2008

(Guest post by Jonathan Butcher)

In education policy, as with any policy area, when discussion turns to reform, there are some basic questions: what are the problems that need fixing?  Who are the interested parties—what is at stake?  Should the change come from the bottom up or from the top down?  Is it raining?  Did I leave my car’s top down?  Do I own a convertible?

Actually, those last few questions only come up if the discussion is taking place inside a government building in Washington because, as everyone knows, there is no place to park in Washington.  So if you left your top down and it starts to rain there is no way you will make it back to your car in time to put it up.  You probably had to park in a metro lot in some swanky Northern Virginia suburb where gas cost $5 before the recent spike in oil prices and there are more Lexus LX’s per capita than any place in the world.  

Change to large systems such as public education can be frightening—but it is often simply because the ideas are misunderstood.  Take a story in the Washington Post last week about charter schools in Louisiana, for example.  New Orleans is now the first city of its size where more than half of the students attend charter schools.  Certainly this is a drastic change: 

“For these new schools with taxpayer funding and independent management, old rules and habits are out. No more standard hours, seniority, union contracts, shared curriculum or common textbooks. In are a crowd of newcomers — critics call them opportunists — seeking to lift standards and achievement. They compete for space, steal each other’s top teachers and wonder how it is all going to work.”  

Hold the phone!  Replacing a system the Post said had a “dismal record and faint prospects of getting better” with new management and scrapping portions of the old system that helped drive it to such a dismal state?  And using public dollars to create this change?  This sort of reform hasn’t happened since…well maybe it was…let me get back to you.

Critics of this change offer a revealing look at the establishment mindset.  One critic charges that “Louisiana school authorities have ‘opened a flea market of entrepreneurial opportunism that is dismantling the institution of public education in New Orleans.’”  Note that this quote uses the word “entrepreneurial” and the idea of taking apart New Orleans’ public education system as though they are bad things.  Well yes, please, bring back union contracts, students sure missed them.  

These charter school operators are “opportunists” in the sense that they are taking advantage of an opportunity to open schools for children whose lives were throttled by Katrina.  One charter school was open for business six weeks after the storm hit, while a public school bureaucracy with more levels than Halo 3 was still looking for its PlayStation.  These charter schools are actually competing for talented teachers in an effort to make the best educational opportunities possible available to students―compare that to an establishment mindset that wistfully refers to the days of payscales.

There are sure to be some challenges for these new charter schools, and as with any change in public policy, the results may be less than perfect.  But students in New Orleans deserve something better than an otherwise “dismal” record.

Leave a Comment » | charter schools, teacher union | Tagged: , | Permalink
Posted by JButcher

Fear and Loathing in Carson City

May 8, 2008

(Guest Post by Matthew Ladner)

I authored a study released by the Nevada Policy Research Institute this week on the Silver State’s education system. Nevada’s education system must address two urgent problems: an ever-growing quantity of students and the low average quality of schools. In spite of these problems, Nevada’s State Board of Education has moved to clamp down on a reform which could help with both problems: charter schools.

Between the year 2000 and the year 2005, Nevada’s school age population increased by 21 percent. This decade began with about 340,000 school age children, but will be nearing 550,000 by 2016.

Nevada is struggling to keep up with these demands. Nevada’s public school spending going for capital outlay in 2003 was over 40 percent higher than the national average on a per pupil basis.

Nevada’s school quality issue represents an even more serious problem. According the Nation’s Report Card from 2007, 43 percent of Nevada 4th graders scored “Below Basic” in reading.

Research shows that children who fail learn basic reading skills in the early grades very often fall further and further behind grade level with each passing year. Moving into middle school, they can scarcely read their textbooks. They begin dropping out in larger numbers in 8th grade.

In other words-the Nevada dropout class of 2015 is moving through the pipeline.

Nevada’s quality and quantity problems are interrelated. The need to construct new public school facilities ultimately draws educational funds out of the classroom. Likewise, the percent of per pupil funding going to service school debt was over sixty percent higher in Nevada than the national average.

A comparison between Nevada and its neighbor, Arizona, however proves that there are solutions to both the quantity and quality problems. Like Nevada, Arizona’s surging population has required a large increase in the supply of schools.

Despite similar rates of enrollment growth, Nevadans spent almost twice as much per student on capital costs as Arizonans in 2003-$1,468 compared to only $776 per pupil in Arizona. Arizona’s interest payments per pupil were also about half of what is paid in Nevada.

How has Arizona managed to manage its quantity problem so much more successfully than Nevada?

In 1994, Arizona lawmakers passed legislation creating choice between public schools and districts, and also one of the nation’s most liberal charter school law.

In 2007, Arizona has 482 charter schools educating over 112,000 children. Arizona charter schools have proven to be extremely diverse- focusing on everything from the arts to back to basics academics to the veterinary sciences.

In addition in 1994, Arizona lawmakers passed a very robust open enrollment law which thousands of students use to transfer between district schools and between school districts.

In 1997, Arizona passed the nation’s first scholarship tax credit law. This program gives individual taxpayers a dollar for dollar credit against state income tax for donations to nonprofit groups giving private school scholarships. In 2007, this program raised $54,000,000 and helped almost 25,000 students attend 359 private schools around the state. Arizona lawmakers created three new private choice programs in 2006.

Arizona’s ability to keep capital costs below the national average came about largely because of this embrace of parental choice in education. Choice options have relieved the need for Arizona’s school district to incur debt in the process of absorbing the increase in the student population.

What has parental choice done for school quality in Arizona? Charter schools comprise an amazing nine of the top 10 publicly funded high schools in the greater Phoenix area. The lone non-charter school on the list is a magnet school, also a choice based school.

Nevada, by comparison, has been hesitant in expanding parental options. In the five states surround Nevada (Arizona, California, Idaho, Oregon and Utah) and these states have 482, 710, 30, 81 and 60 charter schools respectively, collectively educating hundreds of thousands of students. With only 22 charter schools, Nevada is the tortoise of the region.

On November 30 of 2007, the Nevada Board of Education voted 8-0 to impose a moratorium on the approval of new charter schools. Board members told the press that the freeze was necessary because the state Education Department is being “overwhelmed” by 11 charter applications.

Arizona’s State Board for Charter Schools oversees 482 Arizona charter schools with a staff of 8. Nevada’s board overseeing cosmetology currently has 14 full time employees.

In addition, the Nevada legislature created a funding stream for charter school oversight of 2% of the per capita funding. Nevada policymakers must come to recognize the dire need for new high quality schools. Currently, even ultra-high quality charter school operators like KIPP are frozen out of opening schools. If those top 10 schools from Phoenix wished to replicate their success in Nevada, they would be shut out, an absurd denial of opportunity for children.

Nevada policymakers should loathe the status quo and fear the future unless they can radically improve learning, especially for the state’s rapidly growing Hispanic population. They shouldn’t fear or loathe charter schools.

Leave a Comment » | charter schools, competitive effects | Tagged: , , | Permalink
Posted by matthewladner

Next Entries »
Follow

Get every new post delivered to your Inbox.

Join 940 other followers