District or Charter Schools in the District of Columbia?

May 18, 2012

(Guest Post by Matthew Ladner)

DC’s NAEP numbers allowed for some additional controls to be introduced when comparing charter and district schools than I was able to do with the Milwaukee comparison. The following chart shows the percentage of general education program students who qualify for a free or reduced lunch scoring “Basic or Better” on the 2011 NAEP exams. Special education students, ELL students and middle/high income students are not included in order to get a quick closer to apples to apples comparison.

Now of course for a real apples to apples you need a random assignment study, but those have been done and find results favorable to charter schools. This chart doesn’t address the topic of valid stastical significance, but rather whether the differences are meaningful.

Considering that charters get far less money that DCPS per pupil and show higher levels of academic achievement, this looks to be a success, albeit both the blue and the red columns leave much to be desired. The red columns leave much more to desired however, especially when you consider that that they are wallowing in money.


Charter Benefits Are Proven by the Best Evidence

May 7, 2012

It’s National Charter Schools Week, so here is the post I’ve written for the George W. Bush Institute Blog on the issue:

According to the Global Report Card, more than a third of the 30 school districts with the highest math achievement in the United States are actually charter schools.  This is particularly impressive considering that charters constitute about 5% of all schools and about 3% of all public school students.  And it is even more amazing considering that some of the highest performing charter schools, like Roxbury Prep in Boston or KIPP Infinity in New York City, serve very disadvantaged students.

As impressive and amazing as these results by charter schools may be, it would be wrong to conclude from this that charter schools improve student achievement.  The only way to know with confidence whether charters cause better outcomes is to look at randomized control trials (RCTs) in which students are assigned by lottery to attending a charter school or a traditional public school.  RCTs are like medical experiments where some subjects by chance get the treatment and others by chance do not.  Since the two groups are on average identical, any difference observed in later outcomes can be attributed to the “treatment,” and not to some pre-existing and uncontrolled difference.  We demand this type of evidence before we approve any drug, but the evidence used to justify how our children are educated is usually nowhere near as rigorous.

Happily, we have four RCTs on the effects of charter schools that allow us to know something about the effects of charter schools with high confidence.  Here is what we know:  students in urban areas do significantly better in school if they attend a charter schools than if they attend a traditional public school.  These academic benefits of urban charter schools are quite large.  In Boston, a team of researchers from MIT, Harvard, Duke, and the University of Michigan, conducted a RCT and found:  “The charter school effects reported here are therefore large enough to reduce the black-white reading gap in middle school by two-thirds.”

A RCT of charter schools in New York City by a Stanford researcher found an even larger effect: “On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the ‘Scarsdale-Harlem achievement gap’ in math and 66 percent of the achievement gap in English.”

The same Stanford researcher conducted an RCT of charter schools in Chicago and found:  “students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading…. To put the gains in perspective, it may help to know that 5 to 6 percentile points is just under half of the gap between the average disadvantaged, minority student in Chicago public schools and the average middle-income, nonminority student in a suburban district.”

And the last RCT was a national study conducted by researchers at Mathematica for the US Department of Education.  It found significant gains for disadvantaged students in charter schools but the opposite for wealthy suburban students in charter schools.  They could not determine why the benefits of charters were found only in urban, disadvantaged settings, but their findings are consistent with the three other RCTs that found significant achievement gains for charter students in Boston, Chicago, and New York City.

When you have four RCTs – studies meeting the gold standard of research design – and all four of them agree that charters are of enormous benefit to urban students, you would think everyone would agree that charters should be expanded and supported, at least in urban areas.  If we found the equivalent of halving the black-white test score gap from RCTs from a new cancer drug, everyone would be jumping for joy – even if the benefits were found only for certain types of cancer.

Unfortunately, many people’s views on charter schools are heavily influenced by their political and financial interests rather than the most rigorous evidence.  They don’t want to believe the findings of the four RCTs, so they simply ignore them or cite studies with inferior research designs in which we should have much less confidence.

Progress will be made in our application of research to charter school policies by encouraging everyone to focus on the most rigorous studies, of which we have several.  To do that, supporters of charter schools also have to refrain from citing weaker evidence, which only serves to legitimize the use of inferior studies by charter opponents.  As exciting as the outstanding performance of charter schools is in my own Global Report Card research, that evidence shouldn’t be used to endorse charter schools.  Supporters don’t need to rely on the Global Report Card to make the case for charter schools because they have four gold-standard RCTs on their side.  Opponents of charter schools have no equally rigorous evidence on their side.  And that’s the point we should all be making.


Let’s Go Shopping

April 12, 2012

 

 (Guest post by Patrick Wolf)

We interrupt this celebration of the Jay P. Greene Blog’s four years of extraordinary wonderfulness for a “stop the presses” headline:  71% of parents in Detroit have shopped for (and enrolled a child in) an alternative to their assigned public schools within the past five years.  This is only one of many interesting results from the study Understanding School Shoppers in Detroit by Thomas Stewart and me.

Our study is based on the administration of a door-step survey to over 1000 parents living in nearly 300 different city blocks selected at random for canvassing.  We also held follow-up focus groups with parents and older students.

The report was sponsored by Michigan Future, a non-partisan non-profit organization committed to creating 35 high-quality high schools in the city over the coming years.  They are leading community efforts to improve education in Detroit and enlisted us to perform the first-ever demand study of urban schooling.

The people at Michigan Future sought basic research to better understand Detroit parents as shoppers for k-12 schools.  They wanted to know how many parents had experience with school shopping (a lot!), when did they shop (May-August), how did they shop (through social networks, school fairs, web searches, and a limited number of school visits), and what were they looking for (schools with a strong academic program and safe environment).

Charter schools are the most popular schools of choice for Detroit parents, but a staggering 15% of Detroit children currently attend public schools outside of the city.  Nearly 30% of parents said they would transport a child “up to 8 miles” to access a desirable school, and many clearly are doing so.

Stewart and I further determined that 59% of Detroit parents had the characteristics of “veteran” shoppers in that they had exercised choice in the past as well as the present and plan to continue to shop in the future.  About 12% of parents were classified as “emerging” shoppers who were new to school choice and still trying to figure out where the good stores are.  Another 8% of Detroit parents were “potential” shoppers with many of the characteristics that predicted school shopping, such as disappointment with their child’s school and an expressed willingness to travel long distances to a better school, but who had not yet actually shopped.  The final 21% of parents were classified as “unlikely” school shoppers, with attributes and attitudes that suggest they will continue to accept the default of assigned public schools.

Enjoy browsing!


Teacher union protestor: Why didn’t white folks keep charter schools for themselves?

April 8, 2012

(Guest Post by Matthew Ladner)

Wow-background here.

That has got to be the best teacher union protestor since…

 


Here’s Why Victory Looks Like This

March 7, 2012

(Guest post by Greg Forster)

Jay points to the way Democrats and progressives are now saying all the same things we’ve been saying for a decade, but acting like they thought of them, and remarks that this is What Victory Looks Like.

He’s right, and here’s why. To large extent, you have to let people “steal” your ideas in order to get victory. It’s not just a price we need to be willing to pay if necessary. It’s always necessary.

Major reform of a cultural system has to start with ideas and practices germinating outside the core institutions of that system. If major reform were welcome inside the core institutions, it wouldn’t be necessary in the first place. The incubators of reform can’t be seen as fringe groups – this is why organized libertarianism has had much less influence than its intellectual seriousness and devotion of financial resources might lead you to expect. But the reform incubators are never going to be inside the core, either. You need something that’s a happy medium between credibility and independence.

Now, for a long time in America, the Democratic party and the progressive ideological movement have been the “core” institutions governing education. When you ask the American people whom they trust to do the right thing about education, they overwhelmingly say Democrats and progressives. That makes them the core.

The key to victory is to get the core groups to adopt the ideas that incubated in institutions outside the core. The greatest challenge is that the core groups want to defend their “core” turf against outsiders. They want to keep control of the core, and they can’t do that if they admit that outsiders have superior ideas. The solution is to get the core groups to co-opt (i.e. steal) the ideas and pretend they thought of them.

So you’re never going to get (very many) Democrats and progressives saying, “Why, yes, as a matter of fact the conservatives were right about education all along!” Admitting that would require them to sacrifice their status as the cultural core institutions of American education. Instead they’re going to say, “What American schools need are good, liberal, progressive ideas like choice, competition, and accountability.”

That’s what victory looks like.


School Choice Researchers Unite in Ed Week

February 22, 2012

Pictured (L to R): Rick Hess, Jay Greene, Greg Forster, Mike Petrilli and Matt Ladner

(Guest post by Greg Forster)

Today, Education Week carries a joint editorial signed by nine scholars and analysists. We came together to agree that Mom and apple pie are good, Nazis and Commies are bad, and the empirical research supports the expansion of school choice:

Choice’s track record so far is promising and provides support for continuing expansion of school choice policies…Among voucher programs, random-assignment studies generally find modest improvements in reading or math scores, or both. Achievement gains are typically small in each year, but cumulative over time. Graduation rates have been studied less often, but the available evidence indicates a substantial positive impact. None of these studies has found a negative impact…Other research questions regarding voucher program participants have included student safety, parent satisfaction, racial integration, services for students with disabilities, and outcomes related to civic participation and values. Results from these studies are consistently positive…

In addition to effects on participating students, another major topic of research has been the impact of school choice on academic outcomes in the public school system…Among voucher programs, these studies consistently find that vouchers are associated with improved test scores in the affected public schools. The size of the effect in these studies varies from modest to large. No study has found a negative impact.

We have diverse viewpoints on many issues, but we share a common commitment to helping inform public decisions with such evidence as science is legitimately able to provide. We do not offer false certainty about a future none of us knows. But the early evidence is promising, and the grounds for concern have been shown to be largely baseless. The case for expanding our ongoing national experiment with school choice is strong.

This may well be the most important part:

The most important limitation on all of this evidence is that it only studies the programs we now have; it does not study the programs that we could have some day. Existing school choice programs are severely limited, providing educational options only to a targeted population of students, and those available options are highly constrained.

These limitations need to be taken seriously if policymakers wish to consider how these studies might inform their deliberations. The impact of current school choice programs does not exhaust the potential of school choice.

On the other hand, the goal of school choice should be not simply to move students from existing public schools into existing private schools, but to facilitate the emergence of new school entrants; i.e., entrepreneurs creating more effective solutions to educational challenges. This requires better-designed choice policies and the alignment of many other factors—such as human capital, private funding, and consumer-information sources—that extend beyond public policy. Public policy by itself will not fulfill the full potential of school choice.

Although I also feel particularly strongly about this:

Finally, we fear that political pressure is leading people on both sides of the issue to demand things from “science” that science is not, by its nature, able to provide. The temptation of technocracy—the idea that scientists can provide authoritative answers to public questions—is dangerous to democracy and science itself. Public debates should be based on norms, logic, and evidence drawn from beyond just the scientific sphere.

Signatories:

Kenneth Campbell is the president of the Black Alliance for Educational Options, in Washington.

Paul Diperna is the research director for the Friedman Foundation for Educational Choice, in Indianapolis.

Robert C. Enlow is the president and chief executive officer of the Friedman Foundation for Educational Choice.

Greg Forster is a senior fellow at the Friedman Foundation for Educational Choice.

Jay P. Greene is the department head and holder of the 21st-century endowed chair in education reform at the University of Arkansas, at Fayetteville, and a fellow in education policy at the George W. Bush Institute, in Dallas.

Frederick M. Hess is a resident scholar and the director of education policy studies at the American Enterprise Institute, in Washington, as well as a blogger for Education Week.

Matthew Ladner is a senior adviser for policy and research at the Foundation for Excellence in Education, in Tallahassee, Fla.

Michael J. Petrilli is the executive vice president of the Thomas B. Fordham Institute, in Washington.

Patrick J. Wolf is a professor and holder of the 21st-century endowed chair in school choice at the University of Arkansas, at Fayetteville.

Our color-coordinated mechanical lion battle chariots that join together into a giant robot are still under construction.

Defender of the empirical research universe!


Kevin Carey Gets the Facts Wrong

January 30, 2012

(Guest post by Patrick J. Wolf)

In The Atlantic Online resident cool-kid Kevin Carey sings the “vouchers-are-all-bad-but-charters-are-all-good” song that is the official anthem of the beltway crowd of education reform hipsters.  Carey repeats some points from my own research that school choice results would be even better if parents had more extensive information about schools (but see here for how the mere availability of choice improves parent knowledge about schools) and the supply of choice schools was of consistently higher quality.  Fine.  Carey also claims that private school administrators are rapacious (tell that to the nuns that still run many Catholic schools) and politicians who support school vouchers do so for “obviously partisan reasons” while Mr. Carey only cares about the children.

Unlike Kevin Carey I don’t purport to possess the ability to look inside of people’s souls and conclusively discern their true motives.  Still, his broad-brush claim that all voucher backers are merely trying to “club Democrats” (his words) seems demonstrably inconsistent with the behaviors of voucher supporters such as retiring Independent Senator Joe Lieberman, Senator Diane Feinstein (yeah, she loves to club Democrats), former Democratic Mayor Anthony Williams, Wisconsin State Representative Jason Fields (yet another African-American Democrat who supports vouchers), etc.  I really could go on and present a much longer list, but Kevin Carey only uses single examples to make sweeping generalizations so I’ll simply outperform him by using multiple counter-examples to disprove his universal and unqualified claims.

What disturbs me more than Carey’s reckless accusations is his lack of knowledge of the basic facts surrounding school vouchers.  For example, he states casually that, “To this day, vouchers are only available to small handful (sic) of students.”  The facts are that 27 different voucher or tax-credit funded voucher-like programs serve over 210,000 students.  Even Paul Bunyan’s hands couldn’t hold that many kids.

Carey goes on to state boldly that, “Unlike private schools that pick and choose their pupil (sic), charters are open to all students and allocate scarce openings via lotteries.”  The facts are that many voucher programs do not allow private schools to discriminate in admissions.  In Milwaukee, for example, private schools participating in the voucher program must admit students by lottery but public charter schools in the city can pick and choose their pupils — the exact opposite of what Carey claims.

The D.C. voucher program is “a small, benign, and not particularly effective effort that at its core is nothing more than its name suggests: a program that awards scholarship (sic) to a small group of poor families to partially offset the cost of attending private school”, according to Kevin Carey.  Ignore the fact that this is yet another grammatically incorrect sentence from Mr. Carey.  Is it true?  Well, I know a few things about the District of Columbia Opportunity Scholarship Program, having served as the U.S. Department of Education’s independent evaluator of the program and having written six detailed reports on our nation’s only federally-funded school voucher initiative.

Did the D.C. voucher only “partially offset the cost of attending private school” for families, as Carey claims?  In over 99 percent of cases, the D.C. voucher of up to $7,500 was accepted by schools as full payment from the family.  The private schools accepted less than half the per-pupil government resources allocated to D.C. public schools and either provided a highly efficient education to voucher students or, in many cases, covered the extra costs themselves.  Wait a second, I thought Kevin Carey said that private school operators are greedy and avaricious?

Is the D.C. voucher program “not particularly effective”?  Our gold-standard experimental evaluation concluded that the voucher program increased the high school graduation rate of students by 12 percentage points from the mere offer of the voucher and 21 percentage points if a student actually used it.  That makes the D.C. voucher initiative the most effective drop-put prevention program ever evaluated by the U.S. Department of Education.  The same Milwaukee evaluation that Carey references as showing no net achievement benefits for voucher students also reports that Milwaukee voucher students are graduating from high school and enrolling in college at higher rates due to access to private schools through the program.

President Obama proposed in his State of the Union address that teenagers be compelled to remain in school until they turn 18 or graduate.  Who needs such Big-Brother-like compulsion?  When the government provides more students with access to private schools through vouchers the kids stay in school willingly.

Does Kevin Carey ignore the clear and large graduation rate benefits of the D.C. and Milwaukee voucher programs because he thinks it isn’t desirable for low-income minority children to graduate from high school?  If so, then human compassion and a wealth of research proves him wrong.  More likely, Carey ignores the compelling evidence that school vouchers help disadvantaged students go further in school because it is an inconvenient fact that undermines his argument.  He doesn’t want to admit that voucher programs are effective at promoting the most important student educational outcome there is, and he certainly doesn’t want to share that uncomfortable information with his readers.  Move along, nothing to see here.

After lauding school choice only through public charter schools, Carey states that, ”…the market will still require strong oversight from public officials to grant the ‘approved’ status Friedman envisioned over a half-century ago–and the willingness to revoke that approval when performance is sub-par,” which is exactly how the Milwaukee voucher program is designed and operates.

Doesn’t Carey read anything?  A report released last year documented that the Wisconsin Department of Public Instruction, the government agency that oversees the Milwaukee voucher program, has kicked 35 schools out of the program since 2006.  The average student performance in those schools was dramatically lower than the achievement numbers for the schools allowed to remain in the program.  Voucher programs in the U.S. have exactly the kinds of government accountability mechanisms that Carey falsely claims are missing from them, plus market accountability to boot.

After Kevin Carey’s litany of factual errors, he grandly proclaims the path forward for people, like himself, who actually care about the children.  “We can start by purging the worst rhetoric from the school choice conversation.”  Well, Mr. Carey, before you criticize the splinter in your brother’s eye you might want to work on removing the log from your own.  Meanwhile, readers who want accurate information about school vouchers should, like the Titanic, steer clear of The Atlantic.


A Green Revolution for K-12 Education

January 20, 2012

 (Guest Post by Matthew Ladner)

The remarkably successful effort to introduce improved agricultural technologies into developing world agriculture stands as one of the most underrated technological and philanthropic achievements. India is now a major rice exporter, and the average calories consumed per person in the developing world has increased by 25 percent. If you were hoping that I was referencing an effort to put solar panels on schools, go ahead and stop reading now.

The Green Revolution in the developing world extended a similar pattern established in the Industrial Revolution in substituting technology for labor. Producers making continually improving products at steadily falling prices drives material improvement improving quality of life and reducing poverty.

Substituting technology for labor causes serious social disruption. China’s decades long mass migration of subsistence farm workers into the coastal urban centers for instance holds broad similarities to share croppers moving to industrial areas of the United States decades ago.

We humans have a perfectly understandable desire for stability, and we are easily made victim to nostalgia. Think for instance of Willie Nelson’s “Farm Aid” project aimed at protecting the family farm. The sad fact of course is that many family farms had been made obsolete and were no longer economically viable despite considerable government support. Many cling to the reactionary notion that the world was a better place back during some happy golden age, but certainly from a material standpoint this is just silly. Would anyone in their right mind wish for a world in which Bill Gates and Steve Jobs (and you) would have died at the age of 30 as a subsistence farmer? Why not yearn for the days of living in a cave? China doesn’t need as many people to produce far more food. Rather than bewailing the closing of the coal mine like U2 or Sting, these people have moved into other activities to make a much better life for themselves.

Education which has remained exempt from the productivity improvements experienced by most other human activities. Higher education costs have been racing ahead of even health care inflation for decades, and yet we lack even a drop of evidence to suggest that the average college student of today is meaningfully better educated than his or her peers from 1980. Likewise, after the emergence of education unions as major political powers in the 1960s, American K-12 schools have suffered from an efficiency implosion, with average achievement scores rising at a profoundly slower rate than the inflation adjusted spending per pupil.

Philanthropists played a leading role in bringing the Green Revolution to the developing world- a fantastic and frankly underappreciated success. The focus of philanthropists in American education should likewise be in researching models that can successfully substitute technology for labor in order to produce a better service for a lower cost. They should invest not just in developing the products, but also in the means to bring them to scale through things such as charter school, voucher and digital learning statutes. They should as Jay put it attempt to build new rather than to reform old.

Charter schools have become the skunk works for new school models- taking the lead in both digital and blended learning models. These experiments are very young, and will experience a number of failures. Encouraging and expanding this primordial soup of innovation, however, is of the utmost importance. My only disappointment at this point is that we don’t see more attempts at innovation in the private school sector like Christo Rey. If someone can develop a high quality, low-cost private school model which can survive and thrive outside of public subsidy, the battle for education reform will be much closer to finished.

The  ability to substitute technology for labor in education may have opened the door to such a possibility. We are in only the earliest stages of such experiments, and they are happening with considerable public subsidy, but if India can go from famine in 1961 to a major agricultural exporter today, anything is possible. Clayton Christensen warned that organizations cannot disrupt themselves, often even when they recognize a dire need to do so, so new entrants will likely be necessary.


Michigan Eliminates Dunce Cap on Charter Schools

December 15, 2011

(Guest Post by Matthew Ladner)

The Michigan legislature has voted to eliminate caps on charter schools. This has been a long, hard-fought victory for parental choice supporters.

Over a decade ago, I wrote a study for Mackinac looking at public choice policies in Michigan, focusing on the Detroit area. It proved to be an interesting project. I interviewed a number of suburban superintendents, and they laid out a pattern whereby districts began participating in open enrollment once they felt an enrollment pinch from charter schools. District participation in open enrollment then put pressure on neighboring districts to participate, and so it started a bit of a domino effect.

I’ll never forget interviewing a superintendent from an elite inner-ring suburban district who told me quite boldly that in his district, private schools constituted his true competition, and that he wasn’t worried about charter schools or open enrollment. When I asked him why his district chose not to make seats available through open enrollment, he paused and thoughtfully said “I think the feeling historically around here has been that we have a good thing going on, and there has been a desire to keep the unwashed masses out.”

I appreciated his honesty, but I found myself stunned nevertheless. I mean there was no way to see this statement in some sort of racial context.

Personally, I am very happy that the cap has been lifted on charter schools in Michigan. I hope that the day will come when complacent check-book choice districts might reconsider their decision not to admit students whose parents happen not to be able to afford a $400,000 mortgage. Likewise I hope that increased competition will result in closure of some poorly performing charter schools.

The only sad note is that Michigan’s Blaine amendment will continue to prevent any sort of private school choice, and that Catholic schools in Detroit, which have already been disappearing, may very well go extinct entirely, perhaps along with other private schools. Catholic schools can survive, but the outlook in Detroit is grim indeed. These schools stood as nearly the only high quality options in a once great city for many decades, and is a pity to lose them. If anyone is ever going to develop new low-cost high quality private school models, Detroit seems likely to be a greenfield in the future.

The bigger picture however is that Michigan parents will be gaining new school options. Hopefully the Michigan legislature will continue to pursue additional measures to improve K-12 education outcomes in addition to choice, but today the are to be congratulated for this important step.


Review the Charter Research, Don’t Pick the Outlier

November 2, 2011

Julian Betts and Emily Tang at the University of California at San Diego have a new systematic review of the research on charter schools.  They look at more than 30 studies that meet minimal criteria for research quality.  They find that charters have statistically significant positive effects on math and reading achievement in elementary grades and on math in middle school.  There are no significant effects for reading in middle school or for high school student achievement.  The size of the effects are modest, ranging between 2% and 6% of a standard deviation.  (See Table 2)

It’s important to step back and review an entire literature, rather than focus on a single study.  It is sensible to focus on higher quality research, since results are highly sensitive to research design.  But it is completely inappropriate and misleading to pick a single study while ignoring all others of equal or higher quality simply because that one study produces the result you like.

Of course, highlighting the one study she likes is Diane Ravtich’s stock in trade.  All we hear from Ravitch and her Army of Angry Teachers is about the CREDO study.  That’s one study — and not a high quality one.  And even then Ravitch distorts what CREDO finds.

But Betts and Tang’s review includes CREDO and dozens of other studies.  When we look at the full set of research, we see some significant and positive results.  And in Table 4, Betts and Tang show us that if you exclude the CREDO study, the positive effects for charters get stronger so that charters significantly improve math achievement across all grades.

Of course, you shouldn’t exclude that one study, but it is informative that the one study that Ravitch and her Army of Angry Teachers hold up as proof of their view that charters don’t work is clearly an outlier from the full set of research.  And if we focus on the highest quality random-assignment studies of charter schools, the positive results are even stronger.

I wonder if Diane does this in her historical research.  Does she pick the one quotation or document that supports her argument while misleading readers about the entire set of information?  It’s harder to catch Ravitch in this sort of deceptive scholarship in historical work, but in quantitative empirical research, it is the essence of what she does.

(edited for typos)


Follow

Get every new post delivered to your Inbox.

Join 959 other followers