Eduwonk: Vouchers Boring, Bus Service Consolidation Fascinating

July 23, 2008

(Guest post by Greg Forster)

One of Eduwonk’s most important contributions to the education blogosphere is the snarky comment. Snark is frowned upon in these parts (don’t leave a snarky comment on this post, or you’ll be violating site policy). But if I may register a minority report on behalf of the loyal opposition, I think snarky comments do for the marketplace of ideas what short sellers and speculators do for the stock market: they’re not classy or respectable, but they perform an indispensable price-discovery function. (Another similarity is that the people who do them make tons of money – right, Andy? Andy?)

Of course, not all brokerage houses are interested in having their services used by short sellers, and there may be some sense in declaring some education blogs as snark-free zones. The short sellers can always take their business to other brokers – just like I post all my snarky comments on Pajamas Media.

But if Eduwonk is going to dish out snark, he’d better be able to take it. So check out what I found while scrolling through Eduwonk this morning.

In response to John McCain’s big education speech at the NAACP:

If this campaign turns into a debate about vouchers please just shoot me now. I’d prefer a debate that ignores education than that tired fight again.

Two days earlier:

Shouldn’t we be . . . trying to get school districts out of the busing business altogether? Big school districts like to boast about how they bus more passengers each day than Greyhound. That’s true, but also sort of insane if you think about it and consider that their primary mission is teaching and learning. Besides, today’s buses are horrendous polluters even when greener technology is available, control over transportation means control over parental decision-making, and school districts often aren’t even very good at designing efficient transportation schemes or adapting to changing circumstances like $4 gas, which was not exactly an unforeseen issue in the transportation world…Student safety means that, especially for younger students you want to be careful about how you merge transportation schemes, but having local or regional agencies that handle transportation would pay a lot of dividends if was approached with the dual principles of being greener and more parent- and civic-friendly at the front-end.

Got that? A debate about the policy that represents the most fundamental break from the existing system, is most consistently supported by empirical evidence, and is currently the most politically successful movement in the education world is boring enough to induce suicide. But the prospect of transferring control of bus services from school districts to local or regional transportation agencies is fascinating.

Supply your own snarky comment; I’m not allowed.

But before leaving the subject, I will note that if you’ve seen the new Batman movie, you already know whom to call for all your school bus operation needs:

You know how I got these scars? I was in an accident caused by an incompetent school bus driver, because educational transportation is controlled by school districts whose core mission is teaching and learning rather than by local or regional transportation agencies!

1 Comment | vouchers | Tagged: , , | Permalink
Posted by Greg Forster

Weekend PJM Column

June 25, 2008

(Guest post by Greg Forster)

I was out of town earlier this week and didn’t get a chance to post a link to my Pajamas Media column on the D.C. voucher evaluation, which ran over the weekend. It’s here.

Leave a Comment » | research reports, vouchers | Tagged: , , , | Permalink
Posted by Greg Forster

What Does the Red Pill Do If I Don’t Take It?

June 19, 2008

 

(Guest Post by Matthew Ladner)

The hidden highlight from the Evaluation of the DC Opportunity Scholarship Program: Impacts After Two Years report is buried in the Appendix, pp. E-1 to E-2:

Applying IV analytic methods to the experimental data from the evaluation, we find a statistically significant relationship between enrollment in a private school in year 2 and the following outcomes for groups of students and parents (table E-1):

• Reading achievement for students who applied from non-SINI schools; that is, among students from non-SINI schools, those who were enrolled in private school in year 2 scored 10.73 scale score points higher (ES = .30)^2 than those who were not in private school in year 2.

• Reading achievement for students who applied with relatively higher academic performance; the difference between those who were and were not attending private schools in year 2 was 8.36 scale score points (ES = .24).

• Parents’ perceptions of danger at their child’s school, with those whose children were enrolled in private schools in year 2 reporting 1.53 fewer areas of concern (ES = -.45) than those with children in the public schools.

• Parental satisfaction with schooling, such that, for example, parents are 20 percentage points more likely to give their child’s school a grade of A or B if the child was in a private school in year 2.

• Satisfaction with school for students who applied to the OSP from a SINI school; for example, they were 23 percentage points more likely to give their current school a grade of A or B if it was a private school.

I’m trying to figure out why the impact of actually using the voucher program isn’t actually the focus of this study, and in fact is presented in an appendix. Instead all the “mixed” results are studying the impact of having been offered a scholarship whether the student actually used it or not.

I’m going to walk way out on a limb here and predict that the impact on test scores of being offered but not using a voucher will be indistinguishable from zero. If this were a medical study, we would have a group of patients in a control and experimental group offered a drug, some of them choose not to take it, but we ignore that fact and measure the impact of the drug based on the results of both those who took it and those who didn’t. Holding the pill bottle can’t be presumed to have the same impact as taking the pills.

We’ve all been told that exercise is good for our health. Should we judge the effectiveness of exercise on health outcomes by what happens to those who actually exercise, or by the results for everyone that has been told that it is good for you?

This shortcoming has been corrected in the Appendix, but that is getting very little attention. On page 24 the evaluation reads:

Children in the treatment group who never used the OSP scholarship offered to them, or who did not use the scholarship consistently, could have remained in or transferred to a public charter school or traditional DC public school, or enrolled in a non-OSP-participating private school.

So in the report’s main discussion, the kids actually attending private schools have to make gains big enough to make up for the fact that many “treatment” kids are actually back in DCPS. As it turns out, several subsets of students do make such gains, but that’s not the point. The point is we ought to be primarily concerned with whether actual utilization of the program improves education outcomes and with systemic effects of the program. We should indeed study who actually uses this program, and who chooses not to and the reasons why (very important information), but this sort of analysis seems to belong in the appendix rather than the other way around.

Receiving an offer of a school voucher doesn’t constitute much of an education intervention, and it seems painfully obvious that the discussion around this report is conflating the impact of voucher offers with that of voucher use. The impact of voucher use is clear and positive.

12 Comments | methodology, research reports, vouchers | Tagged: , | Permalink
Posted by matthewladner

School Choice Wins in 2008; Unrestricted Eligibility in Georgia

June 18, 2008

(Guest post by Greg Forster)

The Washington Post is now reporting that the House Appropriations subcommittee will fund the DC voucher program for another year. People are saying that the future of the program doesn’t look good, because the subcommittee chairman is blustering about how much he doesn’t like it. But read that Post article carefully. He doesn’t say that the program will be killed next year. The Post reports that he says he’s funding the program for another year “to give District leaders a chance to restructure the program.” He is quoted as saying, “I expect that during the next year the District leaders will come forward with a firm plan for either rolling back the program or providing some alternative options.”

That sounds to me like a man who’s looking for a deal. The DC program is already loaded up with monster payoffs to the District’s patronage-bloated public school system. How hard is it to make those payoffs bigger? And maybe the program will have to accept some more politically motivated restrictions on participation, so that critics will have a trophy to hang on their wall.

Whether those tradeoffs are worth it for the school choice movement – there is a real cost, and not just in dollars, associated with them – is a question I leave for another day. And of course this is just the subcommittee; there could still be more trouble ahead. And maybe next year the critics will get a better offer from the unions than the deal they’re apparently angling to get on behalf of the DC patronage machine.

All I want to do is observe that the program’s chances of survival are now looking a lot better than they did yesterday.

As the political season winds to a close, let’s survey the results:

  • A new personal tax credit for private school tuition in Louisiana
  • A new tax-credit scholarship program in Georgia
  • A new voucher program in Louisiana
  • An expansion of Florida’s tax-credit scholarship program, including a $30 million increase in the cap; a bump up in the value of the scholarship and a linking of the scholarship value to state school spending (which always goes up); and a relaxation of the program’s unreasonably stringent accounting rules (which used to allow not one penny of carryover from year to year in the scholarship organizations’ accounts, and not one penny from eligible donations for administrative expenses).
  • A million-dollar funding increase and guaranteed future funding stream for Utah’s voucher program.
  • Preservation (tentatively) of the DC voucher program in a hostile Congress.

That’s three new programs, two expansions of existing programs and an upset victory in DC. Pretty good for a dead movement, wouldn’t you say?

By the way, how did accountability testing do this year? How many new programs? How many existing programs expanded?

How about instructional and curricular reforms? How’s the Massachussetts miracle holding up?

Anyone? . . . Anyone?

Some of these victories did come at a cost. The two programs in Louisiana are going to score poorly when measured against the gold standard of universal choice. The tax credit is limited to a very small amount of money, which means it offers a very small amount of choice. And the new voucher program is only offered to students who are in grades K-3, low-income, and enrolled in public schools (or entering kindergarten) in a chronically failing school district located in a highly populated parish – which currently means only New Orleans. Plus it’s limited by annual appropriations (currently $10 million). A new grade level will become eligible each year (4th grade next year, then 5th grade, etc.) and Baton Rouge may become eligible if its public schools continue to fail. But this is still an inadequate program. And we can also add the prospect of more restrictions in the DC program to the debit column.

But there was also a huge step forward for universal choice. Georgia’s new tax-credit scholarship program offers school choice for all students. It has no demographic restrictions at all. Any public school student can apply. The only limit is the $50 million program cap – and experience in other states pretty consistently shows that dollar caps rise as programs grow to meet them.

Georgia’s new program is basically the same as the Arizona program funded by individual donations, except that Georgia’s program also allows corproate donations. And that makes a big difference, because it greatly expands the pool of available funds – and hence the size of the program.

Come to think of it, Georgia’s program is the first tax-credit scholarship program to include corporate donations and not place demographic restrictions on who can participate. That’s a potentially powerful combination. It will be exciting to see whether Georgia ends up taking school choice to a whole new level.

1 Comment | instructional reform, personal tax credits/deductions, politics, tax-credit scholarships, vouchers | Tagged: , , , , , , , , | Permalink
Posted by Greg Forster

Congressional Subcommittee Hearing on D.C. School Choice Funding

June 17, 2008

(Guest post by Dan Lips)

Today, the House Financial Services and General Government Appropriations Subcommittee will hold a mark-up of the federal government’s budget for the District of Columbia. The panel must decide whether to include President Bush’s proposed funding for continuing the D.C. Opportunity Scholarship program.

As Greg mentioned yesterday, the Department of Education released its second year evaluation of the scholarship program.  Here’s the Department’s basic summary of what the report found:

Reading achievement improved for three large subgroups of students, comprising 88 percent of participating students. In fact, their gains put them about two to four months ahead of their peers who did not receive a scholarship. While the report found no statistically significant difference in test scores overall between students who were offered a scholarship and students who were not offered a scholarship, achievement trends are moving in the right direction. The positive effects found in this year’s report are larger than those in last year’s report, and whenever statistically significant effects were found, they favored students who were offered scholarships.

The report also found that scholarship parents were more satisfied with their children’s schools and they believed their children’s schools to be safer than their previous public school.   So, test scores for participating kids are tilting higher, and families report being happier when they have a choice.   

 

Of course, these generally positive results won’t be enough to convince some in Congress to support continuing the program.  What strikes me as really odd about this debate is that this program is being held to such a high-bar for proving its effectiveness.  If only Congress were this critical of all government programs.  Every year, the Bush administration tries to terminate as many as 47 federal education programs that have been judged by the federal government to: “have achieved their original purpose, duplicate other programs, are narrowly focused, or unable to demonstrate effectiveness.”  But these programs somehow find a way to live on, supporting Ronald Reagan’s quip: “the closest thing to eternal life on this earth is a government program.” 

 

For D.C. families, it’s clear that the Opportunity Scholarship program is one worth keeping.  Parent activist Virginia Walden Ford penned a good column for National Review Online on the real impact school choice is having for D.C. families.  For more commentary on D.C. school choice, check out William McGurn’s column for the Wall Street Journal or pieces by Kathryn Lopez and Carrie Lukas for NRO. You can also check out a column I wrote for Heritage. 

 

Stay tuned.  I’ll report back when we learn what happens in the House subcommittee mark-up. 

2 Comments | politics, vouchers | Tagged: , , | Permalink
Posted by danlips

The DC Voucher Evaluation

June 16, 2008

(Guest post by Greg Forster)

Today the U.S. Dept. of Education released the fourth annual report on the random-assignment evaluation of the DC voucher program, including academic results for the first two years of the program’s existence. As with last year’s report, across the whole population the voucher students had higher academic outcomes than the control group, but the positive results just barely fell short of the conventional cutoff for statistical certainty. This means that while the voucher students in fact had higher test scores, we cannot be 95 percent confident that their higher scores are due to vouchers and not a statistical fluke. This year it was the reading results that came close to statistical significance, reaching 91 percent certainty. The study also finds statistically certain positive results for three subgroups, which together comprise 88 percent of the voucher population.

Since the previous year’s results were also not statistically significant, this update of the study doesn’t change the balance of the studies on school choice. As before, there are a total of ten random-assignment studies on school vouchers, all ten of which found that the voucher students had higher academic achievement, with eight studies achieving statistical certainty for the positive finding and two not.

In other words, school vouchers are still better supported by high-quality scientific evidence than any other education policy. If you reject vouchers because this study is only 91 percent sure they produce academic improvements, you have no empirical grounds for supporting any other policy, since all other policies are far less well supported by empirical evidence than vouchers.

In a few minutes you’ll be able to see the Friedman Foundation’s response to the DC study, including details and citations on all ten random-assignment studies of vouchers, here.

2 Comments | research reports, vouchers | Tagged: , , | Permalink
Posted by Greg Forster

Responding to Response to Intervention

June 2, 2008

(Editorial Note — See also follow-up post here)

Like many well-meaning instructional reforms, Response to Intervention (RTI) is likely to fail if it is not coupled with other reforms that address the perverse incentives blocking its proper implementation.

The idea behind RTI is that we could avoid placing many students in special education if only we provided them with well-designed instructional approaches in the early grades.  The huge increase in special education enrollments consists almost entirely of growth in Specific Learning Disability (SLD), which is an ambiguous category that is difficult for practitioners to diagnose properly.  Almost any student with a normal range IQ but sub-par achievement could be labeled as SLD.  But of course, students may lag in their achievement because they have been poorly taught, not because they have a problem processing information, as is characteristic of a true SLD.  Schools have a variety of incentives to discount the former explanation and instead push students into special ed.

RTI is a federally-backed program that attempts to address this problem by allowing schools to divert 15% of their special education money into well-designed instructional programs for the early grades.  If students are taught well, they won’t be lagging academically and so will not end up being identified as disabled.

This all sounds great, but it is almost certainly doomed to failure if we do not also address why schools were not previously providing well-designed instruction in early grades or why they are so motivated to identify students as disabled.  Essentially, RTI frees-up money to get schools to do what they presumably should have been doing already — providing well-designed instruction in the early grades.  Unless we think that the main impediment to well-designed instruction was that schools lacked the funding to do it, diverting 15% of special education money to early-grade instruction will not get them to do anything significantly different from what they were already doing.  Even if we thought that the problem was that schools were unaware of the effective approaches that RTI offers, we have no reason to believe that schools will truly adopt or effectively implement those strategies. 

It is a a seductive but entirely mistaken reform approach to believe that schools are eagerly awaiting to be told by the federal government or philanthropists how to teach effectively but are just lacking the critical resources and knowledge to do it.  Schools already hire certified professionals who have been exposed to countless hours of pre-service and in-service training.  Why would we think that the only reason that they are failing to employ an effective technique is because they are unaware of it?  And with school budgets increasing every year, why would we think that the next bit of money is the one that they finally need to pursue effective strategies?

Instead, we have to recognize that educators have reasons for doing what they are doing.  They generally believe that the techniques they’ve adopted are effective, even if they aren’t.  Getting them to switch to something else takes more than just offering it to them.  This is especially the case when they’ve seen untold failed instructional fads come their way.  They’ve learned to tuck their heads down and do what they think works based on their own limited experience and inertia. 

RTI does nothing to address these barriers to instructional reform.  In addition, it does nothing to address the incentives that schools have to place students in special education.  In most states schools receive additional funding when a student is identified as disabled.  If a student is lagging academically and the school would have to devote some resources to helping that student catch-up, the school could either choose to say “my bad” and pay for those extra resources out of their existing budget, or they could say that the student is disabled and get additional money to help that student catch-up.  Of course, they have strong financial incentives to choose the latter explanation.  Research that I’ve done with Greg Forster and that Julie Cullen at UC San Diego has done, confirms that these positive financial incentives play a large role in the growth of special education.  That is, special education is growing, in large part, because we reward schools financially for increasing their special ed enrollment.

I know that many people claim that special education is a horrible financial burden on schools because it costs far more than the subsidies they receive.  But people who say this are either simply advocating for more subsidies or don’t properly understand what a “cost” is.  A cost is an expenditure that one would not otherwise make.  Simply showing that more is spent on special education students than subsidies received does not prove that the subsidy is less than the cost of identifying a student as disabled.  More is spent on students lagging academically whether they are identified as disabled or not. 

The positive financial incentive for identifying students as disabled exists when the subsidy is greater than the expenditure required by the special ed label beyond what would have been spent on that student anyway.  Because proper accounting is almost entirely absent in education, it is difficult to measure these additional costs directly.  But from the research showing the response to financial incentives, we know that there is often a financial reward for putting students in special education.

I don’t mean to suggest that educators are cynically gaming the school finance system or are even aware of its details.  My point is that the systems that school districts have adopted for the evaluation and identification of disabilities are shaped by these financial incentives so that even well-meaning practitioners will tend to over-identify disabilities when there are financial rewards for doing so.

Of course, RTI does nothing to address these financial incentives for increasing special ed enrollments.  In fact, it may contribute to those perverse incentives because schools are rewarded even more by placing more students in special education because they now get to divert 15% of that money for general education, which is essentially fungible.  And to make matters worse, diverting 15% of special education money away from disabled students may short-change truly disabled students who need those resources.

I’m sure that the people backing RTI are completely sincere in their confidence that we could prevent disabilities (and save money) if only we had proper instruction.  But wishing does not make that happen.  Reformers need to stay focused on combining promising instructional reforms with fixing the perverse incentive systems that undermine those instructional approaches. 

The incentive reforms should include changing the process by which we provide financial subsidies so that there are not strong rewards for over-identification of disabilities.  One way to do that is to provide vouchers for students with disabilities equal to the full value of what is spent on them in public schools.  That way schools would have to think twice before identifying a student as disabled.  Sure, they’ll get extra resources if they put a kid in special ed, but they also risk having that student walk out the door with all of his or her resources.  It places a check on perverse financial incentives. 

RTI with special ed vouchers could be a winning combination.  RTI by itself is just increasing federal subsidies for the status quo.

28 Comments | instructional reform, special ed, vouchers | Tagged: , , , , | Permalink
Posted by Jay P. Greene

Grad Rates Higher in Milwaukee Voucher Program

May 31, 2008

In case anyone missed the release of this study this week, Rob Warren of the University of Minnesota has a new study comparing high school graduation rates in Milwaukee’s voucher program and public schools.  The bottom line is that students graduate at much higher rates in the voucher program. 

Warren is careful to emphasize that he cannot draw causal inferences from this work.  That is, the voucher students graduate at higher rates than public students, but he can’t say whether the voucher program caused their higher graduation rate.  That kind of conclusion can only be drawn from a study that compares apples to apples.  With Pat Wolf I am involved in an evaluation that will be able to produce a graduation rate comparison of matched samples of voucher and public students, but results are still a few years down the road.

1 Comment | research reports, vouchers | Tagged: , , | Permalink
Posted by Jay P. Greene

Marion Barry endorses D.C. Opportunity Scholarships

May 13, 2008

(Guest Post by Matthew Ladner)

Former mayor and current D.C. Councilman Marion Barry endorsed the school voucher program today in the Washington Post:

I was fortunate that I could afford the right school for my son. As I have been in years past, I am focused today on those who most need help. We need to give the same opportunity to the District’s low-income parents, and this package would help ensure that all parents in our city have choices about where their children attend school.

Leave a Comment » | vouchers | Tagged: , , , | Permalink
Posted by matthewladner

Charles Murray vs. Michael Oher

May 10, 2008

(Guest Post by Matthew Ladner)

Michael Lewis’ book The Blind Side tells a fascinating story about poverty and education through the lens of football. Lewis focuses on two main stories. First, on the legendary coach Bill Walsh’s struggles in the 1980s to overcome the most fearsome defensive force of the era. Second, on an incredibly disadvantaged young man who beat the odds.

As head coach of the San Francisco 49ers Bill Walsh had one big problem: New York Giants linebacker Lawrence Taylor (LT). Attacking from the left–the blind side of a right-handed quarterback–LT humiliated linemen and punished quarterbacks with bone-crushing sacks.

Lewis’ tale becomes truly fascinating when he goes inside the world of the NFL’s talent search for born left tackles–a rare combination of size, speed and agility. These rare men would rise to become the second highest paid positions in professional football for their ability to protect the quarterback from men like LT. This is where the story intersects with education.

Michael Oher grew up in inner-city Memphis. In and out of foster care, Michael’s lucky break came when his dying grandmother extracted a promise from a family friend to get Michael into a private school.

Michael was enrolled in a private Christian school called Briarcrest. On a cold day, a parent of another Briarcrest student found Michael breaking into the school to stay warm. The parent, Leah Anne Tuohy, a successful interior designer and wife of a Memphis businessman, took Michael in. Despite the fact that Michael scarcely spoke, a bond developed between the Tuohys and Michael and they eventually adopted him.

Although he had never played sports Michael was a natural athlete and was identified immediately by college scouts as a potential NFL left tackle. If Michael could get to college and play football, he was very likely to win a multimillion dollar contract to protect a quarterback’s blind side.

The Tuohys and the faculty at Briarcrest engaged in a Herculean effort to make Michael eligible for college. When Michael came to Briarcrest he had only erratically attended school, could scarcely read and knew little about anything.

Lewis skillfully explains the role of poverty in education, writing, “Michael wasn’t stupid. He was ignorant, but a lot of people mistook ignorance for stupidity, and knowingness for intelligence. He’d been denied the life experience that led to knowingness, which every other kid at Briarcrest took for granted.”

Michael was not unintelligent, but he was profoundly uneducated. Leah Anne would, for example, take Michael to an Italian restaurant and order multiple meals in order teach him the difference between different types of pasta dishes.

The implications of Michael’s story for public policy are profound as well. Lewis writes, “Michael Oher was in possession of what had to be among the more conspicuous athletic gifts…and yet, without outside intervention even his talent would likely have been thrown away…If Michael Oher’s talent could be missed, whose couldn’t? Those poor black kids [in the inner-city] were like left tackles: people whose values were hidden in plain sight.”

With a committed family, school, and private tutors, Michael was accepted to college.

Today he is approaching his senior year at the University of Mississippi, made all-conference as a sophomore and junior, and carries a 3.7 grade point average.

Michael made it. But he is very much the exception. For every six inner-city Memphis public school kids with the athletic ability to play college sports, only one qualifies academically to attend college. This says something about the state of inner-city public education.

“Pity the kid inside Hurt Village [in Memphis] who was born to play the piano, or manage people, or trade bonds,” Lewis wrote. The success of Briarcrest in helping Michael exemplifies the hope that school choice can give to troubled youngsters.

The hole Michael dug himself out of might not have been so deep if not for the dysfunctional Memphis public school system. One cannot help but wonder if Memphis public schools would be so completely indifferent if every student had the opportunity to attend private schools.

Our current education system limits school choice to parents who can afford to buy homes in good neighborhoods or pay private school tuition. Our best teachers often flee the classroom in frustration, or cluster in suburbs far from the students who need them most.

Kids should not require Michael Oher’s incredible luck to make it. Neither should they be stuck in inner-city schools run for the benefit of the adults rather than the kids in the system.

So how does Charles Murray fit into this?

Murray knows far more about IQ testing than I do. I know next to nothing. From what I’ve read of Murray’s works, it does seem obvious that everyone has an upper threshold for academic achievement, a ceiling if you will, and that those ceiling vary from person to person.

It also seems obvious to me, however, that these ceilings are of little practical importance for many inner city children who have never attended a decent school, and who often have parents and grandparents who have never attended a decent school.

In other words, children like Michael Oher have been operating so far below their ceilings that we have every reason to radically improve our education system, especially in the inner cities. I’d even be willing to bet, despite Murray’s characterization of the academic literature on the subject, that if we had before and after adoption IQ tests on Oher, that there would have been substantial growth. I could be wrong about this, and I’d welcome correction, but Michael Oher’s experience begs the question in my mind exactly what it is that IQ tests are actually measuring.

Regardless of such concerns, however, it seems clear to me that efforts to make much more effective use of the huge and tragically mismanaged resources put into inner city schooling should be accelerated.

23 Comments | vouchers | Tagged: , , , , , | Permalink
Posted by matthewladner

« Previous Entries
Next Entries »