The Vitamin C of Education

August 20, 2008

Earlier this week I made my Modest Proposal for B.B. (Broader, Bolder or is it Buying Bananas?).  I noted that Randi Weingarten denounced vouchers as a waste of time despite considerable evidence supporting it, while she embraced the B.B. idea of community schools despite there being absolutely no evidence to support the claim that public schools could improve achievement by expanding their mission to include a host of social services.

Given the lack of evidence for B.B. I generously : ) offered to support a series of large pilot studies of the community schools approach, if Weingarten, Leo Casey, and the B.B. crowd would agree to a similar series of large pilot voucher programs as a way of learning more about both reform strategies.  No word yet but perhaps their internet is broken (just try unplugging it and plugging it back in).

Shital Shah from the Coalition for Community Schools, however, sent me a nice note with a link to a report claiming to contain the evidence supporting their approach.  After reviewing the report I still see virtually no evidence to give us confidence that public schools can increase student achievement by offering everything from legal assistance to health care.

In Appendix B the report lists 21 studies of the community school approach.  Seven of them have no student achievement outcomes.  Seven examine student test scores but only make pre/post comparisons without any control group.  And another seven have comparison groups but none employ random assignment, regression discontinuity, or another rigorous research design.  Four of those seven just compare achievement at schools using the B.B. approach to city or statewide averages.  And of the seven studies with some kind of control group, two find null effects, another finds null effects in math but not reading and even then only among schools with “high implementation” of the approach.  The quality (and quantity) of the evidence supporting community schools is no greater than what we could find to support the healing power of crystals

I understand why Randi Weingarten or Leo Casey would be pushing the educational equivalent of crystal healing.  Their job is to advocate for the interests of their union, not to make fair and reasonable assessments of research claims.  If schools expand their mission to include providing health care and other social services just think of all of the dues-paying nurses and social workers they could add to their rolls.

The greater mystery is why normally tough-minded and rigorous researchers, like Jim Heckman and Diane Ravitch, would sign on to this approach entirely lacking empirical support.  Heckman won the Nobel Prize for Economics for crying out loud.  But then again Linus Pauling won the Nobel Prize for Chemistry and later became a public advocate for mega doses of vitamin C to cure cancer, another intervention completely unsupported by rigorous evidence.

I’ll repeat that I am not against trying the B.B. community school approach with large pilot programs that are carefully studied.  I just can’t see why normally smart people would fully endorse untested approaches while ignoring other interventions, like expanding choice and competition in education, which have considerably more supporting evidence.

(edited for typos)


A Modest Proposal for B.B.

August 18, 2008

The advocates of B.B. (Broader, Bolder; or is it Bigger Budgets? or is it Bloated Behemoth?) have yet to muster the evidence to support widespread implementation of their vision to expand the mission of schools to include health care, legal assistance, and other social services. They do present background papers showing that children who suffer from social problems fare worse academically, but they have not shown that public schools are capable of addressing those social problems and increasing student learning.

And if you dare to question whether there is evidence about the effectiveness of public schools providing social services in order to raise achievement, you are accused of being opposed to “better social and economic environments for children.” Right. And if you question the effectiveness of central economic planning are you also then opposed to a better economy? And if you question the effectiveness of an untested drug therapy are you then opposed to quality health-care?

To help the B.B. crowd generate the evidence one would need before pursuing a reform agenda on a large-scale, I have a modest proposal. How about if we have a dozen large-scale, well-funded pilot programs of the “community school” concept advocated by B.B.? And, at the same time let’s have a dozen large-scale, well-funded pilot voucher programs. We’ll carefully evaluate the effects of both to learn about whether one, the other, or both are things that we should try on an even larger scale.

I’m all for trying out new ideas and carefully evaluating the results. I can’t imagine why the backers of B.B. wouldn’t want to do the same. So as soon as Larry Mishel at the union-funded Economic Policy Institute, Randi Weingarten of the AFT, and Leo Casey of the AFT’s blog, Edwize, endorse my modest proposal, we’ll all get behind the idea of trying new approaches and studying their effects — “community schools” and vouchers.

Wait, my psychic powers are picking something up. I expect that some might say we’ve already tried vouchers and they haven’t worked. In fact, Randi Weingarten just wrote something very much like that when she declared in the NY Daily News that vouchers “have not been shown by any credible research to improve student achievement.” Let’s leave aside that there have been 10 random assignment evaluations (the gold-standard in research) of voucher programs and 9 show significant positive effects, at least for certain sub-groups of students. And let’s leave aside that 3 of those analyses are independent replications of earlier studies that confirm the basic positive findings of the original analyses (and 1 replication does not). And let’s leave aside that 6 of those 10 studies have been published in peer-reviewed journals (including the QJE, the Journal of the American Statistical Association, and the Journal of Policy Studies), three in a Brookings book, and one in a federal government report (even if Chris Lubienski somehow denies that any of this constitutes real peer-review). And let’s leave aside that there have been more than 200 analyses of the effects of expanding choice and competition, which Clive Belfield and Henry Levin reviewed and concluded: “A sizable majority of these studies report beneficial effects of competition across all outcomes… The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

Let’s leave all of that aside and ask Randi Weingarten how many random-assignment studies of the community school concept she has. Uhm, none. How many evaluations of community schools, period? Uhm, still none. But that doesn’t stop her from drawing the definitive conclusion: “Through partnerships with universities, nonprofit groups and other organizations, community schools provide the learning conditions and resources that support effective instruction and bring crucial services to an entire community.” How does she know?

But I’m eager to help her and all of us learn about community schools if she is willing to do the same to learn about vouchers. Better designed and better funded voucher programs could give us a much better look at vouchers’ full effects. Existing programs have vouchers that are worth significantly less than per pupil spending in public schools, have caps on enrollments, and at least partially immunize public schools from the financial effects of competition. If we see positive results from such limited voucher programs, what might happen if we could try broader, bolder ones and carefully studied the results?

And if community schools really deliver all that is being promised, great, let’s do that too. But if our goal is to do what works, why not give both ideas a real try?

(Link added)


Follow

Get every new post delivered to your Inbox.

Join 940 other followers