PJM Column by GF on BB (Just AAMOI)

August 22, 2008

(Guest post by Greg Forster)

Pajamas Media carries my take on BB today. I’m not as harsh on BB as Jay and Matt have been. I’m harsher.

Those of you who have been following the unions’ desperate attempt to distract you from the fact that BB doesn’t have the empirical evidence they claim it has by flinging a bunch of calumny at Jay may find this section of particular interest:

The really funny thing is, we’ve tried bringing social services into schools before. Fifty years ago, schools didn’t serve breakfast and provide teams of guidance counselors. Providing these and other social services in schools was originally justified on grounds that the kids needed these services to do well in school. How has that worked out?

Well, after all the empirical research that’s been done on schools, there’s no serious evidence that educational outcomes have improved as a result of these services. When the unions were challenged to come up with some evidence, they responded that “teachers know” these policies work.

But if the real purpose of providing these services in schools was to enlarge the government education blob, mission accomplished.

I wrote the column before the exchange about evidence and “cherry picking” over the last few days, but I see nothing that needs revision. As Jay pointed out, the evidence to which they now appeal is on the same scientific level as that used to prove the healing power of crystals. Their original “teachers know” argument was actually better – at least it didn’t pretend to be scientific.

Leave a Comment » | money myth, myth of helplessness, teacher union | Tagged: , , , | Permalink
Posted by Greg Forster

The Vitamin C of Education

August 20, 2008

Earlier this week I made my Modest Proposal for B.B. (Broader, Bolder or is it Buying Bananas?).  I noted that Randi Weingarten denounced vouchers as a waste of time despite considerable evidence supporting it, while she embraced the B.B. idea of community schools despite there being absolutely no evidence to support the claim that public schools could improve achievement by expanding their mission to include a host of social services.

Given the lack of evidence for B.B. I generously : ) offered to support a series of large pilot studies of the community schools approach, if Weingarten, Leo Casey, and the B.B. crowd would agree to a similar series of large pilot voucher programs as a way of learning more about both reform strategies.  No word yet but perhaps their internet is broken (just try unplugging it and plugging it back in).

Shital Shah from the Coalition for Community Schools, however, sent me a nice note with a link to a report claiming to contain the evidence supporting their approach.  After reviewing the report I still see virtually no evidence to give us confidence that public schools can increase student achievement by offering everything from legal assistance to health care.

In Appendix B the report lists 21 studies of the community school approach.  Seven of them have no student achievement outcomes.  Seven examine student test scores but only make pre/post comparisons without any control group.  And another seven have comparison groups but none employ random assignment, regression discontinuity, or another rigorous research design.  Four of those seven just compare achievement at schools using the B.B. approach to city or statewide averages.  And of the seven studies with some kind of control group, two find null effects, another finds null effects in math but not reading and even then only among schools with “high implementation” of the approach.  The quality (and quantity) of the evidence supporting community schools is no greater than what we could find to support the healing power of crystals. 

I understand why Randi Weingarten or Leo Casey would be pushing the educational equivalent of crystal healing.  Their job is to advocate for the interests of their union, not to make fair and reasonable assessments of research claims.  If schools expand their mission to include providing health care and other social services just think of all of the dues-paying nurses and social workers they could add to their rolls.

The greater mystery is why normally tough-minded and rigorous researchers, like Jim Heckman and Diane Ravitch, would sign on to this approach entirely lacking empirical support.  Heckman won the Nobel Prize for Economics for crying out loud.  But then again Linus Pauling won the Nobel Prize for Chemistry and later became a public advocate for mega doses of vitamin C to cure cancer, another intervention completely unsupported by rigorous evidence.

I’ll repeat that I am not against trying the B.B. community school approach with large pilot programs that are carefully studied.  I just can’t see why normally smart people would fully endorse untested approaches while ignoring other interventions, like expanding choice and competition in education, which have considerably more supporting evidence.

(edited for typos)

Leave a Comment » | myth of helplessness, vouchers | Tagged: , , , , , , , | Permalink
Posted by Jay P. Greene

A Modest Proposal for B.B.

August 18, 2008

The advocates of B.B. (Broader, Bolder; or is it Bigger Budgets? or is it Bloated Behemoth?) have yet to muster the evidence to support widespread implementation of their vision to expand the mission of schools to include health care, legal assistance, and other social services. They do present background papers showing that children who suffer from social problems fare worse academically, but they have not shown that public schools are capable of addressing those social problems and increasing student learning.

And if you dare to question whether there is evidence about the effectiveness of public schools providing social services in order to raise achievement, you are accused of being opposed to “better social and economic environments for children.” Right. And if you question the effectiveness of central economic planning are you also then opposed to a better economy? And if you question the effectiveness of an untested drug therapy are you then opposed to quality health-care?

To help the B.B. crowd generate the evidence one would need before pursuing a reform agenda on a large-scale, I have a modest proposal. How about if we have a dozen large-scale, well-funded pilot programs of the “community school” concept advocated by B.B.? And, at the same time let’s have a dozen large-scale, well-funded pilot voucher programs. We’ll carefully evaluate the effects of both to learn about whether one, the other, or both are things that we should try on an even larger scale.

I’m all for trying out new ideas and carefully evaluating the results. I can’t imagine why the backers of B.B. wouldn’t want to do the same. So as soon as Larry Mishel at the union-funded Economic Policy Institute, Randi Weingarten of the AFT, and Leo Casey of the AFT’s blog, Edwize, endorse my modest proposal, we’ll all get behind the idea of trying new approaches and studying their effects — “community schools” and vouchers.

Wait, my psychic powers are picking something up. I expect that some might say we’ve already tried vouchers and they haven’t worked. In fact, Randi Weingarten just wrote something very much like that when she declared in the NY Daily News that vouchers “have not been shown by any credible research to improve student achievement.” Let’s leave aside that there have been 10 random assignment evaluations (the gold-standard in research) of voucher programs and 9 show significant positive effects, at least for certain sub-groups of students. And let’s leave aside that 3 of those analyses are independent replications of earlier studies that confirm the basic positive findings of the original analyses (and 1 replication does not). And let’s leave aside that 6 of those 10 studies have been published in peer-reviewed journals (including the QJE, the Journal of the American Statistical Association, and the Journal of Policy Studies), three in a Brookings book, and one in a federal government report (even if Chris Lubienski somehow denies that any of this constitutes real peer-review). And let’s leave aside that there have been more than 200 analyses of the effects of expanding choice and competition, which Clive Belfield and Henry Levin reviewed and concluded: “A sizable majority of these studies report beneficial effects of competition across all outcomes… The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

Let’s leave all of that aside and ask Randi Weingarten how many random-assignment studies of the community school concept she has. Uhm, none. How many evaluations of community schools, period? Uhm, still none. But that doesn’t stop her from drawing the definitive conclusion: “Through partnerships with universities, nonprofit groups and other organizations, community schools provide the learning conditions and resources that support effective instruction and bring crucial services to an entire community.” How does she know?

But I’m eager to help her and all of us learn about community schools if she is willing to do the same to learn about vouchers. Better designed and better funded voucher programs could give us a much better look at vouchers’ full effects. Existing programs have vouchers that are worth significantly less than per pupil spending in public schools, have caps on enrollments, and at least partially immunize public schools from the financial effects of competition. If we see positive results from such limited voucher programs, what might happen if we could try broader, bolder ones and carefully studied the results?

And if community schools really deliver all that is being promised, great, let’s do that too. But if our goal is to do what works, why not give both ideas a real try?

(Link added)

10 Comments | vouchers | Tagged: , , , , , , , , | Permalink
Posted by Jay P. Greene

Broader, Bolder = Bloated Behemoth

August 13, 2008

 

Over at D-Ed Reckoning Ken DeRosa reviews the “evidence” that the AFT’s Leo Casey presents on the effectiveness of the Broader, Bolder approach being pushed by the union-backed Economic Policy Institute (with the support of some impressive people who you would think would know better). 

The issue is not whether kids would benefit from better health care or social services, or even whether receiving those benefits might contribute to higher achievement.  The issue is whether public schools are capable of expanding their mission to effectively provide these additional services, and whether those schools can translate the provision of additional services into higher achievement.

The Broader, Bolder folks provide a list of “background papers” to support their cause.  But those papers are very far in the background in that only a handful of the more than 100 studies cited actually assess the effects of providing students with additional services.  And even fewer look at the effects of public schools providing those services.  Before we endorse a bold new plan for education wouldn’t we want at least a few  evaluations of pilot programs in which public schools actually provided the full set of services being advocated?  I can’t find one such evaluation in the list of 100+ studies provided.

But don’t worry, Leo Casey has stepped into the breach with the solid research we need.  Here’s DeRosa’s commentary bracketing Casey’s, uhm, evidence:

“Leo must have had a few of his underlings pouring over the ERIC databases non-stop finding the requested evidence. Here is Leo’s evidence. I am leaving in all the internal citations and footnotes.

Classroom teachers recognize immediately the educational value of providing a comprehensive array of services to students living in poverty. They have seen the effects of undiagnosed and untreated eye problems on a student’s ability to learn how to read, and of untreated ear infections on a student’s ability to hear what is being said in the classroom. They know that the lack of proper medical care heightens the severity of childhood illnesses and makes them last longer, leading to more absences from school for students who need every day of school they can get. They have seen asthma reach epidemic proportions among students living in poverty, and they know that the lack of preventive and prophylactic medical care leads to more frequent attacks of a more severe nature, and more absences from school. They understand that screening for lead poisoning happens least among children in poverty, even though their living conditions make them the most likely victims, with all of the negative effects on cognitive functions. They know that the stresses of life in poverty make mental health and social work services for students and their families all that more important, and yet they are least likely to receive them. They see how the transience that marks poverty disrupts the education of students again and again, as the families of students are constantly on the move. In short, teachers know that the students living in poverty lack the health and social services routinely available to middle class and upper class students, despite the fact that they need them even more. And they know that the absence of these services has a detrimental impact on the education, as well as the general well-being, of students living in poverty.

I emphasized Leo’s evidentiary citations since they do not conform to the generally accepted norm.”

(edited for typos)

7 Comments | myth of helplessness | Tagged: , , , , , , | Permalink
Posted by Jay P. Greene

%d bloggers like this: