Obama and Universities, the Love is Mutual

August 28, 2008

OpenSecrets.org has a new analysis of campaign donations that confirms the earlier analyses I did here and here.  Universities are a significant source of financial support for Barak Obama.

They write: “Educators contributed at least $2.3 million to his campaign in June and July, surpassed only by lawyers, who make up Obama’s top-giving industry since the campaign’s start, and retirees. Nine of Obama’s top 25 contributors—based on contributions from employees and their families—are universities: University of California, Harvard, Columbia, Stanford, University of Michigan, Georgetown, University of Chicago, University of Washington and University of Pennsylvania, in descending order.”

(HT John J. Miller at Phi Beta Cons)

Leave a Comment » | higher education | Tagged: , , , , , , | Permalink
Posted by Jay P. Greene

Eduwonkette or Jennifer Jennings, Makes No Difference

August 26, 2008

I’m glad that Eduwonkette decided to end her anonymity and identify herself as Jennifer Jennings, a sociology graduate student at Columbia University.  I’m not glad because I think it was inherently wrong for her to blog anonymously.  As I’ve previously written: “The issue is not who Eduonkette is, but whether she is right or not.  Knowing who she is does not make her evidence or arguments any more or less compelling.” 

The problem was that Eduwonkette did not share my belief in the principle that we should focus on the veracity rather than the source of claims.  She repeatedly emphasized the credibility of the source of information.  Emphasizing the credibility of sources while blogging anonymously, preventing analysis of her own credibility, was logically untenable and had to end.  I wished that she would end this inconsistency by embracing the view that ideas are true or false independent of their sources.  Instead she has resolved her inconsistency by ending her anonymity.

Now that we know that Eduwonkette is Jennifer Jennings we can see another prominent example of the logical inconsistency of blogging anonymously while focusing on the credibility of sources.  In 2005 Jennings published an article in the American Educational Research Journal that argued that accountability systems encouraged schools to focus on the achievement of “bubble” students — those close to an achievement cutoff — at the expense of high and low achieving students.  She arrived at this conclusion after visiting a school in Texas and observing it for a period of time.  She was aided in drawing this conclusion by using jargon like “neoinstituionalism” and “normative isomorphism,” but I kind of zoned out during that part of the article.  I’m guessing that neoinstitutionalism is bad following my theory that anything starting with “neo” is supposed to be bad while anything starting with “post” is supposed to be good.

A few years later Matthew Springer published articles in Education Next and in the Economics of Education Review that empirically examine Jennings’ claim of “educational triage.”  The Education Next piece actually began with a lengthy quote from Jennings’ article as a foil for its findings that NCLB accountability improved the achievement of “bubble” students, but not at the expense of lower and higher achieving students.

Jennings then took-on her critic, Springer, but she did so as the anonymous blogger Eduwonkette, never revealing that she was attacking the person who criticized her own research.  And her first argument against Springer was that his work was published in Education Next and “Education Next is not a scholarly journal.”  Jennings targets the source of the critique of her own work while concealing her identity to prevent analysis of her as a source.  The irony is too rich.

Why Jennings did not just focus on the version of the paper published in Economics of Education Review or the unabridged version linked to on Education Next, , which would have been free of the unscholarly taint she perceives in Education Next,  is unclear.  It was obviously important for her to discredit Springer’s argument against her own study by attacking the credibility of Education Next as the source of Springer’s argument — all the while preventing assessment of her credibility by doing all of this anonymously.

Let me be clear that I have no problem with Jennings defending her own work anonymously.  Her arguments against Springer are true or false regardless of who she says or doesn’t say she is.  My point is that by arguing her own case anonymously, Jennings betrays the principles that she appears to endorse.  Namely, if the source of information is important in assessing claims, it would clearly be inappropriate to attack your critic without revealing who you are. 

Even now that Jennings has revealed her identity, I hope that she abandons her reliance on assessing the source of claims.  Doing so would justify her past actions and help us move forward in analyzing ideas rather than analyzing people and motives.

6 Comments | school accountability | Tagged: , , , , | Permalink
Posted by Jay P. Greene

Systemic Effects of Vouchers

August 25, 2008

In an earlier post I listed all analyses of the effects of U.S. vouchers on program participants using random-assignment experiments.  Those studies tell us about what happens to the academic achievement of students who receive vouchers.  But we all recognize that expanding choice and competition with vouchers may also have significant effects on students who remain in traditional public schools.  Here is a brief summary of the research on that question.

In general, the evidence on systemic effects (how expanding choice and competition affects the performance of traditional public schools) has more methodological limitations than participant effects studies.  We haven’t been able to randomly assign school districts to increased competition, so we have more serious problems with drawing causal inferences.  Even devising accurate measures of the extent of competition has been problematic.  That being said, the findings on systemic effects, like on participant effects, is generally positive and almost never negative.

Even in the absence of choice programs traditional public schools are exposed to some amount of competition.  They may compete with public schools in other districts or with nearby private schools.  A relatively large number of studies have examined this naturally occurring variation in competition.  To avoid being accused of cherry-picking this evidence I’ll rely on the review of that literature conducted by Henry Levin and Clive Belfield.  Here is the abstract of their review, in full:

“This article systematically reviews U.S. evidence from cross-sectional research on educational outcomes when schools must compete with each other. Competition typically is measured by using either the HerfindahlIndex or the enrollment rate at an alternative school choice. Outcomes are academic test scores, graduation/attainment, expenditures/efficiency, teacher quality, students’ post-school wages, and local housing prices. The sampling strategy identified more than 41 relevant empiricalstudies. A sizable majority report beneficial effects of competition, and many report statistically significant correlations. For each study, the effect size of an increase of competition by one standard deviation is reported. The positive gains from competition are modest in scope with respect to realistic changes in levels of competition. The review also notes several methodological challenges and recommends caution in reasoning from point estimates to public policy.”

There have also been a number of studies that have examined the effect of expanding competition or the threat of competition on public schools from voucher programs in Milwaukee and Florida.  Here are all of the major studies of systemic effects of which I am aware from voucher programs in the US:

Milwaukee

Martin Carnoy, et al “Vouchers and Public School Performance,” Economic Policy Institute, October 2007;

Rajashri Chakrabarti, “Can Increasing Private School Participation and Monetary Loss in a Voucher Program Affect Public School Performance? Evidence from Milwaukee,” Federal Reserve Bank of New York, 2007; (forthcoming in the Journal of Public Economics)

Caroline Minter Hoxby, “The Rising Tide,” Education Next, Winter 2001;

Jay P. Greene and Ryan H. Marsh, “The Effect of Milwaukee’s Parental Choice Program on Student Achievement in Milwaukee Public Schools,” School Choice Demonstration Project Report, March 2009.

Florida

Rajashri Chakrabarti “Vouchers, Public School Response and the Role of Incentives: Evidence from Florida“ Federal Reserve Bank of New York Staff Report, Number 306, October 2007;

Jay P. Greene and Marcus A. Winters, “Competition Passes the Test,” Education Next, Summer 2004;

Cecilia Elena Rouse, Jane Hannaway, Dan Goldhaber, and David Figlio, “Feeling the Heat: How Low Performing Schools Respond to Voucher and Accountability Pressure,” CALDER Working Paper 13, Urban Institute, November 2007;

Martin West and Paul Peterson, “The Efficacy of Choice Threats Within School Accountability Systems,” Harvard PEPG Working Paper 05-01, March 23, 2005; (subsequently published in The Economic Journal, March, 2006)

Jay P. Greene and Marcus A. Winters, “The Effect of Special Education Vouchers on Public School Achievement: Evidence From Florida’s McKay Scholarship Program”  Manhattan Institute, Civic Report Number 52, April 2008. (looks only at voucher program for disabled students)

Cassandra Hart and David Figlio, “Does Competition Improve Public Schools?” Education Next, Winter, 2011.

Every one of these 10 studies finds positive systemic effects.  It is importantto note that Rouse, et al are ambiguous as to whether they attribute the improvements observed to competition or to the stigma of Florida’s accountability system.  The other four Florida studies perform analyses that support the conclusion that the gains were from competitive pressure rather than simply from stigma.

Also Carnoy, et al confirm Chakrabarti’s finding that Milwaukee public schools improved as the voucher program expanded, but they emphasize that those gains did not continue to increase as the program expanded further (nor did those gains disappear).  They find this lack of continued improvement worrisome and believe that it undermines confidence one could have in the initial positive reaction from competition that they and others have observed.  This and other analyses using different measures of competition with null results lead them to conclude that overall there is a null effect  — even though they do confirm Chakrabarti’s finding of a positive effect.

I would also add that Greg Forster and I have a study of systemic effects in Milwaukee and Greg has a new study of systemic effects from the voucher program in Ohio.  And Greg also has a neat study that shows that schools previously threatened with voucher competition slipped after Florida’s Supreme Court struck down the voucher provision.  All of these studies also show positive systemic effects, but since they have not undergone external review and since I do not want to overstate the evidence, I’ve left them out of the above list of studies.  People who, after reading them, have confidence in these three studies should add them to the list of studies on systemic effects.

The bottom line is that none of the studies of systemic effects from voucher programs finds negative effects on student achievement in public schools from voucher competition.  The bulk of the evidence, both from studies of voucher programs and from variation in existing competition among public schools, supports the conclusion that expanding competition improves student achievement.

(Updated 3/3/11 to include the new Florida study)

4 Comments | competitive effects, vouchers | Tagged: , , , | Permalink
Posted by Jay P. Greene

Voucher Effects on Participants

August 21, 2008

(This is an update of a post I originally wrote on August 21.  I’ve included the new DC voucher findings.)

Here is what I believe is a complete (no cherry-picking) list of analyses taking advantage of random-assignment experiments of the effect of vouchers on participants.  As I’ve previously written, 9 of the 10 analyses show significant, positive effects for at least some subgroups of students.

All of them have been published in peer reviewed journals or were subject to outside peer review by the federal government.

Four of the 10 studies are independent replications of earlier analyses.  Cowen replicates Greene, 2001.  Rouse replicates Greene, Peterson, and Du.  Barnard, et al replicate Peterson and Howell.  And Krueger and Zhu also replicate Peterson and Howell.  All of these independent replications (except for Krueger and Zhu) confirm the basic findings of the original analyses by also finding positive effects.

Anyone interested in a more complete discussion of these 10 analyses and why it is important to focus on the random-assignment studies, should read Patrick Wolf’s article in the BYU Law Review that has been reproduced here.

I’m eager to hear how Leo Casey and Eduwonkette, who’ve accused me of cherry-picking the evidence, respond.

  • These 6 studies conclude that all groups of student participants experienced reading or math achievement gains and/or increased likelihood of graduating from high school as a result of vouchers:

Cowen, Joshua M.  2008. “School Choice as a Latent Variable: Estimating the ‘Complier Average Causal Effect’ of Vouchers in Charlotte.” Policy Studies Journal 36 (2).

Greene, Jay P. 2001. “Vouchers in Charlotte,” Education Matters 1 (2):55-60.

Greene, Jay P., Paul E. Peterson, and Jiangtao Du. 1999. “Effectiveness of School Choice: The Milwaukee Experiment.” Education and Urban Society, 31, January, pp. 190-213.

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E. Peterson. 2002. “School Vouchers and Academic Performance:  Results from Three Randomized Field Trials.” Journal of Policy Analysis and Management, 21, April, pp. 191-217. (Washington, DC: Gains for all participants, almost all were African Americans)

Rouse, Cecilia E. 1998. “Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program,” The Quarterly Journal of Economics, 113(2): 553-602.

Wolf, Patrick, Babette Gutmann, Michael Puma, Brian Kisida, Lou Rizzo, Nada Eissa, and Marsha Silverberg. March 2009.  Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years. U.S. Department of Education, Institute of Education Sciences. Washington, DC: U.S. Government Printing Office. (In the fourth year report the sample size shrunk so that the positive achievement effect barely missed meeting a strict threshold for statistical significance — p < .06 just missing the bar of p < .05.  But this new report was able for the first time to measure the effect of vouchers on the likelihood that students would graduate high school.  As it turns out, vouchers significantly boosted high school graduation rates.  As Paul Peterson points out, this suggests that vouchers boosted both achievement and graduation rates in the 4th year.  Read the 4th year evaluation here.)

  • These 3 studies conclude that at least one important sub-group of student participants experienced achievement gains from the voucher and no subgroup of students was harmed:

Barnard, John, Constantine E. Frangakis, Jennifer L. Hill, and Donald B. Rubin. 2003. “Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City,” Journal of the American Statistical Association 98 (462):299–323. (Gains for African Americans)

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E. Peterson. 2002. “School Vouchers and Academic Performance:  Results from Three Randomized Field Trials.” Journal of Policy Analysis and Management, 21, April, pp. 191-217. (Dayton, Ohio: Gains for African Americans)

Peterson, Paul E., and William G. Howell. 2004. “Efficiency, Bias, and Classification Schemes: A Response to Alan B. Krueger and Pei Zhu.” American Behavioral Scientist, 47(5): 699-717.  (New York City: Gains for African Americans)

This 1 study concludes that no sub-group of student participants experienced achievement gains from the voucher:

Krueger, Alan B., and Pei Zhu. 2004. “Another Look at the New York City School Voucher Experiment,” The American Behavioral Scientist 47 (5):658–698.

(Update: For a review of systemic effect research — how expanded competition affects achievement in traditional public schools — see here.)

35 Comments | vouchers | Tagged: , , , , , , , , , , , , , , | Permalink
Posted by Jay P. Greene

False Claims of Cherry Picking are the Pits

August 20, 2008

Leo Casey over at Edwize is urging me to join the “United Cherry Pickers” union because he thinks I’ve cherry picked the evidence on vouchers in a previous post.  This sounds like a great deal if my dues, like those from AFT and NEA members, can contribute to paying for skyboxes for Leo and his buddies at the Democratic National Convention to make-up for the convention’s shortfall of $10 million.  Where do I sign up?

Making a charge of cherry picking is easy.  Substantiating it requires, well, uhm, evidence.  Evidence isn’t exactly Leo Casey’s strong-suit.

I said that there have been 10 analyses of random assignment voucher experiments.  I said that 9 of those 10 analyses show significant, positive effects (at least for some subgroups).  If I am cherry picking, which random assignment analyses am I leaving out? 

Leo Casey then asserts: “Serious research conducted by respected scholars without an ideological axe to grind has consistently found every major voucher experiment in the United States wanting. John Witte’s and Cecilia Rouse’s definitive analyses of the Milwaukee voucher program and the Indiana University studies of the Cleveland voucher program have shown no meaningful educational performance advantage for students in those two high profile, large scale voucher programs.”

Neither Witte nor the IU studies analyzed random-assignment experiments, making it harder to have confidence in their results, which is why I focus on the 10 analyses using the gold-standard approach. 

Rouse’s study did examine a random-assignment experiment, but Casey mischaracterizes her findings.  She writes: “I find that students in the Milwaukee Parental Choice Program had faster math score gains than, but similar reading score gains to, the comparison groups. The results appear robust to data imputations and sample attrition, although these deficiencies of the data should be kept in mind when interpreting the results.”   Remember, Casey falsely claims that she finds “no meaningful educational performance advantage for students.”

Casey also mischaracterizes my citation of Belfield and Levin’s findings: “[He even cites research that is not on the subject of vouchers: Hank Levin will be most surprised to learn that his research ‘supports’ vouchers.]” 

Since I actually bothered to quote Belfied and Levin’s findings about the effects of expanding choice and competition, I don’t think Hank Levin will be the least bit surprised to read what he wrote.  I’ll repeat the quotation here so that no one is shocked: “A sizable majority of these studies report beneficial effects of competition across all outcomes… The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

If Leo Casey is going to make the charge of cherry picking and improperly citing evidence, he has to deliver proof of those charges.  To the contrary, the facts indicate that Casey is the one cherry picking and improperly citing research.

Is there a union for playing fast and loose with the truth?  Maybe Leo Casey should join it.  Oh, I forgot.  He’s already a member of the AFT.

(Links added)

8 Comments | vouchers | Tagged: , , , , | Permalink
Posted by Jay P. Greene

The Vitamin C of Education

August 20, 2008

Earlier this week I made my Modest Proposal for B.B. (Broader, Bolder or is it Buying Bananas?).  I noted that Randi Weingarten denounced vouchers as a waste of time despite considerable evidence supporting it, while she embraced the B.B. idea of community schools despite there being absolutely no evidence to support the claim that public schools could improve achievement by expanding their mission to include a host of social services.

Given the lack of evidence for B.B. I generously : ) offered to support a series of large pilot studies of the community schools approach, if Weingarten, Leo Casey, and the B.B. crowd would agree to a similar series of large pilot voucher programs as a way of learning more about both reform strategies.  No word yet but perhaps their internet is broken (just try unplugging it and plugging it back in).

Shital Shah from the Coalition for Community Schools, however, sent me a nice note with a link to a report claiming to contain the evidence supporting their approach.  After reviewing the report I still see virtually no evidence to give us confidence that public schools can increase student achievement by offering everything from legal assistance to health care.

In Appendix B the report lists 21 studies of the community school approach.  Seven of them have no student achievement outcomes.  Seven examine student test scores but only make pre/post comparisons without any control group.  And another seven have comparison groups but none employ random assignment, regression discontinuity, or another rigorous research design.  Four of those seven just compare achievement at schools using the B.B. approach to city or statewide averages.  And of the seven studies with some kind of control group, two find null effects, another finds null effects in math but not reading and even then only among schools with “high implementation” of the approach.  The quality (and quantity) of the evidence supporting community schools is no greater than what we could find to support the healing power of crystals. 

I understand why Randi Weingarten or Leo Casey would be pushing the educational equivalent of crystal healing.  Their job is to advocate for the interests of their union, not to make fair and reasonable assessments of research claims.  If schools expand their mission to include providing health care and other social services just think of all of the dues-paying nurses and social workers they could add to their rolls.

The greater mystery is why normally tough-minded and rigorous researchers, like Jim Heckman and Diane Ravitch, would sign on to this approach entirely lacking empirical support.  Heckman won the Nobel Prize for Economics for crying out loud.  But then again Linus Pauling won the Nobel Prize for Chemistry and later became a public advocate for mega doses of vitamin C to cure cancer, another intervention completely unsupported by rigorous evidence.

I’ll repeat that I am not against trying the B.B. community school approach with large pilot programs that are carefully studied.  I just can’t see why normally smart people would fully endorse untested approaches while ignoring other interventions, like expanding choice and competition in education, which have considerably more supporting evidence.

(edited for typos)

Leave a Comment » | myth of helplessness, vouchers | Tagged: , , , , , , , | Permalink
Posted by Jay P. Greene

Why Aren’t We Already Doing It?

August 19, 2008

Last week Charles Murray made an innovative and provocative proposal in the Wall Street Journal:  Let’s not make the college B.A. the standard training for all professions and higher-skill occupations.  Instead, let’s increasingly use certification tests, like the CPA for accountants, to indicate whether people are qualified for certain jobs.  People could prepare for those tests by taking classes from traditional colleges, by taking courses on-line, by studying on their own in the neighborhood library, or in any other way they want.  Expanding the paths by which people could enter high-paying occupations expands equality of opportunity by reducing the financial and logistical barriers that requiring a college B.A. imposes.  And focusing career-oriented training on the skills required for that career and removing other requirements improves the efficiency of that training. 

In this scenario we would have to rely on the K-12 system to provide the basic liberal arts training and civic education we think all people need to be productive citizens.  Post-secondary education would be more focused on career training except for the relatively few people who really want further liberal arts training or additional preparation in a traditional academic field.

Expanding equality of opportunity and improving the efficiency of post-secondary education make Murray’s proposal very appealing.  So appealing that one has to ask why we aren’t already doing it.  The government does not mandate that employers of professionals and high-skilled occupations require a college B.A. — at least not directly.  Why do employers require something that limits the pool of qualified labor from which they could hire and consumes considerably more time and resources than the certification test approach Murray suggests?  If Murray’s proposal is on target, shouldn’t employers already be developing and using certification tests in lieu of the B.A., at least for certain occupations?

There are several reasons why Murray’s vision is not the current reality.  First, developing appropriate certification tests for a number of high-skill occupations may not be as easy as Murray suggests.  Perhaps many employers have not switched from the B.A. model because they don’t think they can meaningfully improve upon it. 

Second, employers and private associations that develop and use certification tests would likely face a flood of employment discrimination lawsuits that challenged the validity of the test.  I am no lawyer (nor do I play one on TV), but I suspect that fear of litigation plays a large role in deterring employers from relying on private certification tests for hiring. 

Third, much of what employers want from their employees (at least in some occupations) is the self-discipline and obedience to authority necessary for completing assigned tasks.  Perhaps employers don’t care too much about what prospective employees do in college as long as they have to complete a long list of assigned tasks demonstrating self-discipline and compliance.  College selects and cultivates these desired traits.

Fourth, even if employers just want to use college as a liability-free way to screen for self-discipline and compliance, it is clear that this is a very inefficient arrangement from a societal perspective.  It would be much more efficient to have employers hire crowds of interns/apprentices at low wages and only keep the most skilled, self-disciplined, and compliant as long-term employees.  While this may be more efficient from a societal perspective, it is far less efficient from the perspective of individual employers.  If they had to sort and train a crowd of prospective employees as interns/apprentices, employers would have to bear the costs that students and taxpayers currently bear in paying colleges to perform these roles.  And having to dump a large number of interns/apprentices who didn’t make the cut would invite another flood of employment discrimination lawsuits.

Murray’s vision has much appeal, but there are also significant barriers to its implementation.  Unless we address those barriers, especially liability issues related to employment, we are unlikely to realize the benefits of expanding equality of opportunity and reducing costs from Murray’s proposal.  There are also practical barriers, like the difficulty of testing for certain skills and traits, that limit the benefits that could be realized.  But we could still take steps toward what Murray has suggested and think about how those ideas could shape reform within the post-secondary education world.

10 Comments | higher education | Tagged: , , | Permalink
Posted by Jay P. Greene

A Modest Proposal for B.B.

August 18, 2008

The advocates of B.B. (Broader, Bolder; or is it Bigger Budgets? or is it Bloated Behemoth?) have yet to muster the evidence to support widespread implementation of their vision to expand the mission of schools to include health care, legal assistance, and other social services. They do present background papers showing that children who suffer from social problems fare worse academically, but they have not shown that public schools are capable of addressing those social problems and increasing student learning.

And if you dare to question whether there is evidence about the effectiveness of public schools providing social services in order to raise achievement, you are accused of being opposed to “better social and economic environments for children.” Right. And if you question the effectiveness of central economic planning are you also then opposed to a better economy? And if you question the effectiveness of an untested drug therapy are you then opposed to quality health-care?

To help the B.B. crowd generate the evidence one would need before pursuing a reform agenda on a large-scale, I have a modest proposal. How about if we have a dozen large-scale, well-funded pilot programs of the “community school” concept advocated by B.B.? And, at the same time let’s have a dozen large-scale, well-funded pilot voucher programs. We’ll carefully evaluate the effects of both to learn about whether one, the other, or both are things that we should try on an even larger scale.

I’m all for trying out new ideas and carefully evaluating the results. I can’t imagine why the backers of B.B. wouldn’t want to do the same. So as soon as Larry Mishel at the union-funded Economic Policy Institute, Randi Weingarten of the AFT, and Leo Casey of the AFT’s blog, Edwize, endorse my modest proposal, we’ll all get behind the idea of trying new approaches and studying their effects — “community schools” and vouchers.

Wait, my psychic powers are picking something up. I expect that some might say we’ve already tried vouchers and they haven’t worked. In fact, Randi Weingarten just wrote something very much like that when she declared in the NY Daily News that vouchers “have not been shown by any credible research to improve student achievement.” Let’s leave aside that there have been 10 random assignment evaluations (the gold-standard in research) of voucher programs and 9 show significant positive effects, at least for certain sub-groups of students. And let’s leave aside that 3 of those analyses are independent replications of earlier studies that confirm the basic positive findings of the original analyses (and 1 replication does not). And let’s leave aside that 6 of those 10 studies have been published in peer-reviewed journals (including the QJE, the Journal of the American Statistical Association, and the Journal of Policy Studies), three in a Brookings book, and one in a federal government report (even if Chris Lubienski somehow denies that any of this constitutes real peer-review). And let’s leave aside that there have been more than 200 analyses of the effects of expanding choice and competition, which Clive Belfield and Henry Levin reviewed and concluded: “A sizable majority of these studies report beneficial effects of competition across all outcomes… The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

Let’s leave all of that aside and ask Randi Weingarten how many random-assignment studies of the community school concept she has. Uhm, none. How many evaluations of community schools, period? Uhm, still none. But that doesn’t stop her from drawing the definitive conclusion: “Through partnerships with universities, nonprofit groups and other organizations, community schools provide the learning conditions and resources that support effective instruction and bring crucial services to an entire community.” How does she know?

But I’m eager to help her and all of us learn about community schools if she is willing to do the same to learn about vouchers. Better designed and better funded voucher programs could give us a much better look at vouchers’ full effects. Existing programs have vouchers that are worth significantly less than per pupil spending in public schools, have caps on enrollments, and at least partially immunize public schools from the financial effects of competition. If we see positive results from such limited voucher programs, what might happen if we could try broader, bolder ones and carefully studied the results?

And if community schools really deliver all that is being promised, great, let’s do that too. But if our goal is to do what works, why not give both ideas a real try?

(Link added)

10 Comments | vouchers | Tagged: , , , , , , , , | Permalink
Posted by Jay P. Greene

Che Studies

August 17, 2008

The Arizona Republic’s Doug MacEachern has a column today on the Raza Studies program in Tucson, Arizona.  Raza Studies is part of their Ethnic Studies program in Tucson public high schools emphasizing Latino history and pride.  But the particular way in which Tucson’s program does this has raised some critical scrutiny.  MacEachern writes:

The ethnic-studies directors make a great many claims that teeter over into the wrong side of truth.  They claim not to “teach” communism, socialism or Marxism in their classes. But they lionize Marxist revolutionaries like “Che” Guevara; they all but worship Marxist education theorist Paolo Friere; and they have developed entire lesson plans celebrating modern Marxists like Subcomandante Marcos, the southern Mexican Zapatista who considers himself a “postmodern Che.” But they don’t “teach” the stuff.

The directors of the program “humbly and respectfully welcome the scrutiny and spotlight” their program has attracted, but then denounce “the tyrannical and fascist perspectives that are held and espoused by our adversaries.”

To defend their program, the directors have produced what one local paper called nine “cohort studies,” which the school district claims show that Raza Studies has a positive effect on the high school graduation rate and state achievement test scores of the students who elect to participate in the program.  MacEachern sent the “studies” to me for my comment.  They were actually just a few bar graphs making simple comparisons between the outcomes of students who did and did not choose to participate in Raza Studies at some (but not all) of Tuscon’s high schools.  There is no way to know from a few bar graphs whether Raza Studies helped, hurt, or had no effect on student achievement since the self-selected group of students who chose to take Raza Studies may have already been higher achieving at the beginning.  A few bar graphs does not an evaluation — or nine cohort studies —  make.

3 Comments | civic values, politics | Tagged: , , , | Permalink
Posted by Jay P. Greene

Broader, Bolder = Bloated Behemoth

August 13, 2008

 

Over at D-Ed Reckoning Ken DeRosa reviews the “evidence” that the AFT’s Leo Casey presents on the effectiveness of the Broader, Bolder approach being pushed by the union-backed Economic Policy Institute (with the support of some impressive people who you would think would know better). 

The issue is not whether kids would benefit from better health care or social services, or even whether receiving those benefits might contribute to higher achievement.  The issue is whether public schools are capable of expanding their mission to effectively provide these additional services, and whether those schools can translate the provision of additional services into higher achievement.

The Broader, Bolder folks provide a list of “background papers” to support their cause.  But those papers are very far in the background in that only a handful of the more than 100 studies cited actually assess the effects of providing students with additional services.  And even fewer look at the effects of public schools providing those services.  Before we endorse a bold new plan for education wouldn’t we want at least a few  evaluations of pilot programs in which public schools actually provided the full set of services being advocated?  I can’t find one such evaluation in the list of 100+ studies provided.

But don’t worry, Leo Casey has stepped into the breach with the solid research we need.  Here’s DeRosa’s commentary bracketing Casey’s, uhm, evidence:

“Leo must have had a few of his underlings pouring over the ERIC databases non-stop finding the requested evidence. Here is Leo’s evidence. I am leaving in all the internal citations and footnotes.

Classroom teachers recognize immediately the educational value of providing a comprehensive array of services to students living in poverty. They have seen the effects of undiagnosed and untreated eye problems on a student’s ability to learn how to read, and of untreated ear infections on a student’s ability to hear what is being said in the classroom. They know that the lack of proper medical care heightens the severity of childhood illnesses and makes them last longer, leading to more absences from school for students who need every day of school they can get. They have seen asthma reach epidemic proportions among students living in poverty, and they know that the lack of preventive and prophylactic medical care leads to more frequent attacks of a more severe nature, and more absences from school. They understand that screening for lead poisoning happens least among children in poverty, even though their living conditions make them the most likely victims, with all of the negative effects on cognitive functions. They know that the stresses of life in poverty make mental health and social work services for students and their families all that more important, and yet they are least likely to receive them. They see how the transience that marks poverty disrupts the education of students again and again, as the families of students are constantly on the move. In short, teachers know that the students living in poverty lack the health and social services routinely available to middle class and upper class students, despite the fact that they need them even more. And they know that the absence of these services has a detrimental impact on the education, as well as the general well-being, of students living in poverty.

I emphasized Leo’s evidentiary citations since they do not conform to the generally accepted norm.”

(edited for typos)

7 Comments | myth of helplessness | Tagged: , , , , , , | Permalink
Posted by Jay P. Greene

« Previous Entries
Next Entries »