Pay No Attention to the Research Consensus Behind the Curtain

April 6, 2017

rN8STVb.gif

(Guest post by Greg Forster)

Noah Smith dresses up a few fussy methodological quibbles and one big, really dishonest bit of fakery in order to cast aspersions on my Win-Win report and distract you from the research consensus behind the curtain.

My report reviewed over 100 empirical findings on private school choice programs, showing that there is a very strong research consensus in favor of positive effects from such programs. Smith identifies two (2) cases where he thinks I ought to have used a different method to classify the findings. I disagree, but frankly, it’s not worth quibbling about. The research consensus in favor of school choice is still clear even if we were to accept Smith’s cavails.

His statement that “vouchers have generally disappointed” is totally unsupported by the evidence – and if he read my report, he knows it.

But his big, dramatic “gotcha!” is that I allegedly omit a well-known study with a null finding. That would indeed be a serious omission.

Unfortunately for Smith, the study he dramatically accuses me of omitting is not a study of private school choice. Here is the abstract with emphasis on Smith’s dishonesty added:

School choice has become an increasingly prominent strategy for enhancing academic achievement. To evaluate the impact on participants, we exploit randomized lotteries that determine high school admission in the Chicago Public Schools. Compared to those students who lose lotteries, students who win attend high schools that are better in a number of dimensions, including peer achievement and attainment levels. Nonetheless, we find little evidence that winning a lottery provides any systematic benefit across a wide variety of traditional academic measures. Lottery winners do, however, experience improvements on a subset of nontraditional outcome measures, such as self-reported disciplinary incidents and arrest rates.

From the very first sentence, Smith explicitly frames his whole article as an article about private school choice. For him to accuse me of omitting a study on private school choice because I omitted this study is dishonest.

Smith owes me an apology and a retraction. If he refuses, Bloomberg owes me a correction.

I’ll hold my breath waiting.


Trump and School Choice

December 14, 2016

img_1358

(Guest post by Greg Forster)

I was grateful to be included in this Washington Post article on Trump and school choice yesterday. My post on Trump’s racism and illiberalism gets a mention, but the Post is right that another division is also important:

Free-market purists believe that parents know best, that they can choose the best schools for their children without intervention, something that could force poor-quality schools to close. On the other end of the spectrum are those who believe that intensive oversight and regulation are necessary to ensure that the schools from which parents are choosing are high-quality.

As long as Mike is taking his lumps out in the wild, wild west of Arizona, maybe he could rethink which side of this unavoidable civil war – unavoidable because opponents of parent choice have made it so – he really wants to be on.

Another point: I don’t blame the Post for describing advocates of parent choice as “free-market purists” while describing opponents of parent choice more neutrally. It is we in the parent choice camp who have chosen to make deep investments in “free market” ideological rhetoric. Everything we’re saying about markets is in fact true, but it’s a bad idea for us to make “markets” and “competition” the main points in favor of choice.

This was one of the main arguments of my recent series on “the next accountability.” As I wrote at the end of the series:

Markets and competition as drivers of efficiency and performance are important. But they do not provide the moral norms and narratives needed to inform the next accountability. The best case for universal school choice does not center on them. These should be secondary, not primary themes.

We should develop ways of articulating these principles as the basis of the next accountability:

  • The purpose of education is to help children develop the knowledge, skills and virtues they need to live a good life—achieving and appreciating the true, good and beautiful—and to live as good citizens of a community where we disagree about what is good.
  • To cultivate these, we need teachers who are wise professionals (possessing the qualities they seek to instill, and guided by an independent professional ethic) and schools that are free communities (where shared purpose, not the arbitrary dictates of distant authorities, shape a shared life).
  • Teachers and schools can educate the individual student for free pursuit of the good life as he or she sees it, and also for good citizenship and respect for others’ rights in a diverse community, because of what we share in common as human beings and as fellow Americans.
  • Teachers and schools should be held accountable to do this by parents and local communities—the more local the better—because they are in the closest moral and social connection to schools, and can therefore hold them accountable in ways that support their social fabric rather than disrupting it.

Is this too much to ask of a highly polarized education reform movement, strongly committed to moral narratives that center on either markets or test scores? I’m looking forward to finding out.


Fun With Peer Review

December 9, 2014

PHD Comics

(Guest post by Greg Forster)

I may have to revise my opinion of Vox; they seem to have taken an interest in the weaknesses of the peer review system. Of course there are a lot of responsible peer-reviewed journals and, well, peers. But there a lot of the other kind as well, and we are long past the point where simply having gone through something called “peer review” ought to count for anything.

One story details how unscrupulous researchers can manipulate journals, including – amazingly – posing as their own reviewers. In highly specialized fields, journal editors may not know who the appropriate reviewers would be, so they rely – apparently uncritically in some cases – on the “recommended reviewers” supplied by the article authors. Who in some cases are simply the authors themselves using another email address. One scientist used 130 email accounts to create a vast, self-validating “peer review and citation ring”; 60 papers were recently retracted after a 14-month investigation uncovered the fraud. A total of at least 110 articles have been pulled in the last two years due to this type of fraud.

Get me off your email list

Figure 1 from the article “Get Me Off Your Fucking Mailing List”

Accepted for publication by the highly reputable International Journal of Advanced Computer Technology

But the other story is a lot better. It details how some journals now survive not by selling subscriptions or getting institutional support, but by charging a fee to publish your paper. They are apparently known as “predatory journals” because they spam the email universe looking for gullible (or, presumably, unscrupulous) people looking to break into publication. “Article mills” (after the analogous “diploma mills”) would seem a more appropriate name.

As you can see above, the “peer review” process becomes somewhat lax in these cases. One pair of scientists slapped the above-referenced article and began submitting it to peer review spammers. They were amused to discover that one journal accepted their article for publication. Another journal not only accepted but published an article (consisting of nonsense text) by Maggie Simpson and Edna Krabappel. It now sends the authors regular demands that they pay their $459 bill.

But it’s not just spam scammers – peer review controls are easy to get past even at some highly reputable publishers.


Are We Allowed to Be Neither Naive Nor Cynical?

July 31, 2013

balance
(Guest post by Greg Forster)

I have a question. Am I permitted to be neither naïve nor cynical about the Tony Bennett emails? Or is there some sort of law that dictates I must be one or the other? Indiana StateImpact places me with the Bennett supporters while Matt seems to think I’m attacking Bennett (I’m not sure how else to interpret “haters gonna hate”). I don’t intend to be either.

I find it difficult to buy the new house line, and I will continue to find it difficult until someone asks Bennett the obvious question: “If this was a glitch in the system, as we are now being told, why did you seek to change the grade only for this one school?” Rick Hess didn’t ask him that question. Matt seems uninterested in asking it, and seems to think I’m a “hater” for asking it. Until that question is answered, I don’t see why I’m a “hater” for pointing out uncomfortable realities.

Is it really so scandalous, does it really make me a “hater,” to acknowledge the obvious fact that politicians are responsive to their donors? When government sets educational standards and has to do what Bennett himself calls a “face validity” test, it is going to know which schools are run by major donors and it is going to be sensitive to that fact. Good grief, are we this naïve?

What we have now is not “the rest of the story” but a failure to seek the rest of the story. Or am I somehow missing something?

On the other hand, Ze’ev and others seem to think I’m saying all standards are arbitrary and there’s no such thing as a rational public consensus. I’m not; I’m just trying to be realistic about what I called “the sausage-making nature of the process” when those standards are being cooked up behind closed doors by a government bureaucracy and its political allies, as opposed to standards that emerge organically from the give and take of a thriving marketplace of options. Technology standards emerge in the context of a system dominated by consumer choice. Educational standards should emerge in the same way.


My Own Personal Narcissus Index

April 19, 2013

John-Stossel

(Guest post by Greg Forster)

Picking back up on our discussion of narcissism, I’m sure you’re all dying to know that my new Win-Win report was featured on John Stossel last night! While you other losers here on JPGB have been wasting your time on Twitter, I’m finally getting the undivided attention of millions that I’ve always known I deserved.

Oh, wait, sorry – I didn’t mean to bash Twitter, because . . . Stossel also tweeted my study. Twitter’s totally cool now!

0035 rotated square
In case you forgot what I look like.

Seriously, I’m always grateful when people bring attention to my work. Stossel highlighted the numbers for impact on public schools: 23 empirical studies have looked at how school choice impacts academic outcomes in public schools, of which 22 found a positive effect and one found no visible difference; no empirical study has ever found a negative impact. He also mentioned the numbers for racial segregation: eight studies, seven positive, one neutral; none negative. (Stossel’s description may have left viewers thinking those public school academic effect studies were participant effect studies – I know it’s hard to do justice to the details in the short time TV allows, but at least I can note the difference here.)

Hope others are finding the report useful – that unbroken line of zeros in the “negative effects” column can’t be publicized too widely!


Gerson Cites Voltron

April 4, 2013

voltron team (original)

(Guest post by Greg Forster)

As long as we’re on the subject of narcissism here on JPGB, may I note that Michael Gerson quoted our Voltron op-ed in Monday’s Washington Post?

But even small, restricted choice programs have shown promising results — not revolutionary but promising. Last year a group of nine leading educational researchers summarized the evidence this way: “Among voucher programs, random-assignment studies generally find modest improvements in reading or math scores, or both. Achievement gains are typically small in each year, but cumulative over time. Graduation rates have been studied less often, but the available evidence indicates a substantial positive impact. . . . Other research questions regarding voucher program participants have included student safety, parent satisfaction, racial integration, services for students with disabilities, and outcomes related to civic participation and values. Results from these studies are consistently positive.”

I’d tweet about it, but I’m too cool for Twitter.


A Good Media Week

March 29, 2013

This week the work of two of my colleagues in the Department of Education Reform was mentioned in national newspapers.  Patrick Wolf’s research finding that the Milwaukee voucher program increased high school graduation rates from 75% to 94% was mentioned in the Wall Street Journal.  And Bob Maranto’s work on the decline in New York City’s murder rate as a result of more effective policing was mentioned in David Brooks’ column in the New York Times.

Way to go!