Jay P. Greene's Blog

(Guest Post by Matthew Ladner)

Next time Jay, make sure they include more cowbell.

As I’ve written before, every stripe of education charlatan has been cherry-picking PISA data to support whatever policies he or she prefers.  From Diane Ravitch’s obsession with imitating Finland to Marc Tucker’s divining of lessons from the “top performers,”  we’ve seen a host of causal claims attributed to the relationship between PISA results and particular practices or policies that are not causal at all.

Over on the Education Next blog, Matt Chingos has a brilliant piece demonstrating the relationship between Christmas spirit and student achievement. Matt even runs some regressions to “prove” his point — something that Diane Ravitch, Marc Tucker, and most other “best practices” gurus can’t or don’t bother doing.  Apparently, spending more on Christmas shopping “predicts” higher student achievement, controlling for some demographic factors.  Clearly, we don’t need smaller classes or better teacher-training to make schools better, we just need more Christmas spirit (or at least consumption).

This is why random-assignment and other research designs that more strongly identify causation are so important.  And this is why we should focus on random assignment research on private and charter choice  rather than the results of weaker research designs on those questions.

2013 has been a very good year.  In addition to having my piece with Brian Kisida and Dan Bowen about field trips to art museums as the most viewed and emailed piece in the Sunday New York Times, and having the research on which that was based as the most viewed piece in Education Next, I’ve now learned that one of my blog posts was the most read post on the Education Next site.

This most-viewed Ed Next blog post was one I wrote about whether high school athletic success comes at the expense of academic success.  It was based on an article that Dan Bowen and I wrote for the Journal of Research in Education.

A few other observations about these popular pieces:

• They are about art and athletics, not math and reading.  Education reformers (including myself) have gone too far in focusing narrowly on math and reading achievement scores, as if those were the only things about schools that matter.  As it turns out, people clearly think that the arts, athletics, and other things are also important and would like to read more articles about them.  I also think they would like schools and policymakers to pursue a diversity of goals and not just maximize math and reading achievement scores.
• Dan Bowen was co-author on both the art and athletics research projects.  Dan just graduated from our doctoral program in education policy and is currently a post-doc at Rice University.  Next year he’ll be back on the academic job market and I think having two #1 research projects won’t hurt.
• Department of Education Reform folks had 3 of the top 10 spots for blog posts and 4 of the top 20 articles in Education Next.  Way to go team!

In addition to being the most viewed and emailed article in the Sunday New York Times, my field trip research with Brian Kisida and Dan Bowen is the most read article in the journal, Education Next, during 2013.  The University of Arkansas may not have the #1 football or basketball teams (and “Well, I say let Harvard have its football and academics. Yale will always be first in gentlemanly club life“) but we are #1 in research on the effects of field trips to cultural institutions.  Go team!

I’ve been poking fun at people using Twitter for serious policy debates.  I acknowledge that Twitter is great for disseminating links, breaking news, and humor, but for conducting serious arguments Twitter has to be just about the dumbest thing on the planet (and I’m so dumb I periodically try to do it).

But there is something worse about trying to have serious policy debates on Twitter — it makes everyone come off as snarky and mean.  I’m sure I’m as guilty of this as anyone.  But there is no doubt that forcing communication in short, 140 character bursts coarsens debate and polarizes differences by removing subtlety and nuance.

There is an antidote to this corrosive effect of Twitter — meeting people in person.  AEI’s Rick Hess has a gathering a couple times a year of policy analysts, scholars, entrepreneurs, and practitioners.  Rick has an excellent ability to bring together top-notch people from a variety of backgrounds and perspectives.  I always learn something useful at these meetings, but much more importantly, I get to meet people whom I might otherwise only know through Twitter or blogs.

The vast majority of times, when I meet people in person with whom I’ve had run-ins on the internet I discover that these folks are good, friendly, and a pleasure to talk to.  I almost always discover that we don’t disagree on policy nearly as much as we thought when we were taking shots at each other on Twitter or the blogs.  Occasionally, I find that people who I liked and agreed with on the internet turn out to be jerks.  But, on average, in-person meetings greatly reduce the personal animosity and bickering promoted by the internet.

There is no going back on the increasing use of the internet for policy debates.  More information, more quickly disseminated, and more easily accessed are good things that people rightfully want.  But it comes at a price.  To reduce that price we should continue to invest in in-person gatherings, like the ones Rick hosts.

What do you see in this picture?  The new PISA results are out and education charlatans of every stripe are finding proof of their own preferred policy solution.

Dennis Van Roekel of the National Education Association sees addressing poverty as the solution: “The United States’ standings haven’t improved dramatically because we as a nation haven’t addressed the main cause of our mediocre PISA performance — the effects of poverty on students.”  There is some evidence for this, but the OECD analysis finds that student socioeconomic status only explains 15% of the variance in test results.  And according to the Wall Street Journal coverage, “Jack Buckley, the commissioner of the National Center for Education Statistics, noted that American students from families with incomes in the highest quartile did not perform as well as students with similar backgrounds in other countries.

According to the San Jose Mercury News, “‘None of the top-tier countries,’ said Randi Weingarten head of the American Federation of Teachers, ‘nor any of those that have made great leaps in student performance, like Poland and Germany, has a fixation on testing like the United States does.'”  Except many of the big gainers, particularly in Asia, do fixate on testing.

Best Practices guru, Marc Tucker, was on NPR this morning saying something about how “what you will find among the top performers” is that they”provide more resources to kids who are harder to educate than kids who are easier to educate.”  But in the San Jose Mercury News Tucker seems to suggest that more resources is not the solution when he asks, “Why are we not getting more bang for the buck?”  And on NPR Tucker credited Singapore’s success to “not just more teachers but better teachers.”  But the Wall Street Journal cites the OECD analysis, saying it “found a low connection between class size and test scores.”  And in the country Tautology Land better teachers are the ones who produce better scores.

It is possible to do credible social scientific analyses of international test scores if you do something like a regression that systematically examines variation in performance within and across countries controlling for other variables.  See for example work by Ludger Woessmann.  But just eyeballing the top performers and making up stories about why they succeeded based on picking and choosing characteristics about them is pure quackery.  As I’ve said before, best practices are the worst.

So, reach for your Duck Dynasty duck quacker and watch as folks make up stories about the picture above.  Personally, I see a cute little dog.