The Wagner Epic Continues

No, not that Wagner.  There is more on Tony Wagner, the snake-oil salesman educational consultant.  My op-ed on Wagner ran in the Northwest Arkansas Morning News.  I’ve also reprinted the text below, since it is easier to read that than the scanned pdf in the link.

The first community discussion on Wagner’s book, The Global Achievement Gap, was held last nightIt wasn’t too bad.  A number of teachers (at tables other than mine) expressed resentment at the suggestion that they weren’t already aware that critical thinking and creativity were desirable.  But administrators and GT teachers seemed more enamored with the book.  And the reaction from parents and community members included a fair degree of skepticism. 

It’s hard to get people to think critically about people who push a focus on critical thinking.  To be for critical thinking is like being for goodness and light.  The tricky part is in how you get there.  To the extent that Wagner has any concrete suggestions, he seems to be taking folks down the wrong path.  He wants less emphasis on content and less testing.  But he shows no evidence that higher levels of critical thinking can be found in places or at times when there was less content and less testing.  In fact, the little evidence he does provide would suggest the opposite.

Some smart folks are pushing back against these data-free educational consultants.  Sandra Stotsky had an op-ed on Wagner last weekDan Willingham had an excllent blog post on Alfie Kohn as did Stuart Buck.  And Robert Pondiscio at Core Knowledge ,  AndyRotherham at Eduwonk , and Ken De Rosa at D-Ed Reckoning have added their two cents (which, with the new stimulus package, will become 2 trillion cents).

So here is my op-ed pasted below:

Fayetteville Public Schools Need Evidence, Not Snake-Oil (submitted title)
By Jay P. Greene

              The Fayetteville Public Schools purchased 2,000 copies of Tony Wagner’s The Global Achievement Gap and organized a series of public fora to discuss how that book might guide our schools.  The District is to be commended for engaging the community in this process.  But it is unclear why the District selected Wagner’s book as the focus of this discussion.

Wagner’s book makes claims about what skills students really need to learn, what is blocking them from learning those skills, what countries are more successful in teaching these skills, and what some schools are doing to remedy the problem.  But he provides no systematic evidence to substantiate any one of these claims.  In short, the book is a series of anecdotes that more closely resembles what one would find in a self-help manual than in a work of social science.  If we apply our critical thinking skills, which Wagner says are essential, we should reject this book as a sound basis for planning the future of Fayetteville schools.

First, Wagner says there are seven essential survival skills that our children need to learn.  How does he know that these are the essential skills?  He chatted with a CEO on an airplane and selected a few more to interview.  Does he review any research on the types of skills that predict who will become successful adults?  No. Wagner relies upon the authority of his experience and the experiences of a handful of corporate executives to identify the essential skills.  Accepting claims on this basis would be the sort of thing we would hope people with critical thinking skills might reject.

Frankly, the seven skills he lists — critical thinking, collaboration, adaptability, initiative, communication, analysis, and imagination – seem reasonable enough, but they are also so vague as to be unhelpful in informing schools about what to do.  How exactly do we produce critical thinking or adaptability or creativity?  It’s not as if educators have been unaware of these goals, but they haven’t generally been effective at developing strategies to achieve them.

Then Wagner identifies what he believes is blocking the acquisition of these seven essential skills – high stakes testing.  What evidence does he present to support this claim?  Again, he presents no systematic evidence to demonstrate that there is a tradeoff between the content knowledge required in accountability testing and the essential skills he wants.  Couldn’t it be the case that improving mastery of basic skills and content knowledge provides the foundation for these seven skills?  It’s hard to be imaginative, analytical, etc… without knowing subject matter and basic skills of literacy and numeracy.  Einstein may have said “imagination is more important than knowledge,” as the book’s dedication indicates, but Einstein couldn’t have succeeded without a firm grasp of advanced mathematics.

If Wagner were right that accountability testing undermines essential skills, then surely these skills must have been more plentiful before testing became as salient as it is today.  But Wagner does not (and cannot) provide any evidence to show that.  Instead, he shows (on p. 74) that students in the United States significantly lag students in Finland, Hong Kong-China, Japan, and Korea in certain problem solving skills on an international test called PISA.  As it turns out, high stakes testing is extremely prominent in most of these countries with strong problem-solving results – a fact curiously at odds with Wagner’s claims.  If accountability testing undermines essential skills, why do countries with such strong accountability systems manage to succeed so well in teaching the essential skills Wagner wants?

Wagner describes three model schools that he says have been effective at teaching essential skills (although he again fails to provide any evidence that they are as successful as he claims).  But it is by no means clear that the approaches adopted by these three schools are the only valid approaches or that they could be replicated easily by others.  Replication is especially problematic because the three models he provides are all charter schools or alternative schools of choice.  Perhaps the secret of these schools’ success has something to do with school choice and not the features he describes.  If true, it’s not clear how Fayetteville could imitate the success of these schools.

To achieve our goals in education we have to adopt approaches backed by systematic evidence.  If we believe critical thinking, collaboration, and creativity are the most important goals for schools, then we need systematic evidence on systems of teacher preparation, curriculum, and pedagogy that effectively produce those goals.  There is a growing body of scientific research on these issues, including a number of studies sponsored by the U.S. Department of Education’s Institute for Education Sciences, that the Fayetteville Public Schools might wish to consider rather than consulting with the latest peddler of educational snake-oil.

10 Responses to The Wagner Epic Continues

  1. Jay, I think that this is a real problem, and obviously a systemic one. There needs to be some mechanism of quality control, somewhere. I can’t see a solution that entails some self-appointed experts passing judgment on everything that comes down the pike, but the current system is not working.

  2. There are two problems here. First, as you note, we get lots of bad and/or distorted data that muddies decision making and allows people like Wagner and Kohn to misdirect people. The second problem, one that may not be immediately obvious, is that it reduces public support.

    I’ve talked with several businesspeople over the years who are, or have been, involved in public education, and I get two things from them. One is a sense of futility, because they want to help improve outcomes but don’t trust any of the data (with good reason); ultimately this drives some people out entirely.

    The other thing I see is a desire to avoid controversy. A company is going to go to great lengths to avoid controversy in its public giving and support, and when they see the raging battles over how to teach reading and math, for example, they stay out – why pour in resources when you’re going to tick off half the people involved?

    I don’t know what the solution is either – we can’t rely on the community to sort it out for themselves.

  3. I agree with you, Dan, that the problem is systemic. And I agree that centralized panels of experts cannot be the solution. Even the What Works Clearinghouse, which was started with the best of intentions, fails because the process is ultimately politicized and meaningful, consistent standards are impossible to invent or implement. In the end, quality is a matter of judgment.

    So how do other applied fields effectively apply judgment to sort out competing research claims? Basically, you want the people making decisions about which research to implement to experience the consequences of their decisions. If the there are competing research claims about the best manufacturing process for semi-conductors, the firms that choose wisely will be rewarded and those that choose poorly will suffer. Ultimately we need choice and competition or some other method of attaching consequences to decisions to incntive education officials to choose wisely among research claims.

    • Greg Forster says:

      I have one bone to pick with you here, Jay. You write that if the success of Wagner’s three model schools is due to school choice, it’s not clear how Fayetteville could imitate that success.

      How about by offering students a city-funded school voucher, with the funding for the voucher offset by the reduction in locally-borne costs for the government school system as students leave it? If the city needed a bridge loan to provide financing to get over the gap between paying for the voucher and realizing the cost savings, it could probably find a philanthropist willing to do that.

      Better yet, pay for the program by deciding that next year you’re not buying 2,000 copies of a snake oil book and then sponsoring a bunch of public discussions about it. How much are they spending on these public discussion events, anyway?

  4. […] Greene goes after the education guru on his blog and in an op-ed in the Northwest Arkansas Morning News.  The Fayetteville Public School system […]

  5. […] Jacobs highlights Sandra Stotsky, the critic mentioned in Jay Greene’s blog and quoted here. Jay Greene continues the discussion and posts his editorial, with Bugs Bunny imagery and bonus Dan Willingham comment.  Dan Willingham […]

  6. That Fayetteville Public Schools is investing so much in Wagner’s words highlights one of the biggest problems with moving quality research to practice: that quality research is often not translated in a way that speaks to public school administrators. Thus, they end up utilizing information that sounds nice but doesn’t contain hard empirical data. One important move that researchers must make is to involve those close to practice and policy – both administrators and policymakers – to help them translate good research for those that might have a chance to apply it.

  7. […] Jay Greene piles on here and here, arguing that Wagner “shows no evidence that higher levels of critical thinking can be found […]

  8. Dave W. says:

    I just read the book, and while I have some disagreements with Wagner’s take on the effects of technology on student motivation, I am in full agreement that what we currently call “education” amounts to 12 years of mostly mind numbing rote memorization with no context or connection or relevance to students lives. As a teacher of over 20 years, I happen to agree that our curriculum is too much guided by what best serves textbook publishers and bureaucrats, and by a misplaced belief that the goal is to do well on tests that test passive memorization. We are pumping out students who hate school, hate learning, and yet think they are bright because they got all Bs (because the pressure is on to pass students so that schools look good and everyone keeps their job). You ask for evidence to back all this up; but it seems that what you really want is a certain type of evidence. Anecdotal evidence is evidence; as is history, and personal experience. Did you decide, for example, how to raise your children by conducting controlled experiments, evaluating spreadsheets and analyzing a budget? Or did you, like the rest of us, follow the wisdom and experience of your parents and friends, combined with some of your own (unclinically derived and tested) intuition and preferences? “Hard data,” as we might call it, is not always available or appropriate or even possible for certain aspects of life. But despite this, there is plenty of hard data cited in Wagner’s book, including actual questions from national standardized “accountability” tests we give our students, questions from the tests used in countries who are beating us in measures of entrepreneurialism and high-tech innovation, and rankings of student achievement which show the US losing ground to countries we recently considered “third world.” It is not true, as you imply, that Wagner blames this all on standardized tests. He cites a number of factors and conditions in these other countries to account for their outcomes; however, he sees our use of multiple choice tests as one of the most pernicious threats to the quality of education here in our country, and therefor focuses more on that topic.

    Your accusation of “snake oil salesman” falls pretty flat. It might possibly make sense (but I doubt it) applied to Wagner’s motives, but it doesn’t explain why the many other individuals he spoke to would agree to have their thoughts and words in his book. They earn no royalties for being quoted. If you doubt the truth or accuracy of Wagner’s observations or those of the business leaders he interviewed, on what do you base these doubts? My hunch–since you provide no other kind of evidence for this–is that they are based on your own prejudices, preconceptions and assumptions–i.e. anecdotal experiences. (But that’s just my hunch–I can’t back it up with research!)

    • Dave W. says:

      Just read your bio and CV. Impressive. Now I understand your need to discredit people like Wagner.

Leave a Reply to Dave W. Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: