The TIMSS Rorschach Test

The Rorschach inkblot test is a psychology test that was used to assess personality and emotions.  The way in which people saw ambiguous images, like the one above, was supposed to say something about who they really were.

The same is true for the interpretations being applied to the results of the 2007 TIMSS (Trends in International Mathematics and Science Study) released today.

Over at Flypaper, Mike Petrilli interprets the gains the US has made in math but not science as suggesting that accountability testing is shifting resources toward math and away from science: “The lesson is that what gets tested gets taught. Under the No Child Left Behind act, and state accountability systems before that, elementary schools have been held accountable for boosting performance in math and reading. There is evidence that American elementary schools are spending less time teaching science, and this is showing up in the international testing data.”

And Mike interprets the relatively good results that Minnesota had (yes, MN took the test as if it were a country) as supporting rigorous standards: “There’s also good news out of Minnesota today, which has made dramatic gains since adopting new, more rigorous math standards.”

But also at Flypaper, Diane Ravtich offers different interpretations.  She sees the gains even in math results as “actually small, only four points.”  She also declines to credit NCLB for any of those gains, even as a perverse result of resource shifting away from science.  She notes that gains were at least as large in the US during the period prior to implementation of NCLB.  And on the topic of Minnesota she takes issue with Mikes explanation for success: “Minnesota showed dramatic gains on TIMSS not because of ‘new, more rigorous standards,’ but because of that state’s decision to implement a coherent grade-by-grade curriculum in mathematics.”  Umm, I would explain the difference but I got so bored trying to distinguish standards from curriculum that I dozed off for a bit.

Rather than focusing on the gains (or lack of gains) made by the US relative to itself in the past, Mark Schneider at Education Week focuses on the comparison between the US and other countries.  He notes that while the US looks relatively strong on the TIMSS, that is distorted by the large number of  “low-performing countries in the calculation of the international average [including Jordan, Romania, Morocco, and South Africa that] drives down that average, improving the relative performance of our students.”

He further notes that we fare worse on the PISA, which reports results from the 30 OECD countries who are our major trading partners and economic competitors: “We do better in TIMSS than we do on PISA, but this is a function of the countries that participate in each, and we should not let the relatively good TIMSS results lull us into a false sense of complacency. Even in the relatively easier playing field of TIMSS, we are lagging far too many countries in overall math performance and in the performance of our best students.”

And at Huffington Post Gerald Bracey was able to offer his reaction to the results last week, before they were released.  He wrote: “It might be good to keep a few things in mind when considering the data:

1. The Institute for Management Development rates the U. S. #1 in global competitiveness.

2. The World Economic Forum ranks the U. S. #1 in global competitiveness.

3. The U. S. has the most productive workforce in the world.

4. “The fact is that test-score comparisons tell us little about the quality of education in any country.” (Iris Rotberg, Education Week June 11, 2008).

5. ‘That the U. S., the world’s top economic performing country, was found to have schooling attainments that are only middling casts fundamental doubts on the value, and approach, of these surveys…'”

Bracey also said that our students could beat up the students in other countries with higher TIMSS scores.  (Actually, I made that last bit up.)

To summarize, Mike Petrilli sees evidence supporting his past concerns about the narrowing of the curriculum and the need for rigorous standards.  Diane Ravitch sees no evidence to alter her negative view of NCLB.  Mark Schneider, the former head of the National Center for Education Statistics, sees the need to review more testing.  And Gerald Bracey doesn’t even have to see the results to know that our education system is doing a great job.  And when I look at the inkblot I see a pudgy guy with a beard and male-patterned baldness laughing.

(edited for clarity)


7 Responses to The TIMSS Rorschach Test

  1. Brian Gottlob says:

    The productivity of any workforce is a a number of factors. The preferred measure, “multi-factor” productivity, is used a function of labor, capital, and what is generally referred to as technology but is really a residual that reflects the accumulated know how of a workforce. The U.S. has much deeper capital and has much more accumulated “know how” than many of the countries with which it is compared on educational tests and it is the capital and “know-how” that largely accounts for the higher productivity of the U.S. workforce. Emerging countries are becoming more capital intensive and with each year accumulated “know-how” increases. We will not be able to rely on the “U.S. has the most productive workforce” excuse for that much longer.

  2. Brian Gottlob says:

    …and there is perhaps no greater evidence of the failure of U.S. public schools (and universities) to teach grammar, spelling, and composition (not to mention editing), than my prior post.

  3. Patrick says:

    I had to take one of those as part of my testing for dyslexia and what not. I told the guy he was holding the picture upside down and he took me seriously and flipped it over.

    In the end, the guy told me I was emotionless – the irony was so immense, I could not even laugh.

  4. Patrick says:

    And I think our productivity is higher not because we score better on tests – we don’t – but because our economy is set up in a way to reward the most talented.

    At least it used to. Now that kids see you can get $700 billion for not only failing to generate a profit but for dragging down the entire economy with your failure. I’m sure our test scores will decline even further since we reward failure in education and in the private market now.

  5. kevin says:

    I read Bracey on Huffington and got excited about how he was trying to link educational outcomes to something MEANINGFUL in the real world rather than abstract test scores. But I don’t see the goal of public education as increasing worker productivity – thats too close to the goal of increasing test scores for me. I’m concerned with touchy-feely outcomes like increasing happiness, decreasing incarceration rates, health. When I look at the TIMMS scores of countries that do well in those areas those scores become irrelevant. Japan does well, but I’m not trying to live in Japan. If we start doing as well as Japan on TIMMS will we also start doing better of increasing stress and increasing the number of teenage suicide cults?
    I’d rather my students live. Look at Denmark – happiest people in the world – we schooled them . Really though? They’re still happier than we are.
    I do concede that we do have a lot to learn from Japan when it comes to teacher collaboration.

  6. What does TIMSS really measure? Does doing well on test scores mean real learning has taken place or that you just have an entrenched test prep culture? Kevin has made a good point about Japan. The same could be said about Singapore who again did pretty well in TIMSS. It has a very high stakes examinations-heavy school curriculum that has caused almost parents who can afford it, to hire an army of private tutors to ensure high test scores for their kids. Singapore success is due to this state of test preparedness. Is that what you want in the US?

  7. Julia Vida+ Hungarian researchers says:

    I would just like to say thank you for this post I like it very much! We did a study about the reception of PISA in Hungary and its impact on the Hungarian education policy discourse. One of our main findings was that PISA was an “empty container” which could be filled with content according to the interpreter’s interests and objectives. If interested,u can download our study from here along with other studies about PISA in other EU countries:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s