Jay P. Greene's Blog

(Guest post by Barry Garelick)

In Jay Greene’s recent blog post, “The Dead End of Scientific Progressivism,” he points out that Vicki Phillips, head of education at the Gates Foundation misread her Foundation’s own report.  Jay’s point was that Vicki continued to see what she and others wanted to see: “‘Teaching to the test makes your students do worse on the tests.’ Science had produced its answer — teachers should stop teaching to the test, stop drill and kill, and stop test prep (which the Gates officials and reporters used as interchangeable terms).”

I was intrigued by the education establishment’s long-held view as Jay paraphrased it.  This view has become one of the “enduring truths” of education and I have heard it expressed in the various classes I have been taking in education school the last few years.  (I plan to teach high school math when I retire later this year).  In terms of math education, ed school professors distinguish between “exercises” and “problems”.  “Exercises” are what students do when applying algorithms or routines they know and can apply even to word problems. Problem solving, which is preferred, occurs when students are not able to apply a mechanical, memorized response, but rather have to apply prior knowledge to solve a non-routine problem.  Moreover, we future teachers are told that students’ difficulty in solving problems in new contexts is evidence that the use of “mere exercises” or “procedures” is ineffective and they are overused in classrooms.  One teacher summed up this philosophy with the following questions: “What happens when students are placed in a totally unfamiliar situation that requires a more complex solution? Do they know how to generate a procedure?  How do we teach students to apply mathematical thinking in creative ways to solve complex, novel problems? What happens when we get off the ‘script’?”

As someone who learned math largely though mere exercises and who now creatively applies math at work, I have to question this thinking. I believe that students’ difficulty in solving new problems is more likely to be because 1) applying prior knowledge to new or non-routine problems is hard for everyone, and 2) it is even harder for students who may lack the requisite knowledge and/or mastery of skills—not because they possess such knowledge and mastery.   So while the educationists distinguish between “exercises” and “problems”, the view refuses to distinguish between novices and experts.

Daniel Willingham, a cognitive scientist who teaches at the University of Virginia, finds the distinction between novice and expert to be quite important.  He maintains that it takes time and effort for knowledge to accumulate to the point that connections between learned material and new and difficult problems can be made.  Willingham refers to the difficulty that novices have with thinking critically as “inflexible thinking.”  In fact, he characterizes such difficulty as perfectly normal and to be expected among students.  Willingham argues that understanding the deep structures of a discipline such as mathematics is an important goal of education, “but if students fall short of this, it certainly doesn’t mean that they have acquired mere rote knowledge and are little better than parrots.” Rather, they are making the small steps necessary to develop better mathematical thinking. Simply put, no one leaps directly from novice to expert.

I was therefore extremely interested to see the sample problem of the Balanced Assessment in Mathematics (BAM) in Appendix 1 of the Gates Foundation’s preliminary report. The BAM is to be used to assess teacher effectiveness and according to the Gates Foundation’s preliminary report, “In comparison to many other assessments, BAM is considered to be more cognitively demanding and measures higher order reasoning skills using question formats that are quite different from those in most state mathematics achievement tests. There is also some evidence that BAM is more instructionally sensitive to the effects of reform-oriented instruction than a more traditional test (ITBS).”

The sample problem is reproduced below and is from the 8th grade mathematics assessment:

The diagram makes it clear how one is to count the number of tiles needed, so the first question is relatively easy.  Question 2 requires more thought, and the student must be able to extend the counting algorithm for the 4ft pond in question 1 to ponds of other sizes; i.e., they must understand that 4 is added to the product of the number of one foot long squares that fit on one side of the pond times 4, or 4n + 4 where n is the length of one side of the square pond in feet.

There are four additional questions that become increasingly difficult:  “How many paving stones are needed to surround a fish pond that is 20 feet by 20 feet?”, “Chris has 48 paving stones.  Find the size of the largest square pond the paving stones can surround.”  “The garden center sells many different sizes of square fish ponds. Write down a rule that will help Chris figure out how many paving stones are needed to surround square  ponds of different sizes.”  “The garden center decides to sell rectangular ponds.  Find a rule that will help Chris figure out how many paving stones are needed to surround rectangular ponds of different sizes.”

This set of questions is fairly challenging to beginning algebra students. (It is even more challenging if they have had no algebra at all, but since this is a problem for 8th grade students, I am assuming that they have had some experience with algebraic expressions and equations. This is not always a safe assumption but that’s a topic for another article.) The sample problem is illustrative of the type of problem that the education community deems coach-proof since it appears that memorization of problem solving techniques and “drill and kill” exercises will not work here.  But in fact, practice with some exercises would help students in tackling such a problem—specifically, having students express in mathematical terms certain situations described in English.  For example, “Three more than two times what John’s age will be in five years” (3 + 2(x + 5) ).  These types of exercises are frequently deemed by the education establishment to be inauthentic and irrelevant to the deeper underlying concepts of math unlike “reform oriented instruction” which purportedly provides such deep understanding through so-called authentic problems and a minimum of “exercises”.

Interestingly, the TIMSS exam—an international exam given every three years—also contains questions of this sort, as well as more straightforward problems.  For comparison’s sake, I looked at TIMSS eighth grade questions in algebra (found at http://nces.ed.gov/timss/pdf/TIMSS8_Math_ConceptsItems_2.pdf ) and found a similar type of problem:

This problem requires students to understand and ultimately express the relationship between the number of smaller squares on each side of the larger square, and the number of triangles contained in the square.  Question C, in fact, requires the student to be able to express the relationship mathematically in order to calculate the number of triangles.  The TIMSS report in which this appears provides some interesting data related to this question; namely the percent of students taking the test in each country that obtained the correct answer:

The top five scoring countries for this question ranged from 44 to 49% correct.  For the more straightforward problems the top five scores tend to be in the 70 and 80 percent range.   The US students obtained 19 percent correct for this problem; on more straightforward problems, the US scores in the 50 and 60 percent range. Thus, for all students, regardless of country, non-routine problems prove to be difficult.  Of interest, however, is that the five top scoring countries for this particular problem are Asian,  frequently criticized for using drilling and “inauthentic” exercises which they maintain do not properly prepare students for solving non-routine problems.

If Vicki Phillips’ statement about teaching to the test is any indication, however, the educational establishment will see what they want to see.  They will likely proclaim that the higher scores obtained by Asian countries on non-routine problems serves as evidence that the Asian countries use “reform-oriented instruction”.  Either that, or they’ll shrug their shoulders and say “It’s the culture; what can you do?”  (See http://www.educationnews.org/commentaries/104502.html )

In any event, whether the education establishment realizes it or not, the new generation of coach-proof tests that will be used to evaluate teachers, appear to be measuring the skills students are expected to be learning.  And by teaching what should be taught, teachers are teaching to the test, whether the Gates Foundation and its look-alikes realize it, like it, or not.

BIO: Barry Garelick is an analyst for a federal agency and is cofounder of the U.S. Coalition for World Class Math. (http://usworldclassmath.webs.com/ )  He plans to teach math later this year.

This entry was posted on Sunday, January 23rd, 2011 at 8:52 am and is filed under instructional reform. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.