(Guest post by Barry Garelick)

In Jay Greene’s recent blog post, “The Dead End of Scientific Progressivism,” he points out that Vicki Phillips, head of education at the Gates Foundation misread her Foundation’s own report. Jay’s point was that Vicki continued to see what she and others wanted to see: “‘Teaching to the test makes your students do worse on the tests.’ Science had produced its answer — teachers should stop teaching to the test, stop drill and kill, and stop test prep (which the Gates officials and reporters used as interchangeable terms).”

I was intrigued by the education establishment’s long-held view as Jay paraphrased it. This view has become one of the “enduring truths” of education and I have heard it expressed in the various classes I have been taking in education school the last few years. (I plan to teach high school math when I retire later this year). In terms of math education, ed school professors distinguish between “exercises” and “problems”. “Exercises” are what students do when applying algorithms or routines they know and can apply even to word problems. Problem solving, which is preferred, occurs when students are not able to apply a mechanical, memorized response, but rather have to apply prior knowledge to solve a non-routine problem. Moreover, we future teachers are told that students’ difficulty in solving problems in new contexts is evidence that the use of “mere exercises” or “procedures” is ineffective and they are overused in classrooms. One teacher summed up this philosophy with the following questions: “What happens when students are placed in a totally unfamiliar situation that requires a more complex solution? Do they know how to generate a procedure? How do we teach students to apply mathematical thinking in creative ways to solve complex, novel problems? What happens when we get off the ‘script’?”

As someone who learned math largely though mere exercises and who now creatively applies math at work, I have to question this thinking. I believe that students’ difficulty in solving new problems is more likely to be because 1) applying prior knowledge to new or non-routine problems is hard for everyone, and 2) it is even harder for students who may lack the requisite knowledge and/or mastery of skills—not because they possess such knowledge and mastery. So while the educationists distinguish between “exercises” and “problems”, the view refuses to distinguish between novices and experts.

Daniel Willingham, a cognitive scientist who teaches at the University of Virginia, finds the distinction between novice and expert to be quite important. He maintains that it takes time and effort for knowledge to accumulate to the point that connections between learned material and new and difficult problems can be made. Willingham refers to the difficulty that novices have with thinking critically as “inflexible thinking.” In fact, he characterizes such difficulty as perfectly normal and to be expected among students. Willingham argues that understanding the deep structures of a discipline such as mathematics is an important goal of education, “but if students fall short of this, it certainly doesn’t mean that they have acquired mere rote knowledge and are little better than parrots.” Rather, they are making the small steps necessary to develop better mathematical thinking. Simply put, no one leaps directly from novice to expert.

I was therefore extremely interested to see the sample problem of the Balanced Assessment in Mathematics (BAM) in Appendix 1 of the Gates Foundation’s preliminary report. The BAM is to be used to assess teacher effectiveness and according to the Gates Foundation’s preliminary report, “In comparison to many other assessments, BAM is considered to be more cognitively demanding and measures higher order reasoning skills using question formats that are quite different from those in most state mathematics achievement tests. There is also some evidence that BAM is more instructionally sensitive to the effects of reform-oriented instruction than a more traditional test (ITBS).”

The sample problem is reproduced below and is from the 8th grade mathematics assessment:

The diagram makes it clear how one is to count the number of tiles needed, so the first question is relatively easy. Question 2 requires more thought, and the student must be able to extend the counting algorithm for the 4ft pond in question 1 to ponds of other sizes; i.e., they must understand that 4 is added to the product of the number of one foot long squares that fit on one side of the pond times 4, or 4n + 4 where n is the length of one side of the square pond in feet.

There are four additional questions that become increasingly difficult: “How many paving stones are needed to surround a fish pond that is 20 feet by 20 feet?”, “Chris has 48 paving stones. Find the size of the largest square pond the paving stones can surround.” “The garden center sells many different sizes of square fish ponds. Write down a rule that will help Chris figure out how many paving stones are needed to surround square ponds of different sizes.” “The garden center decides to sell rectangular ponds. Find a rule that will help Chris figure out how many paving stones are needed to surround rectangular ponds of different sizes.”

This set of questions is fairly challenging to beginning algebra students. (It is even more challenging if they have had no algebra at all, but since this is a problem for 8th grade students, I am assuming that they have had some experience with algebraic expressions and equations. This is not always a safe assumption but that’s a topic for another article.) The sample problem is illustrative of the type of problem that the education community deems coach-proof since it appears that memorization of problem solving techniques and “drill and kill” exercises will not work here. But in fact, practice with some exercises would help students in tackling such a problem—specifically, having students express in mathematical terms certain situations described in English. For example, “Three more than two times what John’s age will be in five years” (3 + 2(x + 5) ). These types of exercises are frequently deemed by the education establishment to be inauthentic and irrelevant to the deeper underlying concepts of math unlike “reform oriented instruction” which purportedly provides such deep understanding through so-called authentic problems and a minimum of “exercises”.

Interestingly, the TIMSS exam—an international exam given every three years—also contains questions of this sort, as well as more straightforward problems. For comparison’s sake, I looked at TIMSS eighth grade questions in algebra (found at http://nces.ed.gov/timss/pdf/TIMSS8_Math_ConceptsItems_2.pdf ) and found a similar type of problem:

This problem requires students to understand and ultimately express the relationship between the number of smaller squares on each side of the larger square, and the number of triangles contained in the square. Question C, in fact, requires the student to be able to express the relationship mathematically in order to calculate the number of triangles. The TIMSS report in which this appears provides some interesting data related to this question; namely the percent of students taking the test in each country that obtained the correct answer:

The top five scoring countries for this question ranged from 44 to 49% correct. For the more straightforward problems the top five scores tend to be in the 70 and 80 percent range. The US students obtained 19 percent correct for this problem; on more straightforward problems, the US scores in the 50 and 60 percent range. Thus, for all students, regardless of country, non-routine problems prove to be difficult. Of interest, however, is that the five top scoring countries for this particular problem are Asian, frequently criticized for using drilling and “inauthentic” exercises which they maintain do not properly prepare students for solving non-routine problems.

If Vicki Phillips’ statement about teaching to the test is any indication, however, the educational establishment will see what they want to see. They will likely proclaim that the higher scores obtained by Asian countries on non-routine problems serves as evidence that the Asian countries use “reform-oriented instruction”. Either that, or they’ll shrug their shoulders and say “It’s the culture; what can you do?” (See http://www.educationnews.org/commentaries/104502.html )

In any event, whether the education establishment realizes it or not, the new generation of coach-proof tests that will be used to evaluate teachers, appear to be measuring the skills students are expected to be learning. And by teaching what should be taught, teachers are teaching to the test, whether the Gates Foundation and its look-alikes realize it, like it, or not.

*BIO: Barry Garelick is an analyst for a federal agency and is cofounder of the U.S. Coalition for World Class Math. (http://usworldclassmath.webs.com/ ) He plans to teach math later this year.*

The distinction between exercises and problems is also made in Russian mathematics education (упражнения vs. проблемы).

A strong ability to solve problems can _only_ be acquired by working on and solving a large number of problems (a stand-alone command of the mathematics involved, and related skills, acquired by working through exercises, is both required and assumed).

The included BAM and TIMSS questions don’t make the Russian “problem” cut.

Grade 6 problem: The value of the product of a number and its digit sum is 2008. Which is the number?

Western mathematics curricula’s dearth of problems is their greatest weakness.

Thanks for that observation.

By the way, I checked on the statistics for the TIMSS problem. Only 11% of the Lituanian and Romanian students answered the question I excerpted correctly. In the Russian Federation, only 9% of the students answered it correctly.

I distinctly recall the “aha” feeling I used to get when I finally figured out a pattern within, or solution to, a difficult math problem. In school, I was taught using the “new math” curriculum which downplayed the acquisition of skills. I acquired skills, at the kitchen table, with my physicist father who drilled me with practice problems, the rote kind. With these foundational skills, the “thinking-through” aspect of math was possible: without these skills, my mind meandered hopelessly.

Now I homeschool two teen boys. I’ve noticed that contemporary texts seem to teach HOW to think about math, but not how to DO math. In other words, the newer didactic methods teach thought, not skill. In my wee opinion, this inverts the joy-laden process of figuring out a difficult problem by taking away that “aha” moment which comes only after scrubbing at a problem until the answer become clear(er).

I suppose teachers feel they have to bring up the lower half of the class, so instead of letting kids struggle, they spoon-feed the answers, and thus steal from them the process of discovery. It’s a shame, really, because those kids will never like math. They’ll never experience how concentrated and focused problem solving will pay off as new ways of thinking take root.

[...] The Educationist View of Math Education January 25, 2011 — pwceducationreform The Educationist View of Math Education [...]

Problem solving, which is preferred, occurs when students are not able to apply a mechanical, memorized response, but rather have to apply prior knowledge to solve a non-routine problem.These people should love the SAT.

And yet, in my real life the most useful mathematical skills are the ones that I do apply mechanically. It didn’t start out that way — it took many repetitions of the same questioning and decision-making process before I even knew what mechanical process would work.

Exercises versus problems is — A Bogus Dichotomy in Math Education.

just as

Basic Skills Versus Conceptual Understanding [is]– — A Bogus Dichotomy in Math Education” by Dr. H. Wu in The American Educator, American Federation of Teachers, Fall 1999

While Math educators may advocate for coach-proof problems, commercial textbook writers provide coaching so as to make the math, student-proof.

For an international comparison:

Problem 17. (From a 5th grade Singapore math textbook) “Mrs. Chen made some tarts. She sold 3/5 of them in the morning and 1/4 of the remainder in the afternoon. If she sold 200 more tarts in the morning than in the afternoon, how many tarts did she make?

(“Reading Instruction for Arithmetic Word Problems:

If Johnny can’t read well and follow directions, then he can’t do math”

It is at

http://www.math.umd.edu/~jnd/subhome/Reading_Instruction.htm)

My guess is that TIMSS avoids problems even half as difficult.

In contrast:

“Recalculating The 8th-Grade Algebra Rush” reprints this simple NAEP problem: Problem requiring Two Operations

There were 90 employees in a company last year. This year the number of employees increased by 10 percent. How many employees are in the company this year?

A) 9

B) 81

C) 91

D) 99

E) 100

The correct answer is D. Ten percent of 90 is 9.

Nationally, barely one in three of eighth-graders and less than half of those in Algebra I or above answered correctly.

The square fish pond problems are standard Reform – inspired textbook and state assessment type problems. The rectangular are trickier.

Step zero. Do not do a geometric or other content analysis.

Step 1. Fill in the chart.

Here:

1 2 3

8 12 16

The numbers increase by 4 both times, so assume this happens forever.

For Maryland state assessment on “pretend” algebra, the sequence must be:

A. Arithmetic: The numbers increase by same amount;

B. Geometric: The numbers are multiplied by same amount;

C. c n^2 (constant times square of n).

For the TIMSS problem;

Step zero. Do not do a geometric or other content analysis.

Step 1. Fill in the chart.

Here:

1 2 3

2 8 18

Clearly not type A or B, so must be C: c n^2. Then c = 2.

This algorithm will produce WRONG answers for Problems 10 & 11 in Pattern Recognition in Math Instruction

http://www.math.umd.edu/~jnd/Patterns.pdf

The Reform movement is big on real-life problems.

But Step zero is Do not consider biology.

This leads to correct answers on exams, which are not feasible biologically.

For example,

Problem 14 (1992 NAEP mathematics assessment) at

http://www.math.umd.edu/~jnd/Patterns.pdf

MD Algebra sample test Item #19 in

http://www.math.umd.edu/~jnd/Halloween.html

Relatedly:

From http://www.math.umd.edu/~jnd/The-Math-Wars.html

Similarly, in their much quoted 1998 article, “The Harmful Effects of Algorithms in Grades 1-4″, Constance Kamii and Ann Dominick wrote: “Algorithms not only are not helpful in learning arithmetic, but also hinder children’s development of numerical reasoning …”

Invoking Piaget’s constructivism, Kamii and Dominick wrote: “Children in the Primary grades should be able to invent their own arithmetic without the instruction they are receiving from textbooks and workbooks.”

As Wu has noted: “Why not consider the alternate approach of teaching these algorithms properly before advocating their banishment from classrooms?”

Wu also wrote: “What is left unsaid is that when a child makes up an algorithm, the act raises two immediate concerns: One is whether the algorithm is correct, and the other is whether it is applicable under all circumstances.” To carefully check and correct many new student algorithms periodically is a sizable task. Also as Liping Ma has documented[17], many teachers do not have sufficient knowledge of the mathematics they are teaching, to be qualified algorithm checkers. Teachers currently insist that students present fractional answers in simplest terms so that there will be a unique right answer and they will not have to check whether each student’s unsimplified fraction is equivalent to the answer in the teachers’ manual.

What is wrong with simplifying fractions to the simplest form? It has nothing to do with teachers checking answers with their manual. It is a mathematical convention like hundreds of other conventions used around the globe.

Hey,

I wanted to do crash course on Gate maths . To brush up my skills and clear some of my doubts . As relocating is difficult so wanted to do gate online coaching searching around i came across http://www.wiziq.com/course/19502-gate-2014-ies-2013-engineering-mathematics-crash-course . If anybody has studied from this course please let me know ..