Borat | Jay P. Greene's Blog

How Well Aligned Is Kazakhstan to NAEP Standards?

June 25, 2008

(Guest Post by Matthew Ladner)

Recently I appeared on the Horizon public affairs program together with Arizona Superintendent of Public Instruction Tom Horne, to discuss the No Child Left Behind law and our state AIMS test. Superintendent Horne and I have a public disagreement about the relative reliability of NAEP compared to that of the state’s own version of the Terra Nova exam. NAEP finds Arizona consistently below the national average in all subjects and grade levels, while the state’s Terra Nova finds us above the national average in all subjects tested and grade levels. One of these sets of finds is much more consistent with the socio-economic profile of the K-12 population than the other, given that Arizona ranked second to the bottom on Jay and Greg’s Teachability Index study.

During the discussion, Superintendent Horne said the main reason Arizona students perform poorly on the national NAEP test, also known as the Nation’s Report Card, is due to a non-alignment of standards. If, for example, Arizona does not teach the math concepts in fourth grade that appear on the fourth grade math NAEP, one could expect lower average grades.

The explanation seems quite plausible, and doubtlessly there are some states that have better aligned their standards to NAEP than others. But how big a deal is this, in terms of Arizona’s performance? A study by the American Institute of Research shows probably not much.

The study compared international science scores for eighth graders to eighth grade NAEP science scores through an equating proceedure. Singapore came in first, with 55 percent of students ranked as “proficient” or above. Massachusetts was the highest-performing U.S. state, with 41 percent proficient. Just 20 percent of Arizona eighth grades ranked proficient.

Alignment error ought to be much greater between nations than between American states. Perhaps, for example, Norway chooses not to teach science until 9th grade. One would be hard pressed to buy into the notion that countries such as Singapore, Korea, Estonia, Hungary, and Slovakia simply have national standards more closely aligned to the American NAEP test than Arizona.

When we get clobbered in science proficiency by countries like Estonia, we have problems that go much deeper than standards alignment. I could start looking up GDP per capita in Estonia, but that would be cruel. We need to be willing to think outside the box and figure out what other countries are doing right.

Nation (or State) 8th Grade Science Scores	Percent Scoring “Proficient” or Above	Nation (or State) 8th Grade Science Scores	Percent Scoring “Proficient” or Above
Singapore	55	Lithuania	25
Taipei	52	Slovenia	24
South Korea	45	Russia	24
Hong Kong	44	Scotland	24
Japan	42	Belgium	22
Estonia	41	Latvia	21
Massachusetts	41	Malaysia	20
England	38	Arizona	20
Hungary	38	Israel	18
Netherlands	31	Bulgaria	17
Australia	30	Italy	17
Sweden	28	Norway	15
New Zealand	26	Romania	14
Slovakia	26	Serbia	12