(Guest Post by Matthew Ladner)
Political scientist Donald Campbell postulated that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” The Army of Angry Teachers has seized upon the Atlanta cheating scandal as proof that the whole process of testing and transparency is destructive and ought to be done away with.
Arne Duncan weighs in on the Atlanta cheating scandal as a part of a roundtable at the WaPo. Duncan provides a bit of much-needed perspective on the problems of testing, noting that although the Atlanta scandal is the worst uncovered, that it involves 44 schools out of thousands in Georgia.
Secretary Duncan goes on to acknowledge a number of problems in state academic testing, including the far larger problem of states dummying down their cut scores in order to proclaim improvement.
One elephant in the room: test security. It isn’t difficult to infer that while the state of Georgia performed erasure analysis on the tests (thus uncovering the cheating) that they failed to let it be known that they would be doing so on a large-scale (and thus failed to deter the cheaters, who thought they could get away with it). States need to not only employ these techniques, they need to employ them as deterrents.
People are quite clever, however, and constantly develop new ways to cheat if provided incentives to do so. It seems possible that a system of third-party administration of tests will need to be developed as we attach greater consequences to test scores, including school ratings and merit bonuses. This could be a simple as the way you took the SAT test, or it could have a more high-tech look to it.
Another Campbell’s Law problem that strikes me as more serious than systematic answer changing by staff is the practice of teaching to test items. I fear that this is quite widespread, although it is difficult to quantify. The idea behind the standards movement is to teach to a set of academic standards, and to use testing to measure success. If teachers instead teach to a set of test items then the whole process can devolve into a farce.
A skillfully managed system of student testing can and has played a leading role in improving student outcomes. It’s difficult to pull off, and easy to foul up. We should be concerned about staff led cheating. We should be even more concerned about low cut scores, item exposure and test study guides.
Posted by matthewladner 
(Guest Post by Matthew Ladner)
Jay P. Greene's Blog 
