Below is my oped in the Arkansas Democrat Gazette on how Arkansas should drop the PARCC test next year and switch to a nationally normed referenced test to reduce teaching to the test, avoid unproductive fights over standards and testing, and stop bossing local schools and teachers around. It would also cost a lot less.
Opportunity arises
State has chance for better tests
Posted: February 14, 2015 at 1:58 a.m.
The Legislature is considering a number of proposals to alter the state’s current requirement that public schools administer the PARCC test, one of two federally funded tests aligned with the Common Core standards. These proposals are motivated by a variety of concerns, some of which are legitimate and reasonable and some which are not.
Whatever its legitimacy, the current debate is an excellent opportunity for Arkansas to reconsider why it requires testing and what kind of test would best serve these public purposes.
Past testing has been motivated primarily by two goals: providing transparency and directing the behavior of educators. The evidence is now clear that transparency has meaningful benefits for parents, communities and policymakers, but attempting to use testing to drive what educators teach and how they teach it has been counterproductive. Starting next academic year, Arkansas should switch to using a norm-referenced test that provides transparency but does not attempt to control how schools and teachers do their job.
The PARCC test, which Arkansas plans to administer this spring, is a criteria-referenced test, which is the type of test that tries to drive what educators teach. It is aligned with a set of standards–in this case, Common Core–and rewards schools and teachers that emphasize teaching the particular content covered by the test and expected by those standards.
Criteria-referenced tests cause three serious problems. First, because they are aligned with a set of standards, schools and educators are able to “teach to the test.” But it is also possible for educators to focus too narrowly on the content covered in the test. All criteria-referenced tests are inherently gameable. In trying to tell schools and teachers what they should teach, they also provide educators with a road map for how to drill students in the particular content of the test to raise scores artificially. This undermines the transparency benefit of testing by pushing schools to manipulate results through an inordinate amount of test preparation. It also causes schools to narrow their curriculum to coach tested content to the exclusion of other important content and subjects. The arts, history, and sciences all suffer from criteria-referenced testing in math and reading.
Second, because criteria-referenced tests are aligned with a particular set of standards, they invite a perpetual and destructive struggle over what those standards should be. I have no opinion about the merits or defects of the Common Core standards, but I do know that Brookings Institute scholar Tom Loveless and others have shown that states with “better” standards have fared no better on independent measures of academic achievement than states with what are considered lousy standards. It isn’t so easy to know what the right standards should be, and it’s even harder to push educators in productive ways to teach those standards. Given how the quality of past state standards has made no difference in student outcomes, why should people bother to fight to impose a set of standard and aligned testing over the objections of many who have legitimately different opinions about what educators should teach?
Third, criteria-referenced tests are trying to use a crude instrument from a great distance away to get schools and educators to do what state or national policymakers think is best. Even if those remote elites are well-intentioned–and sometimes they are not–schools are still run best by local educators and communities.
There is a different type of test that Arkansas could administer that avoids these dangers of excessive and narrow test prep, unnecessary political fighting, and centralized control over education. Norm-referenced tests are not aligned with any particular set of standards, but can still provide general measures of how well students are performing academically. They meet our reasonable goal of wanting transparency about how students are progressing in school. But because they are based on a generic curriculum rather than a particular set of standards, it really isn’t possible for schools to game them by focusing exclusively on a narrow set of content.
In addition, because norm-referenced tests are not pushing a particular set of standards and content, they do not invite political struggles. They also don’t boss around local schools and teachers because they aren’t trying to make them teach particular content or in a particular way.
There are plenty of already-developed norm-referenced tests from which Arkansas could choose. Because they can be bought off the shelf and do not have to be customized to Arkansas, they would also be cheaper than trying to develop another Arkansas-specific criteria referenced test. It is probably wisest to go ahead with PARCC this year, given how late in the process we are, but a change to a norm-referenced test could be made for next school year.
Don’t let anyone tell you that the federal government requires us to adopt PARCC or some other criteria-referenced test. Education is the prerogative of states, and federal law prohibits the national government from mandating specific tests or curriculum, despite the fact that it has taken steps in that direction. Arkansas needs political leaders with the courage to stand up and do what is best for Arkansas.
————v————
Jay P. Greene is the 21st Century Endowed Professor in Education Reform at the University of Arkansas, where he is head of the Department of Education Reform.
Editorial on 02/14/2015
Jay P. Greene's Blog 
[…] 2. Arkansas should drop PARCC for a NRT […]
The first time I’ve disagreed with something on this fine blog. Even a ‘generic curriculum’ requires a body of knowledge and skills students must master.
My classroom tests are criterion-referenced; I expect my students to master a specific body of knowledge and skills. Even before Common Core I taught to standards without test prep; as a math teacher I respect that students must master a specific body of knowledge so as to progress through their academic career. What criterion-referenced tests have exposed, especially for urban students, is the paucity of content knowledge and skills taught, which over time becomes that dreaded achievement gap.
If there are teachers/schools/districts gaming the system to raise test scores to a criterion-referenced test it may take longer for parents to recognize this (college remediation, weak job prospects upon graduation), but of course parents need transparency into curriculum and pedagogy so as to make the best choices for educating their children as early as possible.
‘Generic curriculum’ still requires young people to master a common body of knowledge and skills; what’s been happening in urban schools for decades, back when my parents were teachers in DC Public schools well B.R.E. (Before the Rhee Era) were teachers not teaching anything and students thus were woefully prepared to participate and prosper in society. I would much prefer to know what my child knows and can do versus how well they performed versus other students.
” narrow test prep, unnecessary political fighting, and centralized control over education..” aren’t issues caused by criterion-referenced tests. These are issues caused by inept education leadership, from politicians down through school districts and to the school site. If I were an elected official I would first seek the consent of the governed on implementing objective tests, and if receiving such would work only for those tests to reveal learning, and let parents/taxpayers use that data as they choose.
Indeed districts should be able to choose whatever assessment they desire, but a criterion-referenced test should be one of those choices. I want to know what my child knows and can do, thus can use that data to influence my education choices for them.
pdexiii-
I’ve been curious about this for a long while. Washington D.C. used to give the Stanford 9 as their “state test” B.R.E.
If we are talking about math, at some point as you go up grade levels you get tested on new concepts. Making something up here, 4th graders get tested on long division and 3rd graders don’t. It doesn’t seem impossible to infer grade level math curriculum from such things, schools have specialists etc.
In Arizona, far too many educators have told me that the standards are a pdf file floating on the web and there is massive teaching to the test items going on. I have however had others respond in a similar fashion to your response, so I am not sure what to make of it.
So let’s just throw a hypothetical out there. Assume your state did what Jay suggests. How would you change your approach, if at all, to teaching your students math?
Before we adopted Common Core in CA there was a .pdf file of released test questions. I would give those questions as a regular assessment on my own to see how my students were performing, but did not alter my pedagogy or curriculum too much. I figured if I were teaching the math the best I could, my students would be ok. My test scores were never super but were accurate: the students scoring Proficient were my ‘A’ and ‘B’ students.
I would not change my approach under that testing approach. Yet, even for a norm-referenced test there has to be a common body of knowledge that’s taught. If not you end up with a test that once again reveals the disparity in background knowledge students bring from home, which is partly what drove us towards criteron-referenced tests we use today.
Schools may have specialists (I’ve never met them), but the real issue is time. Like Earl Weaver used to say, you win the pennant in September with the decisions you make in February. Once the school year starts a teacher’s focus becomes the students. You must use the ‘off-season'(summer) to make those curriculum and pedagogy adjustments-inclusions because there’s no time for it from September to June. Although I look forward to doing that in the summer, that may not be the case with other teachers/schools.
To the extent that state standards were actually done well- with one grade’s work setting up the next etc. then they would seem to be useful vis a vis everyone doing their own thing. To the extent teachers are ignoring the standards and focusing exclusively on over-exposed items then a good deal of that utility could be squandered. NAEP does show a good deal of progress in 4th and 8th grade math since the early 1990s so maybe it’s working (??)
It seems to me you could still have standards but use a nnr instead of a criterion based test.
Reblogged this on James V. Shuls Blog and commented:
Replace Arkansas with Missouri and PARCC with Smarter Balanced.
Hopefully Mr. Greene you can sign this petition against the abuse of high stakes tests that 750 educators from higher learning institutions have signed. Then pass the word Mr. Greene! http://Www.nepc.colorado.edu/publication/esea
Mr. Ladner, Even when California mandated Algebra 1 for all 8th graders we were still getting 6th graders who couldn’t multiply past their 6’s, let alone operate on rational numbers. This is the main reason even under that mandate many schools did not have all their 8th graders in Algebra 1, for fear of the dismal API/AYP scores they’d have. Indeed even then California’s standards did not overlap, but I suspect teachers felt the urge to re-teach when they saw students who couldn’t do things they should know. I learned the hard way my second year that it’s better to forge ahead with the curriculum and spiral/re-teach where you can vs assuring “everyone gets it before you move on” because you set students up for failure if you don’t. I’d say 80-90% of my students come back and thank me for ‘being hard on us’ because even though they may have struggled in my class math was easier for them later.
Even though there are lots of teachers out there, in your poor/urban districts there’s a lack of committed ones, which also fuels the achievement gap. Not only are there not enough teachers who want to teach poor/black/brown students for 6, 8, 10 years or more, but there are few if any incentives ($$$) for teachers to do so. Property-tax based school funding doesn’t help, either; a school in a poor neighborhood can’t afford a cadre of 10-20 year teachers, so they end up scooping up the energetic but cheap TFA’ers.
pdexiii-
When did you start teaching, and has there been any noticeable change in the % of kids who can’t perform grade level math over time?
1995-96: Kobe and I have been hacking away for about the same time!
I haven’t seen a noticeable increase, which to me amounts to a decrease. After 20 years and there’s no growth, that’s falling behind. Putting a number on it, I’d say 30-40% tops come to us with the requisite knowledge and skills, but there’s been a decline in the skills that I think support math success. Students don’t learn cursive thus can’t take notes, are horrid spellers and don’t write complete sentences. Word problems flummox our kids because of 1) vocabulary and 2) limited out-of-the house experiences. We have too many students who couldn’t tell you their street address. Hell, I’ve seen a general decline in the level of general athleticism which I attribute to too much Madden, NBA2K, FIFA and Call of Duty.
Sadly, we can identify a student’s prior knowledge to the elementary school from which they come, too.
NAEP says California had 17% of 8th graders proficient or better in math back in 1996 and 28% in 2013, so 30-40% sounds quite plausible.
The question is whether we are getting something out of these criterion based tests that we would not get out of an NNRT. Ironically at this point it very clear that the Stanford 10 has any criterion based test beat hands down on comparability across state lines.
[…] Update: Jay P. Greene makes a case for norm-referenced tests. […]
I agree that we should drop it. It is to hard. Adults who have taken Parcc Practice Test don’t know what they are doing!!!