Rick Hess has a thoughtful post today on last week’s dust-up over whether choice schools should be required to take state tests. Rick is generally sympathetic with the arguments I was making but raises two objections.
First, Rick worries about whether I (and others) are being consistent in opposing testing requirements for choice schools while having “long slammed districts and promoted school choice by pointing to reading and math scores.” He continues, “I’ve got a lot of sympathy for those who feel like Greene’s position constitutes something of a bait-and-switch, with choice advocates are changing the rules when it suits them.”
Second, Rick thinks there is an inconsistency in my suspicion that test-prep and manipulation are largely responsible for test score improvements by Milwaukee choice schools after they were required to take high-stakes tests, while I interpret research from Florida as showing schools made exceptional test score gains when faced with the prospect of having vouchers offered to their students if scores did not improve. Why would I believe the former is an artifact of test prep, but not the latter?
Let me deal first with Rick’s second objection because it is easier and quicker to address. I was concerned about whether test prep and manipulation were responsible for the exceptional gains made by low-graded schools that faced the prospects of voucher competition if their results did not improve. So, Marcus Winters and I examined results from the Stanford-9, a nationally normed low-stakes test, as well as the state’s high-stakes FCAT, to see if the results were similar. Here is what we wrote:
Schools are not held accountable for their students’ performance on the Stanford-9. As a result, they have little incentive to manipulate the results by “teaching to the test” or through outright cheating. Thus, if gains are witnessed on both the FCAT and the Stanford-9, we can be reasonably confident that the gains reflect genuine improvements in student learning.
The results were similar, showing exceptional gains on both high and low stakes exams, which gave me confidence that the improvements in FL were real. In Milwaukee we do not have a similar check on whether learning gains were real after high stakes testing requirements were imposed. In the absence of a low-stakes check, I’m highly skeptical of whether choice schools suddenly improved in quality when they were required to administer the high-stakes tests that the study subjects had been taking all along with lower results.
Rick’s first point — essentially, that I am being hypocritical in opposing testing for choice schools but not for traditional public schools — requires a more complicated response. I would be happy opposing state testing requirements for all schools (choice and traditional public) if those schools had some reasonable mechanism for accountability. Choice schools are accountable without testing requirements because parents can choose whether to send their children (and the resources that follow those students) to those schools or not. If those schools are not accomplishing what parents want, choice schools have difficulty attracting and retaining students and resources.
Most traditional public schools, however, have no meaningful system of accountability. They receive students and resources regardless of whether they are accomplishing what families want or not. If schools are not held accountable by choice, then they have to be accountable by some mechanism. One way to produce this accountability is to require that they administer state tests and meet certain performance benchmarks. This type of top-down accountability is far less efficient and comprehensive than choice accountability, but it may have to do in the absence of choice. But if charter, private, and Tiebout choice were to expand to the point where no school was guaranteed students and revenues regardless of performance, then I’d be fine with getting rid of all testing requirements.
Of course, there would still be plenty of information about schools because most schools in choice systems voluntarily administer tests and report results. They just choose their own tests, just like how they choose their own standards, curriculum, and pedagogy. And since tests only capture a tiny portion of what most schools are trying to accomplish, parents would collect information on these other outcomes of education just as consumers collect information on the quality of other complicated services their children receive, including summer camp, piano lessons, babysitters, etc… We don’t have state required testing — or even any testing — for most of these services, so parents rely on reputation, word of mouth, direct observation, and other techniques to collect information and make choices. No system is perfect and people will make mistakes, but I’d rather that parents make their own mistakes than have bureaucrats impose mistakes upon them.
This skepticism about state testing does represent a shift in my thinking that has been underway for a few years now. I’m sure someone could dig up an old quote from me embracing top-down accountability in a way that I would not do now. But I’ve seen more evidence and collected more experience over the last several years that has made me much less enamored of state testing. I’m convinced that state tests are highly imprecise, very limited in what they cover, subject to test-prep and manipulation, unable to capture the diversity of school goals and circumstances, and seldom used to make intelligent decisions about improving schools. Simply put, I am no longer a supporter of top-down school accountability regimes. But until we have expanded choice further, I see no practical alternative to continuing state testing for schools not subject to meaningful choice accountability.
Jay P. Greene's Blog 
I would add that your study with Marcus looked at numerous state tests, not just Florida’s, and it did identify a case where scores were significantly higher on the official, high-stakes test than on a similar low-stakes test given to the same students – indicating that there was some divergence between real learning and the ephemeral results of “test prep.”
In other words, you have never been against the idea that the results of high-stakes tests could be distorted by the high stakes; you just didn’t think that was happening in Florida because the evidence was pretty strongly against it.
You suggest, incorrectly, that there wouldn’t be accountability for non-choice schools and thus top down accountability is implemented as an alternative. What then of districts that have choice? Take large, impoverished urban areas for example, where the proliferation of choice has been the greatest. Students and parents have a “choice” in whether or not to go to a charter (choice) school. That is accountability, is it not? What is the criteria then during a transition from “non-choice” to “choice”? In the interim, you would suggest a position that is inconsistent.
By the way I am sure you are also familiar with Campbell’s Law: “The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Ok, Jay, so at your suggestion, I followed you over here. So as I now understand it, no choice schools must continue the testing because they do not have any other accountability measure, but choice schools have accountability because people can vote with their feet. My question now is how do these choice parents get the information to vote with their feet? As I see your proposal, no choice parents can say, “Boy my school is crappy; look at these test scores.” Choice parents get to say, “Boy, this school must be wonderful; look at this snappy brochure.”
You said: ” … Choice schools are accountable without testing requirements because parents can choose whether to send their children (and the resources that follow those students) to those schools or not. If those schools are not accomplishing what parents want, choice schools have difficulty attracting and retaining students and resources.
Most traditional public schools, however, have no meaningful system of accountability. They receive students and resources regardless of whether they are accomplishing what families want or not. If schools are not held accountable by choice, then they have to be accountable by some mechanism…”
Which really amounts to an out-and-out lie…
(I wont address here the manipulation in the first lie that Choice schools are public schools – they are not; they are – legally and for taxation purposes – private schools, sucking taxpayer dollars earmarked for public education out of public school classrooms)
CHARTER schools (lets dispense with the deceptive and manipulative playing around with the language that is using the misnomer “choice”) pick and choose their student populations….
They do this by imposing difficult application processes on hopeful families, discouraging children with disabilities and ESL populations and “counselling out” (ie EXPELLING) difficult to manage/teach children.
It is the administration of a CHARTER school that has final decision on whether or not to accept a student – ergo, its not really a family’s CHOICE in the end…
Public schools, on the other hand, have to take all comers and have to try to meet their needs with diminished resources because of the continuing drain on public school funds by the profiteering CHARTER SCHOOL businesses wanting to make a quick buck off our children, who have suddenly become “cash cows” (see Boston Consulting Group’s Marketing Matrix for reference).
When you have a diverse public school population, with diverse needs that must be met, with diverse learning abilities, with inadequate resources to meet all those needs, its quite obvious that standardised testing, by definition, will never be a legitimate tool for measuring teaching effectiveness…
I don’t think it would be hypocritical to say you wanted testing for public schools but oppose it now for private schools.
Have you all forgotten the pre-school-choice years?
Back when school districts and education leaders said “give us more money, don’t watch over our backs, yes 99 percent of our teachers are great, just send us the kids and you better not give them any other competing options! We promise better results. Seriously, things will improve, just leave us be!”
After decades of that, we all had enough. We wanted proof, or choice, or both. The testing and accountability happened because the establishment opposed choice but demanded more money for decades without showing any meaningful improvement.
Patrick,
Interesting you say “private schools.” I thought charters were supposed to be public schools also. Or maybe they are just public schools when there is public money to be had, but private when it comes to accountability.
Personally, I hope the high-stakes testing continues (even though it is completely misguided) and demonstrates that the reason “non-choice” schools were considered failing in the first place was fallacious as “choice” schools suffer under the same yoke. At the end of the day, I hope this testing regime puts the nail in the coffin for “choice” versus “non-choice”.
What about districts like Chicago where the parents have voted with their feet and most of the charters are underenrolled?
Competition only works on an even playing field. If you own the referee and the scoreboard, it’s a recipe for poor quality and victory.
Someone who cannot effectively engage in dialect really has no business judging education. Keep up the good work, you weaken the case for “choice”.
That is in reference to the original author, Jay P. Greene.
Sami,
Although I agree with your position I am confused by your use of the word “dialect.” I think you meant “dialogue.”
Sorry, meant dialectic.