(Guest post by Greg Forster)
What’s the best way to top off a really Walt-on-Lost level bad week for Common Core? How about a scandal in which emails show one of its most prominent supporters having manipulated his state’s “high standards” system to ensure that a particular school (one founded by a major donor) scored high? Just as that same state becomes the latest to move toward dropping out of a CC testing consortium?
I’d like to take Andy’s bafflement about CC supporters not anticipating pushback on the costs of implementation and double it in this case, if not triple it: why on earth did they discuss this so transparently in their government email accounts, which made it inevitable that the whole ugly show would eventually come out?
I feel sympathetic to Tony Bennett here. Any kind of evaluation system must involve qualitative as well as quantitative testing. That is, you not only have to make sure the numbers are accurately collected, crunched and reported, you have to make sure that what the system is calling “good” really is good. Of course you could in theory test your system by comparing it to the results of other systems, but if that’s all you do, the whole thing is circular. Ultimately you have no choice but to pick some examples of cases that you presuppose to be very good or very bad (or in the middle, for that matter) based on some kind of opinion – maybe yours, maybe your organization’s, maybe a consensus of experts, maybe a popular majority – and see if your system ranked those cases in accordance with the presupposed opinion. It is logically impossible to remove this element of judgment. You just can’t fully test a system for evaluating schools without at some point picking out some super-schools and asking “did these score well?”
Of course, everything hinges on what basis you use for selecting those cases – in other words, whose opinion of which cases are “good” you presuppose, and why. In the real world, if the standards are being set by government, that is always going to be a political process in which one or another set of powerful constituencies are privileged. The Bennett emails reveal the sausage-making nature of the process. What I want to emphasize is that this is not because Bennett is in some way specially corrupt but that this is what any such process must always look like. It is, again, logically impossible to avoid this type of qualitative reality check, and it would be naïve in the extreme to think that any set of political actors would carry out that reality check in any way other than something like what the Bennett emails reveal.
The lesson here is not “Bennett is corrupt” but “all educational standards privilege someone’s opinion of what is a good school, and government privileges the opinion of powerful interests.”
Jay P. Greene's Blog 
Sorry Greg, but I disagree with your interpretation of the fact any kind of evaluation system must involve qualitative as well as quantitative testing. There is, indeed, a qualitative judgement involved in deciding, for example, the components that go into a “school grade” and their respective weight. Such is deciding whether math grades will count for 20% or 40%, how much weight give to ELA grades, whether to include measures of absenteeism, violence and similar and at what “weight.” But those are large-grain decisions that are not really arbitrary — they should emerge from a broad public discussion and are easy to understand by the average citizen. I doubt anyone has difficulty deciding whether absenteeism or school violence, or student achievement are “good” or “bad.” So the discussion is more about which ones to include, and their weight in the total. Let’s not make school rating sound more arbitrary than it already is.
I don’t see where you disagree with me, Ze’ev. I never said qualitative decisions were “arbitrary.” I just said they had to be made. In your first sentence you deny that qualitative testing is necessary, but later you admit that it is – you’re just anxious to emphasize that it isn’t arbitrary and can “emerge from a broad public discussion and are easy to understand by the average citizen.” So you really agree with me that decisions about qualitative questions must be made; you’re just anxious that people will misunderstand and leap to the conclusion that there is no reasonable and public basis for such decisions. I’m more anxious about a technocratic science-worship that denies the need to make such decisions at all. I did say that they could be made on the basis of public consensus, although I’ll admit that my use of the phrase “popular majority” may be taken to imply that I think all such decisions are arbitrary even when made by the public. I don’t think that; I think the civic community can make reasoned public judgments that are not simply the imposition of the arbitrary will of 51% upon the arbitrary will of 49%. However, such reasoned public judgments can only happen in the organic discourse-community of civil society; when I said government couldn’t make these decisions without privileging the powerful, I meant if government is the sole driver of these decisions – the state imposing its standards on society rather than the other way around. If society comes to a reasoned public judgment about what is good (e.g. that murder is wrong, and it is the kind of wrong the laws can forbid and punish) then the state is capable of institutionalizing those decisions. That is its job.
I was busy elsewhere, hence the delay.
Greg, I do think we agree. I, indeed, was concerned with the “arbitrariness” aspect you depicted and how it might be interpreted. Not with having both qualitative and quantitative elements. Perhaps my leading sentence was hard to parse — sorry. All I was trying to say was that the “arbitrariness” needs not be arbitrary or opaque, but rather should emerge from a logical and accessible public discourse.
And indeed, today we hear the explanation of grades 11-12 issue. Anyone can readily comprehend it and ask the obvious question “OK, so did it apply to all such schools, or only to the Christel one?”
Rick Hess *did* ask this question and Bennett’s, answer was “I think grades changed for all 13. All 13 didn’t have 11th or 12th grades the way our system would recognize them, so they were all being calculated at zero for those particular measures.” I am sure Indiana’s reporters are checking that as we speak.
Peace … so far (smile).
RIck asked whether the change did, in the end, apply to all schools. He did not ask why Bennett explored the possibility of changing the grade for only one school.
Let’s get Tony’s side of the story before drawing any final conclusions. I am hearing that the school in question did not have 11th and 12th grade students, and that the initial formula was giving them zeros for things like Advanced Placement and similar items that they had little chance to place on.
Stand by….
Color me skeptical. If it were something like that, they would never have considered changing the grade only for this one school without altering the larger system. But they did consider that (or so the news is reporting). Also, Bennett was given the chance to comment for this story; hard to think that he wouldn’t have produced a legitimate explanation if there were one, or that the news organization wouldn’t have included it in the story if he had. At this point, it’s difficult for me to imagine any explanation for these emails that wouldn’t make the process and the decisions look bad. I’m just trying to emphasize that Bennett is not the problem.
It’s not like this is the first report we’ve had of a state grading system being jiggered to produce certain results.
Perhaps the whole system and not the grade for this single school did change. This blog post provides some additional context:
http://blogs.indystar.com/education/2013/07/29/some-context-for-tony-bennetts-emails/
The first report back from the front during a battle is almost always wrong. Let’s see what Tony has to say for himself before relying upon a few out of context emails and the impartiality of the press to draw conclusions.
But the emails clearly show they did actively consider changing just the grade of one school; they would not have done that if the problem were a glitch in the system (i.e. schools without a certain grade getting zeros in that grade).
And as I said before, Bennett has already had a chance to respond. His responses are in the article.
In the item you link to, Scott Elliott notes essentially the same thing I said here – that with any accountability system, you have to do a “reality check” by asking questions such as “did these specific schools, which we regard as high-performing, get bad grades?” But it is equally true that the selection of which schools are used for such reality checks is colored by political power. The Bennett emails highlight this. The proper conclusion is not that Bennett is bad but that government accountability systems are always subject to this kind of problem.
Two elements of the formula were Advanced Placement and High School Graduation. The school in question ended in 10th grade and there were a dozen other schools in a similar boat.
I don’t know why this didn’t make it into the story but it isn’t the first time we’ve seen an incomplete news story.
[…] to ditch exams aligned with Common Core reading and math standards developed by PARCC; Common Core foes are already using the report as another one of their justifications for opposing the […]
Why didn’t he use this formula when public schools didn’t look so good?
http://www.indystar.com/article/20130731/NEWS05/307310091/Break-given-Christel-House-could-spared-two-IPS-schools-from-state-takeover
[…] “Tony Bennett Is Having a Bad Week”, Greg Forster, Jay P. Greene’s Blog, July 29, 2013 (“Any kind of evaluation system must involve qualitative as well as quantitative testing. That is, you not only have to make sure the numbers are accurately collected, crunched and reported, you have to make sure that what the system is calling ‘good’ really is good.”). […]