Say It All Together Now Class: TESTING ≠ ACCOUNTABILITY

March 28, 2014

(Guest Post by Matthew Ladner)

So I tuned into the Fordham Foundation podcast yesterday, only to find that Fordham is stubbornly holding onto a misapprehension that their own research ought to have disabused them of long ago, namely that standardized testing equates to “accountability.”

This came up in a discussion of the Arizona ESA court ruling.  Broad misunderstandings of the program were on display, especially regarding the term “accountability.”

Sigh. Let’s start with the basics. The dictionary defines the word accountability as:

the quality or state of being accountable; especially :  an obligation or willingness to accept responsibility or to account for one’s actions.

Standardized testing is widespread in education, but “accountability” is scarce indeed.  So in my state more or less every public school student takes the AIMS test, but you would struggle to find anyone who is held “accountable” for the results.  Forty percent of 4th graders in Arizona scored below basic on the 2013 NAEP reading test, but good luck finding a policymaker, superintendent, teacher or parent who was held “accountable.”  Strangely enough, no one accepted responsibility for this sad state of affairs, making this a very unique form of accountability one where no one  is ever held responsible.

So what Mike and Michelle seem to actually be talking about is academic transparency to the public.  Arizona’s ESA program is indeed lacking in any form of academic transparency to the public.  There are a variety of forms this could take, some much more sensible than others, and Arizona policymakers would do well to pick one of them.  If they don’t pick one of the reasonable models, one must assume that an unreasonable model will be imposed sooner or later.  I’ve testified on a number of occasions at the Arizona legislature that policymakers should embrace transparency in the program. I will keep on doing it in the future.

Now let’s discuss the difference between faux and vrai accountability.

Kathy Visser, the mother of Jordan Visser, an ESA student featured in the above video, testified before the Arizona legislature regarding accountability recently.  She more or less noted that for all this shallow talk of “accountability” in this particular hearing (holding the same flawed understanding of the term displayed by Fordham) that there is in fact accountability in the ESA program. Everyone who educates Jordan is directly accountable to her.

Good luck getting that level of accountability in the public school system.

I followed up with Ms. Visser in a subsequent conversation. She experienced a number of difficulties in the public school system that are sadly common for special needs families. She had an open enrollment request denied without explanation with a public school official going so far as to hang up the phone on her. She consulted a specialized attorney who helps special needs families, but found the $15,000 retainer financially out of reach.  Fortunately the attorney told her about the ESA program.

Ms. Visser first tried a private school for Jordan.  She related that Jordan did not have a terrible experience in the private school, but that she decided to try the customized education approach with private tutors and therapists featured in the video above. Ms. Visser agreed that with a school voucher like the McKay Scholarship Program, she would have been able to hold the public schools accountable for the services they provided Jordan.  With an ESA, she can hold all providers accountable private schools, tutors, therapists, you name it.

That my friends is true accountability, you know, the kind where people actually get held responsible for their results.   Not the largely phony kind of accountability where states administer dummied down academic exams with massive item exposure, dropping cut scores, and all sorts of statistical games and tricks and other problems that I have read about in Fordham reports with most states obscuring things further behind fuzzy labels whose scale almost no one understands.

The type of “accountability” that Fordham is talking about however has proven to be baloney in most states for decades now.  Even in states with the most useful testing systems, like Massachusetts and Florida, you won’t find any parents wielding the type of authority exercised by Kathy Visser. It’s long past time for us to recognize the difference between genuine accountability and mere bullshit accountability.

UPDATE: In the interest of fairness please note that Mike did say he supports the ESA program in the podcast and expressed that we should let this experiment play out.  My point is not to claim that the ESA program is perfect (it isn’t) but rather that our notions of what constitutes “accountability” badly need a reboot.

9 Comments | Education Savings Accounts, school accountability | Tagged: , , | Permalink
Posted by matthewladner

In Defense of “Achievement Gap Mania”

October 19, 2011

(Guest Post by Matthew Ladner)

So the early appearance of the 2011 NAEP has given me reason to update a project, leaving me with some interesting charts to burn off. The above chart measures the national White-Black achievement gap for all four of the main NAEP exams for the 2003-2009 period. Mind you, that on these exams, 10 points is approximately equal to a year worth of average academic progress. These are White scores minus Black scores, with the 2003 gaps in Blue and the 2009 gaps in Red.

In Jay’s post below, you can watch a Fordham discussion that includes debate over whether we have fallen into the grip of “achievement gap mania.” If so, we have precious little to show for it. We did have some narrowing of the achievement gap between 2003 and 2009, but at two and a half plus grade level gaps in all four subjects. Start your low-calorie, carrot juice diet and mark your calendar for 2075 or so, assuming that we can maintain today’s glacial pace of closing.

The news is approximately as dismal on the White-Hispanic front:

While I do sympathize with the argument that we need to get everyone to understand their stake in education reform, I must say that there is a reason why people are passionate about achievement gaps. The term “disgraceful” does not begin to describe the catastrophic failure represented in the charts above. Black and Hispanic children score little better than what the average 1st to 2nd grade Anglo student would score on a 4th grade reading test. It’s only the developmentally critical literacy acquisition window after all.

The focus on the achievement gap is important because it cuts to the heart of American ideals. We believe in equality of opportunity. We believe in meritocracy. We believe in class mobility and self-determination. Call it the triumph of hope over experience if you wish, but we believe that public education can help achieve all of this and we refuse to give up on the notion.

The terrible truth of course is that our public education system is pervasively classist to an extent that goes far deeper than the naive equity funding attorneys ever seemed to grasp. If we auctioned the limited supply of high quality public school seats on Ebay rather than covertly through mortgages, perhaps all of this would more transparent. If we could tag our highly effective instructors, we could watch a time-lapse film of them fleeing dysfunctional school systems for the leafy suburbs and/or leaving the profession entirely. Increased resources could in theory ameliorate these problems, but strangely enough they didn’t.

Why? Paul Hill said it best:

Money is used so loosely in public education—in ways that few understand and that lack plausible connections to student learning—that no one can
say how much money, if used optimally, would be enough. Accounting systems make it impossible to track how much is spent on a particular
child or school, and hide the costs of programs and teacher contracts.  Districts can’t choose the most cost-effective programs because they
lack evidence on costs and results. 

The sad thing is, some are so desperate to maintain the above paragraph that they are willing to ignore the consequences, including the two charts above. They comfort themselves with excuses. Blah blah poverty yadda yadda video games. Whatever. I’m not saying that achievement gaps are the sole responsibility of schools, or that we will live to see them completely closed. I agree with Rick Hess that there are serious shortcomings to a reform strategy solely based on gaps.

We can however do a hell of alot better than this. We focus on achievement gaps not because it is expedient, but because it is necessary.

24 Comments | myth of helplessness | Tagged: , , | Permalink
Posted by matthewladner

Strawman Alert!

April 20, 2011

 

(Guest Post by Matthew Ladner)

I went to read the Fordham Report on ESEA reauthorization. I didn’t even make it past the preface without finding a gigantic strawman argument:

The local controllers.

These folks, led by conservative and libertarian think tanks such as the Heritage Foundation and the Cato Institute, want Uncle Sam, for the most part, to butt out of education policy—but to keep sending money. They see NCLB as an aberrant overreach, an unprecedented (and perhaps unconstitutional) foray into the states’ domain. Many within this faction also favor reform, particularly greater parental choice of schools, but at day’s end their federal policy position resembles that of the system defenders. They want to keep federal dollars flowing, albeit at a much more modest rate than those on the left; but they want to remove the accountability that currently accompanies these monies. They have given up on Uncle Sam as an agent for positive change, period. And they have enormous confidence that communities, states, and parents, unfettered from and unpestered by Washington, will do right by children.

I’ll let the Cato Institute speak for itself, but as the coauthor of a piece on NCLB with Gene Hickok for the Heritage Foundation, I must say that this characterization of Heritage is sloppy. Gene and I noted some very real problems with the formulation of NCLB, and recommended a process by which states could negotiate with the federal Department of Education to have a single unified system of school accountability. No burning down the Federal Department of Education, no abandonment of accountability and transparency, nor any fever dreams of federally driven vouchers for all.

NCLB led to a net increase in transparency, and put a bright spotlight on achievement gaps- both very admirable outcomes. NCLB’s formula however contains dozens of ways for districts and schools to fail AYP and back loaded proficiency requirements will be changed, or else AYP with either lose all credibility, or else will lead states to dummy down their tests to absurd levels. The only reasonable assumption to make is that those that crafted the original law intended to reboot the provisions well before 2014. The Safe Harbor provision is not going to save the day, lawmakers must change the law.

Gene and I suggested a reboot that would allow states to have a single system of school accountability (many have a state system and AYP, which often contradict each other). States proposing a reasonable system- something AYP will no longer be in 2014 absent changes-could have a single system for ranking schools. I’m fine with the Federal Department of Education being tough-minded about approving alternatives. No federalist bone in my body would ever compel me to approve a cruel joke of a testing system (I’m looking straight at you Mississippi) and I’m not certain that the Obama administration has a federalist bone in any case. They did however win the election, and they may win the next one as well.

Call me crazy (it’s been too long since anyone has) but I think the federal government allowing parents the clarity of a single system of accountability is good thing if the state is proposing something that provides transparency and will nudge improvement out of the system. Not “perfection” by some arbitrary deadline, but sustained improvement. This strikes me as an especially good idea when the federal system is set to implode.

9 Comments | NCLB | Tagged: , | Permalink
Posted by matthewladner

Fordham Responds on Nationalizing Education

March 30, 2011

Over at Flypaper, Fordham’s Kathleen Porter-Magee responds to my post yesterday about the mistake of the current Gates-Fordham-AFT-USDOE effort to nationalize key aspects of our education system.  She writes:

Of course, many people agree that Betamax had the superior technology (the picture was sharper, the cassettes were smaller, it was better at high-speed duplication, etc.). So, in effect, market forces standardized the inferior technology.

But rather than belabor the VHS-Betamax analogy, let’s talk about the actual case of state standards. Is Greene correct in his contention that the market was on its way to standardizing high-quality state standards? Not even close.

In fact, for more than a decade we have been conducting a natural experiment where we let market forces drive standards setting at the state level. The result? A swift and sure race to the bottom. A majority of states had failed to set rigorous standards for their students—and had failed to create effective assessments that could be used to track student mastery of that content. In fact, the whole impetus behind the Common Core State Standards Initiative was to address what was essentially a market failure in education.

That said, I do agree with Greene that too much government intervention will stifle innovation. That’s precisely why I think government “standardization” should begin and end with standards. Let the government define what students should know and be able to do.  Then let market forces determine which curricula and pedagogy will best help students master that essential content.

To which Ze’ev Wurman replies:

I have a lot of respect for Kathleen and hence I am stumped.

She writes that the results of the NCLB’s “natural experiment” with states setting their standards are clear: “A swift and sure race to the bottom.”

Yet just a few years back no other than the Fordham Institute itself examined this exact issue,the behavior of proficiency standards under NCLB, and declared:

“These trends do not indicate a helter-skelter ‘race to the bottom.’ They rather suggest more of a walk to the middle.”

Perhaps Kathleen meant to write about the rigor of content standards rather thanproficiency standards. But there, too, many states have improved their standards, rather than lowering them. This can be clearly visible in — yet again — Fordham’s own recent “State of the Standards” report that shows that in 2010, 27 state ELA standards were graded worse than in 2005 and 11 improved (with 12 grades unchanged). In math only 10 state standards were graded worse and 29 improved, with 11 graded the same. I might add that grading criteria in 2010 were more demanding than in 2005 as can be clearly seen from Massachusetts’ standards that did not change between 2005 and 2010, yet were graded lower in 2010 than in 2005. In other words, by Fordham’s own analysis — of which Kathleen must be aware as she co-authored it — state content standards have improved somewhat over the years.

So which one is it? Is there a race to the bottom, or isn’t there? Based on Fordham’s own research there was an improvement in content standards and no race to the bottom in proficiency standards. Yet Kathleen is unequivocal in claiming a race to the bottom. Is it a simple error, or has Fordham started to twist its own findings in its push to support national standards?

And I add:

In addition to the misleading claim of “race to the bottom” that Ze’ev notes, Kathleen’s post is in error on two other points:

1) VHS was not the “inferior technology.” It was cheaper, had longer tapes, and the market clearly preferred those things over whatever qualities Betamax possessed. Kathleen’s conviction that she and some central government-backed committee of like-minded people know what is best for the country regardless of what the market says is precisely the problem with the Gates-Fordham-AFT-USDOE effort to nationalize key aspects of education policy.

2) The claim that Kathleen and Fordham want no more than to nationalize standards without touching curriculum, pedagogy, or assessment is simply disingenuous. For example, Checker once again made common cause with the AFT, Linda Darling-Hammond, etc… in backing the Shanker Manifesto, which calls for “Developing one or more sets of curriculum guides that map out the core content students need to master the new Common Core State Standards.” Checker may claim that this effort is purely voluntary, but that would only be credible if he and Fordham clearly and forcefully opposed any effort by the national government to “incentivize,” push, prod, or otherwise require the adoption of national curriculum based on the already incentivized national standards. And of course, USDOE (without any opposition from Fordham that I have noticed) is already moving forward with developing national assessments even before national curriculum has been developed. One does not need to be from one of “the more feverish corners of the blogosphere” to recognize the odd coalition of Gates-Fordham-AFT-USDOE as coordinating an effort to nationalize key aspects of our education system.

11 Comments | instructional reform, school accountability | Tagged: , , , , | Permalink
Posted by Jay P. Greene

Mandating Betamax

March 29, 2011

I just returned from the Association for Education Finance and Policy annual conference in Seattle, which was a really fantastic meeting.  At the conference I saw Dartmouth economic historian, William Fischel, present a paper on Amish education, extending the work from his great book, Making the Grade, which I have reviewed in Education Next.

Fischel’s basic argument is that our educational institutions have largely evolved in response to consumer demands.  That is, the consolidation of one-room schoolhouses into larger districts, the development of schools with separate grades, the September to June calendar, and the relatively common curriculum across the country all came into being because families wanted those measures.  And in a highly mobile society, even more than a century ago, people often preferred to move to areas with schools that had these desired features.  In the competitive market between communities, school districts had to cater to this consumer demand.  All of this resulted in a remarkable amount of standardization and uniformity across the country on basic features of K-12 education.

Hearing Fischel’s argument made me think about how ill-conceived the nationalization effort led by Gates, Fordham, the AFT, and the US Department of Education really is.  Most of the important elements of American education are already standardized.  No central government authority had to tell school districts to divide their schools into grades or start in the Fall and end in the Spring. Even details of the curriculum, like teaching long division in 4th grade or Romeo and Juliet in 9th grade, are remarkably consistent from place to place without the national government ordering schools to do so.

Schools arrived at these arrangements through a gradual process of market competition and adaptation.  Parents didn’t want to move from one district to another only to discover that their children would be repeating what they had already been taught or were  inadequately prepared for what was going to be taught.  To attract mobile families, districts informally and naturally began to coordinate what they taught in each grade.  Of course, not everything is synced, but the items that are most important to consumers often are.

That’s how standardization in market settings works and we have a lot of positive experience with this in industry.  VHS became the standard medium for home entertainment because the market gravitated to it, not because some government authority mandated it.  If we followed the logic of Gates-Fordham-AFT-USDOE we would want some government-backed committee to decide on the best format and provide government subsidies only to those companies that complied.

Instead of ending up with VHS, they may well have imposed Betamax on the country, even though market competition would have shown that approach to be inferior.  Sony was the industry leader and if a government-backed committee were in charge they almost certainly would have had the most influence.  The Fordham folks might want to keep this in mind.  A government-backed committee is almost certain to prefer what the AFT wants over what Fordham may envision since the teacher unions are like Sony except only 100 times more powerful.

Even worse, once government-enforced standardization occurs it becomes extremely difficult to change.  If we had a government-backed panel decide on Betamax, we may have been stuck with that format for decades.  We almost certainly would have stifled the innovation that led to DVDs and now Blue-Ray.  Once Sony had entrenched their format, what incentive would they have had to change it?

Similarly, once the Gates-Fordham-AFT-USDOE coalition settles on the details of nationalizing standards, curriculum, and testing, it will become extremely difficult to change anything about education.  Terry Moe and Paul Peterson’s dreams of technology-based instruction may never leave the dream stage because it may fail to comply with certain provisions of the national regime.  If I were the AFT, I’d almost certainly insert those details into the regime to prevent the reductions that technology may bring to the need for teaching labor.  No one should be naive enough to think the Edublob won’t figure out how to use nationalization to block that and other threatening innovations.

I’m also sure that Bill Gates would have preferred being able to get a government-backed committee to enshrine Microsoft-DOS or Windows forever.  But thanks to market competition we have Google innovating with cloud computing.  And I’d bet that Google would love to get government backing for their approach if they could.  Dominant companies almost always favor government regulation.

So I understand why the AFT, USDOE, and Gates favor the current effort to nationalize education.  The mystery to me is why Fordham is protecting the right-flank of this movement or why some conservative governors have gone along.  Don’t they realize that it will enshrine arrangements that favor the teacher unions and are bad for kids?

8 Comments | instructional reform, school accountability | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Does Fordham Support a National Curriculum?

August 5, 2010

 

(Guest post by Greg Forster)

For weeks, Checker has been calling us “paranoid” for worrying that the national standards machine Fordham has helped create will be hijacked by the teacher unions.

Today, there lands in my inbox the new Gadfly from Fordham, featuring a guest editorial by Eugenia Kemble of the Shanker Institute. Kemble’s argument, in a nutshell: Now that we have national standards, the next thing we need is a national curriculum. That way we don’t just ensure that all schools set outcome targets and measurements in the one best way that’s right for everyone regardless of their individual needs; all schools will do everything in the one best way that’s right for everyone regardless of their individual needs. And we’ll have a benevolent dictator who will make sure that everyone will do everything in the one best way, and who will never abuse that power.

I paraphrase.

On Kemble’s list of the heroic, wonderful people she admires who have been pushing not just for national standards but a national curriclum are Bill Schmidt and Randi Weingarten at the AFT; teacher union shill Diane Ravitch; and . . . Checker Finn.

Inquiring minds want to know:

  1. Does the Fordham Foundation support a national curriculum?
  2. Given that Fordham is offering up the Gadfly as a platform from which Kemble can advocate using national standards as the first step toward broader federal control of schools, does the Fordham Foundation still consider it “paranoid” to be worried that national standards will be used as a first step toward broader federal control of schools?

I’ll hold my breath and wait for Checker to give us a clear, unambiguous answer.

5 Comments | instructional reform | Tagged: , , , , , | Permalink
Posted by Greg Forster

Let’s Get Ready to Rummmmmble!

August 24, 2009

(Guest Post by Matthew Ladner)

I had a chance to watch the Fordham Foundation’s with ice cream ascendant is there a future for chocolate fudge event on charters and vouchers online. John Kirtley scored an early knockout when he noted that in Jacksonville Florida recently there all of 6 charter schools but 90 private schools serving low-income students through the Step Up for Students Tax Credit program. Kirtley then noted that not all 6 charter schools primarily serve low-income children. He likely could have added that not all six are high quality schools, but that would have been running up the score. Kirtley asked his debate opponents how much longer single mothers with children in the schools should have to wait for high quality school options.

DOWN GOES FRAZIER! DOWN GOES FRAZIER!

Kirtley’s opponents, Kevin Carey and Susan Zelman, raised the predictable totem of “accountability.” This of course is a real issue and a superficially powerful totem, but when you look behind the curtain, the Great and Powerful Oz is just an old man.

I live in a state where 44% of 4th graders scored below basic in 4th grade reading in 2007 and even a little worse in 2005. Who, pray tell, was held “accountable” for that sorry performance? Was a single administrator or teacher fired? Not that I am aware of. Did the public elect a new Superintendent of Public Instruction? Nope- the incumbent was reelected in 2006.

Who was held accountable? Try “not a single human being at all.” Public school “accountability” in short, is a cruel joke with kids as the victims.

Those who want to pretend that giving an all too often dummied down state test tied to a set of often sorry state academic standards constitutes “accountability” have confused their means with their ends. It isn’t the end all be all of accountability, nor is it necessarily really accountability at all.

Done well, I believe standards and testing can be a productive education reform. Choice programs however should be an opt-out of that system into one that is different, but which still contains a vitally necessary level of transparency. Something like the Stanford 10 will work nicely.

Kirtley’s point was the key: if we are really interested in helping disadvantaged children, all options must be on the table. Otherwise, pro-charter but anti-private choice folks do indeed come across like the gradualist white liberal wimps who urged the leaders of the civil rights movement to be “patient.”

Patience can be a virtue, but not when your hair is on fire.

7 Comments | charter schools, vouchers | Tagged: , , , , | Permalink
Posted by matthewladner

Eformray Ealismray

December 18, 2008

drevil3

(Guest Post by Matthew Ladner)

Well, a few days after I begged my friend Mike Petrilli to ixnay on the averickmay alktay now he has labeled the Fordham position on federal education policy making as “Reform Realism.” Get it- it’s a lot like “smart growth!”

Don’t get me wrong, even a crusty “Local Controller” like me finds things to admire about the Fordham position. But easplay, no more inguisticlay inspay!

In addition, I thought the Senate voted down national standards 98 to nothing in the late 1990s. Perhaps this is a new fantastic version of realism- but who knows what will happen next?

6 Comments | blogging, instructional reform | Tagged: , | Permalink
Posted by matthewladner

%d bloggers like this: