In Defense of Failure

September 18, 2008

Good can come from failure.  Abraham Lincoln’s failure to capture a Senate seat set the stage for his presidential run.  Winston Churchill’s failure at Gallipoli prepared him for the strategic challenges of WW II.  Recognizing failure and trying to move forward is almost always better than pretending that you haven’t really failed.

So why is the federal government preventing the failure of financial firms, such as AIG or Bear Stearns?  Providing government loans to AIG or guaranteeing the buyer of Bear Stearns against loss did not alter any of the financial reality.  Nothing new was produced or created.  No new capital was created; it was simply transferred from taxpayers (either by contributing to inflation or by adding to national debt) to people who do work for or business with these firms. 

Conversely, if these firms had instead been allowed to fail by going bankrupt nothing would have been destroyed.  All of the financial capital held by these firms would still exist.  And all of the human capital of the people who works for those firms would still exist.  Bankruptcy doesn’t mean that you take all of the capital of a firm, put it in a pile, and blow it up.  It’s all still there.  What bankruptcy does is it forces people to reorganize what they will do with that financial and human capital.  That is, they are forces to recognize their failure and figure out a better way to do things.  

And even more importantly, failure forces people involved with these firms to experience the consequences of their actions.  Those people — and everyone else — learns from those consequences and hopefully changes their behavior in the future.  To prevent failure is to prevent learning.

The same is true for schools.  Good can come from failure.  Of course, we’d prefer to avoid failure, if at all possible.  But if the reality is that students or educators or schools have failed, then insulating people from that reality doesn’t do anyone any favors.  No new knowledge is created by hiding failure and none is destroyed by recognizing it.  Admitting that students, educators, or schools have failed allows us to reorganize how we do things and to all learn important lessons.

13 Comments | school accountability | Tagged: , , , | Permalink
Posted by Jay P. Greene

Eduwonkette or Jennifer Jennings, Makes No Difference

August 26, 2008

I’m glad that Eduwonkette decided to end her anonymity and identify herself as Jennifer Jennings, a sociology graduate student at Columbia University.  I’m not glad because I think it was inherently wrong for her to blog anonymously.  As I’ve previously written: “The issue is not who Eduonkette is, but whether she is right or not.  Knowing who she is does not make her evidence or arguments any more or less compelling.” 

The problem was that Eduwonkette did not share my belief in the principle that we should focus on the veracity rather than the source of claims.  She repeatedly emphasized the credibility of the source of information.  Emphasizing the credibility of sources while blogging anonymously, preventing analysis of her own credibility, was logically untenable and had to end.  I wished that she would end this inconsistency by embracing the view that ideas are true or false independent of their sources.  Instead she has resolved her inconsistency by ending her anonymity.

Now that we know that Eduwonkette is Jennifer Jennings we can see another prominent example of the logical inconsistency of blogging anonymously while focusing on the credibility of sources.  In 2005 Jennings published an article in the American Educational Research Journal that argued that accountability systems encouraged schools to focus on the achievement of “bubble” students — those close to an achievement cutoff — at the expense of high and low achieving students.  She arrived at this conclusion after visiting a school in Texas and observing it for a period of time.  She was aided in drawing this conclusion by using jargon like “neoinstituionalism” and “normative isomorphism,” but I kind of zoned out during that part of the article.  I’m guessing that neoinstitutionalism is bad following my theory that anything starting with “neo” is supposed to be bad while anything starting with “post” is supposed to be good.

A few years later Matthew Springer published articles in Education Next and in the Economics of Education Review that empirically examine Jennings’ claim of “educational triage.”  The Education Next piece actually began with a lengthy quote from Jennings’ article as a foil for its findings that NCLB accountability improved the achievement of “bubble” students, but not at the expense of lower and higher achieving students.

Jennings then took-on her critic, Springer, but she did so as the anonymous blogger Eduwonkette, never revealing that she was attacking the person who criticized her own research.  And her first argument against Springer was that his work was published in Education Next and “Education Next is not a scholarly journal.”  Jennings targets the source of the critique of her own work while concealing her identity to prevent analysis of her as a source.  The irony is too rich.

Why Jennings did not just focus on the version of the paper published in Economics of Education Review or the unabridged version linked to on Education Next, , which would have been free of the unscholarly taint she perceives in Education Next,  is unclear.  It was obviously important for her to discredit Springer’s argument against her own study by attacking the credibility of Education Next as the source of Springer’s argument — all the while preventing assessment of her credibility by doing all of this anonymously.

Let me be clear that I have no problem with Jennings defending her own work anonymously.  Her arguments against Springer are true or false regardless of who she says or doesn’t say she is.  My point is that by arguing her own case anonymously, Jennings betrays the principles that she appears to endorse.  Namely, if the source of information is important in assessing claims, it would clearly be inappropriate to attack your critic without revealing who you are. 

Even now that Jennings has revealed her identity, I hope that she abandons her reliance on assessing the source of claims.  Doing so would justify her past actions and help us move forward in analyzing ideas rather than analyzing people and motives.

6 Comments | school accountability | Tagged: , , , , | Permalink
Posted by Jay P. Greene

AFT Goes Up in Smoke

August 6, 2008

(Guest Post by Matthew Ladner)

[Editor’s note — See new post on “Broader, Bolder” here.]

P.J. O’Rourke once described the early Clinton administration as “running the country by dorm-room bull session.” Some recent ferment among education progressives makes me wonder if they too have fallen back onto some old college habits. Catherine Johnson over at Kitchen Table Math for instance wrote on Randi Weingarten’s first speech as AFT President. Weingarten engages in NCLB bashing, and then lays out a vision for the future of public education:

“Imagine schools that are open all day and offer after-school and evening recreational activities and homework assistance … and suppose the schools included child care and dental, medical and counseling clinics, or other services the community needs,” Ms. Weingarten said. “For example, they might offer neighborhood residents English language instruction, GED programs, or legal assistance.”

Personally, I’m trying to imagine a system of public schools that could teach 4th grade kids how to read after spending $40,000 or more on their education. In 2007, 34% of American public school 4th graders scored below basic in reading on the NAEP. If we can’t trust schools to teach kids how to read, just why would we want them trying to fix our teeth or attempting to resolve our legal issues?

Weingarten echoes the “bigger and bolder” crowd, who seem to believe that schools can become more effective by becoming less focused on academics. Given the AFT opposition to standardized testing, these schools social welfare centers will ideally be free to thrive without the burden of academic transparency.

This of course is precisely the wrong direction to take. Paul Hill recently conducted a series of studies for the Gates Foundation concerning the stubborn lack of academic progress despite increased public school spending. After a series of studies, Hill reached the conclusion:

“…money is used so loosely in public education – in ways that few understand and that lack plausible connections to student learning – that no one can say how much money, if used optimally, would be enough. Accounting systems make it impossible to track how much is spent on a particular child or school, and hide the costs of programs and teacher contracts. Districts can’t choose the most cost-effective programs because they lack evidence on costs and results.” (Hat tip: Nevada Policy Research Institute’s Steven Miller)

Summarizing then, public schools have yet to do a cost-benefit analysis on the nearly $10,000 per year per child they are already spending. They therefore have a very poor idea about which of their activities help achieve the goal of producing a well educated child, and which do not. They, in essence, just do what they do, which certainly helps explain how a school system could burn through tens of thousands of dollars without teaching a child to read.

Let me be specific. In Arizona, 44% of 4th graders score below basic in reading. Despite that fact, we have elementary school days that include a regular coursework in art, music and physical education. These offerings are of course enriching and wonderful for many children. Why however would a 3rd grader who can’t read be taking courses in art or music? We know that children not gaining basic literacy skills in the early grades are all but doomed to academic failure.

Could it be the case that schools should reallocate their resources under the theory that one’s lifelong ability to appreciate music and art would be greatly enhanced by learning how to read?

We’ve got quite a problem to sort out here and I will submit that the last thing we would want to do is get schools even less focused on academic achievement. It isn’t hard to imagine burning through even more money while still failing to teach basic academic skills to large numbers of kids: schools have been doing it for more than 40 years.

(edited for typos)

13 Comments | myth of helplessness, school accountability, teacher union | Tagged: , | Permalink
Posted by matthewladner

NCLB: Less Than Meets the Eye, More Than Nothing

July 29, 2008

Given all of the wailing and gnashing of teeth, one would think that NCLB was a crushing burden on the school system.

In actuality NCLB demands very little. It only requires that states wishing to receive Title I funds have to establish goals for student success, select tests for measuring progress towards those goals, and report results from those tests broken out by subgroups.

The sanctions for failing to make progress toward those goals are almost non-existent. Schools failing to make progress have to offer tutoring or allow students to transfer to better-performing public schools in the same district (if one can be found). But, as we have previously discussed on this blog and in this article, there is widespread non-compliance with even these minimal sanctions. Too often schools fail to inform parents properly of their options under NCLB or direct students into their own tutoring programs, resulting in very few students taking any resources out of their local school, let alone district. Without placing school funds in jeopardy, the only possible sanction is public embarrassment. And that plus $4 will get you a latte at Starbucks.

I do not believe that a single tenured teacher out of the more than 3 million teachers currently working in public schools has been fired, experienced a pay-cut, or otherwise been meaningfully sanctioned because of NCLB. I do not believe that a single student out of the 50 million enrolled in public schools has been held back a grade, been denied a diploma, or otherwise been meaningfully sanctioned as a result of NCLB. (Some states have retention and graduation requirements as part of their state accountability systems, but those policies are not required for NCLB.) Yes, chronically failing schools might eventually face “restructuring” but that is likely to be yet more bark and no bite. Next they’ll be put on double secret probation.

So what supports complaints about “pressure cooker NCLB testing,” or “NCLB-post traumatic stress disorder,” or other “NCLB outrages”? If NCLB has almost no real consequences for teachers or students, what is all of the fuss about? The overwrought reaction seems to have more to do with a political campaign over the future direction of education policy than the actual effects of the current policy.

The most important future policy that the higher volume of squealing is meant to influence is increasing education spending. A center-piece of the complaints about NCLB is that it is an unfunded mandate. Let’s leave aside the fact that federal spending on education has increased 41% since passage of NCLB. And let’s leave aside that NCLB is not actually a mandate, since states do not have to comply with NCLB if they do not want Title I funds (which have increased 59% since 2001).

Besides neither being unfunded nor a mandate, the argument that NCLB is an unfunded mandate is especially odd because it makes one wonder what all of the funding that schools received before NCLB was for. It’s as if the unfunded mandate crowd is saying: “The $10,000 per pupil we already get just pays for warehousing. If you actually want us to educate kids, that’ll cost ya extra.” Remember, that NCLB just asks states to establish and meet their own goals. Didn’t they have goals before NCLB?

While NCLB demands much less than the overwrought rhetoric about it suggests, it does not demand nothing. Most importantly, NCLB entrenched the idea that we should take regular measures of student achievement and report the results, including results for subgroups. Even this is a smaller thing that it may seem at first glance since 37 states had already adopted state testing and accountability systems before passage of NCLB. But NCLB brought the laggard states on-board to this growing national consensus that we ought to have some systematic measures of how our students are doing. It also made reversal of this growing testing and accountability culture more difficult by placing it in federal as well as state law.

Greg Forster has already made the case for why this shift under NCLB has been important, so I will not repeat it here. I would just emphasize that the controversy over NCLB is not really about what NCLB does, but about the broader policy shift that it represents and the extra funding that folks hope they may get as they acquiesce to that policy shift.

26 Comments | school accountability | Tagged: , , , , | Permalink
Posted by Jay P. Greene

When “Sorry” Means “J’Accuse”

July 28, 2008

The letter "J'accuse"

The following column about a letter appeared on the front page of the Cleveland Plain Dealer on Wednesday, July 23.  The letter is in the form of an apology, but it is actually a series of accusations about testing and accountability.  Like another front page letter of accusation, this one has all of Emile Zola’s moral outrage but has none of Zola’s justification.

I’ve reprinted it here with my comments in blue italics.

Students pass state test, but at what cost to their education?

by Regina Brett

The school report cards came out in June.

Rocky River Middle School passed the 2008 Ohio Achievement Tests, earned an Excellent rating from the state and met the requirements for Annual Yearly Progress.

For all of those accomplishments, Principal David Root has only one thing to say to the students, staff and citizens of Rocky River:

He’s sorry.

Root wants to issue an apology. He sent it to me typed out in two pages, single spaced.

He’s sorry that he spent thousands of tax dollars on test materials, practice tests, postage and costs for test administration.

Actually, he did not spend the money.  The taxpayers did when they decided through their elected representatives to adopt a testing and accountability system.  They then hired David Root to implement this policy in his capacity as principal at a public school.

Sorry that his teachers spent less time teaching American history because most of the social studies test questions are about foreign countries.

I guess the people of Ohio thought it was important for students to learn about foreign countries when they, through their elected representatives and hired agents, devised the state curriculum and test.  Besides, if students learned more about foreign countries they might know who Emile Zola was.

Sorry that he didn’t suspend a student for assaulting another because that student would have missed valuable test days.

Sounds pretty irresponsible.  Would he have made a different decision if the student would have missed valuable instructional days?  If so, whose fault is that?  Oh yes, I forgot that this is an accusation, not an apology.

Sorry he didn’t strictly enforce attendance because all absences count against the school on the State Report Card.

So, is David Root saying that he cheated on the state accountability system?  Isn’t this like lying to your boss about your job performance?  Will he be fired, sanctioned, or resign to make amends for his infraction?

He’s sorry for pulling children away from art, music and gym, classes they love, so they could take test-taking strategies.

Why didn’t he just follow the state curriculum and let the scores show what students knew? The decision to take time away for “test-taking strategies”  was completely unnecessary given that more than 90% of Rocky River students have been scoring above the proficient level in reading, math, and writing.  It sounds like they would have done just fine on the state test without working on test-taking strategies and having spent more time on art, music, and gym.

Sorry that he has to give a test where he can’t clarify any questions, make any comments to help in understanding or share the results so students can actually learn from their mistakes.

How reliable would the results be if principals could clarify questions, help in understanding, or share secure test items that would be re-used on future tests?  Does every assessment have to be a formative assessment?

Sorry that he kept students in school who became sick during the test because if they couldn’t finish the test due to illness, the student automatically fails it.

This sounds like a difficult decision.  Football coaches similarly have to think about whether to take injured players out of the game versus having the players tough it out.  We pay leaders to make these difficult decisions, balancing competing interests wisely.

Sorry that the integrity of his teachers is publicly tied to one test.

Actually, the state accountability system — let alone “the integrity of his teachers” —  is not based on one test.  The overall rating of Rocky River Middle School is based on several test results (in Reading, Math, Writing, Social Studies, and Science), the progress students have made in those subjects, and (as we already heard about) the possibly fraudulent attendance rate.

He apologized for losing eight days of instruction due to testing activities.

I thought Root didn’t want one test, so it takes time to administer several.  While testing takes place on eight days it does not (or at least does not have to) consume the entirety of those days.  My understanding is that the average student only spent two mornings being tested, as testing occurred in different grades and subjects for different students across eight days.

For making decisions on assemblies, field trips and musical performances based on how that time away from reading, math, social studies and writing will impact state test results.

I would hope that the principal would think about how assemblies, field trips, and musical performances impact instructional time for other academic subjects regardless of whether those subjects are part of a state accountability system.

For arranging for some students to be labeled “at risk” in front of their peers and put in small groups so the school would have a better chance of passing tests.

Again, if smaller group instruction would help certain students, the principal should arrange for that regardless of the state accountability system.  And the principal would have to think of a way to provide that necessary assistance without stigmatizing the students who need it.

For making his focus as a principal no longer helping his staff teach students but helping them teach test indicators.

Why didn’t he just help his staff teach the subjects with confidence that the test indicators would show what they had learned?  This is especially puzzling given how likely it is that students at Rocky River would pass the state test without paying any special attention to test-taking strategies.

Root isn’t anti-tests. He’s all for tests that measure progress and help set teaching goals. But in his eyes, state achievement tests are designed for the media to show how schools rank against each other.

Seems like the state accountability system does measure progress and help set teaching goals.  What’s wrong with it also informing the public and policymakers (via the media) about how their schools are doing?

He’s been a principal for 24 years, half of them at Rocky River Middle School, the rest in Hudson, Alliance and Zanesville. He loves working with 6th, 7th and 8th graders.

“I have a strong compassion for the puberty stricken,” he joked.

His students, who are 11, 12, 13 and 14, worry that teachers they love will be let go based on how well they perform.

One asked him, “If I don’t do well, will you fire my teacher?”

He cringed when he heard one say, “I really want to do well, but I’m not that smart.”

Has a single tenured teacher in Ohio (or in the United States) been let go based on performance on state accountability tests?  Maybe he should reassure the students that their concern is misplaced.

He wants students to learn how to think, not take tests.

Can’t they do both?

“We don’t teach kids anymore,” he said. “We teach test-taking skills. We all teach to the test. I long for the days when we used to teach kids.”

Why not just return to those days and let the test results show what kids have learned?

Unless we get back to those days, principals and teachers all over Ohio will continue to spend your tax dollars to help students become the best test takers they can be.

The people of Ohio decided to adopt an accountability system because the schools weren’t doing an adequate job teaching kids to think without it.  The “just trust us to do a good job” approach wasn’t working.

(edited to add color)

7 Comments | school accountability | Tagged: , , , , , , , | Permalink
Posted by Jay P. Greene

Eduresponses to Edubloggers

July 10, 2008

My recent posts on the release of our new study on the effects of high-stakes testing in Florida and posts here and here on the appropriateness of releasing it before it has appeared in a scholarly journal, have produced a number of reactions.  Let me briefly note and respond to some of those reactions.

First, Eduwonkette, who started this all, has oddly not responded.  This is strange because I caught her in a glaring contradiction: she asserts that the credibility of the source of information is an important part of assessing the truth of a claim yet her anonymity prevents everyone from assessing her credibility.  I prefer that she resolve this contradiction by agreeing with my earlier defense of her anonymity that the truth of a claim is not dependent on who makes it.  But she has to resolve this one way or another — either she ends her anonymity or she drops the argument that we should assess the source when determining truth.

But apparently she doesn’t have to do anything.  Whose reputation suffers if she refuses to be consistent?  Her anonymity is producing just the sort of irresponsibility that Andy Rotherham warned about in the NY Sun and that I acknowledged even as I defended her.  The only reputation that is getting soiled is that of Education Week for agreeing to host her blog anonymously.  If she doesn’t resolve her double-standard by either revising her argument or dropping her anonymity, Education Week should stop hosting her.  They shouldn’t lend their reputation to someone who will tarnish it.

Mike Petrilli over at Flypaper praises our new study on high stakes testing but takes issue with referencing comments by Chester Finn and Diane Ravitch about how high stakes is narrowing the curriculum in the “pre-release spin.”  I agree with him that this study is not “the last word on the ‘narrowing of the curriculum.’”  But to the extent that it shows that another part of the curriculum (science) benefits when stakes are applied only to math and reading, it alleviates the concerns Checker and Diane have expressed. 

As we fully acknowledge in the study, we don’t have evidence on what happens to history, art, or other parts of the curriculum.  And we only have evidence from Florida, so we don’t know if there are different effects in other states.  But the evidence that high stakes in math and reading contribute to learning in science should make us less convinced that all low stakes subjects are harmed.  Perhaps school-wide reforms that flow from high stakes in math and reading produce improvements across the curriculum.  Perhaps improved basic skills in literacy and numeracy have spill-over benefits in history, art, and everything else as students can more effectively read their art texts and analyze data in history.

Andy Rotherham at Eduwonk laments that what I describe as our “caveat emptor market of ideas” doesn’t work very well.  I agree with him that people make plenty of mistakes.  But I also agree with him that “in terms of remedies there is no substitute for smart consumption of information and research…”  There is no Truth Committee that will figure everything out for us.  And any process of reviewing claims before release will make its own errors and will come at some expense of delay.  Think Tank West has added some useful points on this issue.

Sherman Dorn, who rarely has a kind word for me, says: “Jay Greene (one of the Manhattan Institute report’s authors and a key part of the think tank’s stable of writers) replied with probably the best argument against eduwonkette(or any blogger) in favor of using PR firms for unvetted research: as with blogs, publicizing unvetted reports involves a tradeoff between review and publishing speed, a tradeoff that reporters and other readers are aware of.”  He goes on to have a very lengthy discussion of the issue, but I was hypnotized by his rare praise, so I haven’t yet had a chance to take in everything else he said.

2 Comments | journalism, research reports, school accountability | Tagged: , , , , , , | Permalink
Posted by Jay P. Greene

New Study Release Tomorrow

July 7, 2008

Keep your eyes peeled for the release tomorrow by the Manhattan Institute of a new study on the effect of high-stakes testing on achievement in low-stakes subjects. The study, led by Marcus Winters and co-authored by me and Julie Trivitt, examines whether achievement in math and reading comes at the expense of science on Florida standardized tests.  Because there are meaningful consequences for performance in math and reading, but not for the rest of the curriculum, many people have worried that schools would improve their math and reading results by skimping on science and other subjects.

These concerns are not just coming from the usual critics of school accountability.  Even accountability advocates have expressed second thoughts.  For example, Chester Finn writes in the National Review Online: “Do the likely benefits exceed the ever clearer costs? Boosting skill levels and closing learning gaps are praiseworthy societal goals. But even if we were surer that NCLB would attain them, plenty of people — parents, teachers, lawmakers, and interest groups — are alarmed by the price. I don’t refer primarily to dollars. (They’re in dispute, too, with most Democrats wrongly insisting that they’re insufficient.) I refer to things like a narrowing curriculum that sacrifices history, art, and literature on the altar of reading and math skills…”

Diane Ravtich has similarly stepped on the high-stakes brakes, expressing concern about the crowding out of other academic subjects and activities: “a new organization called Common Core was launched on February 26 at a press conference in Washington, D.C., to advocate on behalf of the subjects that are neglected by the federal No Child Left Behind legislation and by pending STEM legislation. These subjects include history, literature, the sciences, the arts, geography, civics, even recess (although recess is not a subject, it is a necessary break in the school day that seems to be shrinking or disappearing in some districts). I serve as co-chair of CC with Toni Cortese, executive vice-president of the American Federation of Teachers.”

To find out whether these concerns are supported by the empirical evidence from Florida, tune into the Manhattan Institute web site tomorrow to see the study.

Leave a Comment » | school accountability | Tagged: , , , , , , , , , | Permalink
Posted by Jay P. Greene

Next Entries »