When “Sorry” Means “J’Accuse”

July 28, 2008

The letter "J'accuse"

The following column about a letter appeared on the front page of the Cleveland Plain Dealer on Wednesday, July 23.  The letter is in the form of an apology, but it is actually a series of accusations about testing and accountability.  Like another front page letter of accusation, this one has all of Emile Zola’s moral outrage but has none of Zola’s justification.

I’ve reprinted it here with my comments in blue italics.

Students pass state test, but at what cost to their education?

by Regina Brett

The school report cards came out in June.

Rocky River Middle School passed the 2008 Ohio Achievement Tests, earned an Excellent rating from the state and met the requirements for Annual Yearly Progress.

For all of those accomplishments, Principal David Root has only one thing to say to the students, staff and citizens of Rocky River:

He’s sorry.

Root wants to issue an apology. He sent it to me typed out in two pages, single spaced.

He’s sorry that he spent thousands of tax dollars on test materials, practice tests, postage and costs for test administration.

Actually, he did not spend the money.  The taxpayers did when they decided through their elected representatives to adopt a testing and accountability system.  They then hired David Root to implement this policy in his capacity as principal at a public school.

Sorry that his teachers spent less time teaching American history because most of the social studies test questions are about foreign countries.

I guess the people of Ohio thought it was important for students to learn about foreign countries when they, through their elected representatives and hired agents, devised the state curriculum and test.  Besides, if students learned more about foreign countries they might know who Emile Zola was.

Sorry that he didn’t suspend a student for assaulting another because that student would have missed valuable test days.

Sounds pretty irresponsible.  Would he have made a different decision if the student would have missed valuable instructional days?  If so, whose fault is that?  Oh yes, I forgot that this is an accusation, not an apology.

Sorry he didn’t strictly enforce attendance because all absences count against the school on the State Report Card.

So, is David Root saying that he cheated on the state accountability system?  Isn’t this like lying to your boss about your job performance?  Will he be fired, sanctioned, or resign to make amends for his infraction?

He’s sorry for pulling children away from art, music and gym, classes they love, so they could take test-taking strategies.

Why didn’t he just follow the state curriculum and let the scores show what students knew? The decision to take time away for “test-taking strategies”  was completely unnecessary given that more than 90% of Rocky River students have been scoring above the proficient level in reading, math, and writing.  It sounds like they would have done just fine on the state test without working on test-taking strategies and having spent more time on art, music, and gym.

Sorry that he has to give a test where he can’t clarify any questions, make any comments to help in understanding or share the results so students can actually learn from their mistakes.

How reliable would the results be if principals could clarify questions, help in understanding, or share secure test items that would be re-used on future tests?  Does every assessment have to be a formative assessment?

Sorry that he kept students in school who became sick during the test because if they couldn’t finish the test due to illness, the student automatically fails it.

This sounds like a difficult decision.  Football coaches similarly have to think about whether to take injured players out of the game versus having the players tough it out.  We pay leaders to make these difficult decisions, balancing competing interests wisely.

Sorry that the integrity of his teachers is publicly tied to one test.

Actually, the state accountability system — let alone “the integrity of his teachers” —  is not based on one test.  The overall rating of Rocky River Middle School is based on several test results (in Reading, Math, Writing, Social Studies, and Science), the progress students have made in those subjects, and (as we already heard about) the possibly fraudulent attendance rate.

He apologized for losing eight days of instruction due to testing activities.

I thought Root didn’t want one test, so it takes time to administer several.  While testing takes place on eight days it does not (or at least does not have to) consume the entirety of those days.  My understanding is that the average student only spent two mornings being tested, as testing occurred in different grades and subjects for different students across eight days.

For making decisions on assemblies, field trips and musical performances based on how that time away from reading, math, social studies and writing will impact state test results.

I would hope that the principal would think about how assemblies, field trips, and musical performances impact instructional time for other academic subjects regardless of whether those subjects are part of a state accountability system.

For arranging for some students to be labeled “at risk” in front of their peers and put in small groups so the school would have a better chance of passing tests.

Again, if smaller group instruction would help certain students, the principal should arrange for that regardless of the state accountability system.  And the principal would have to think of a way to provide that necessary assistance without stigmatizing the students who need it.

For making his focus as a principal no longer helping his staff teach students but helping them teach test indicators.

Why didn’t he just help his staff teach the subjects with confidence that the test indicators would show what they had learned?  This is especially puzzling given how likely it is that students at Rocky River would pass the state test without paying any special attention to test-taking strategies.

Root isn’t anti-tests. He’s all for tests that measure progress and help set teaching goals. But in his eyes, state achievement tests are designed for the media to show how schools rank against each other.

Seems like the state accountability system does measure progress and help set teaching goals.  What’s wrong with it also informing the public and policymakers (via the media) about how their schools are doing?

He’s been a principal for 24 years, half of them at Rocky River Middle School, the rest in Hudson, Alliance and Zanesville. He loves working with 6th, 7th and 8th graders.

“I have a strong compassion for the puberty stricken,” he joked.

His students, who are 11, 12, 13 and 14, worry that teachers they love will be let go based on how well they perform.

One asked him, “If I don’t do well, will you fire my teacher?”

He cringed when he heard one say, “I really want to do well, but I’m not that smart.”

Has a single tenured teacher in Ohio (or in the United States) been let go based on performance on state accountability tests?  Maybe he should reassure the students that their concern is misplaced.

He wants students to learn how to think, not take tests.

Can’t they do both?

“We don’t teach kids anymore,” he said. “We teach test-taking skills. We all teach to the test. I long for the days when we used to teach kids.”

Why not just return to those days and let the test results show what kids have learned?

Unless we get back to those days, principals and teachers all over Ohio will continue to spend your tax dollars to help students become the best test takers they can be.

The people of Ohio decided to adopt an accountability system because the schools weren’t doing an adequate job teaching kids to think without it.  The “just trust us to do a good job” approach wasn’t working.

(edited to add color)

7 Comments | school accountability | Tagged: , , , , , , , | Permalink
Posted by Jay P. Greene

Savage Comments

July 22, 2008

The blow-hard radio host Michael Savage has never been known for his careful and thoughtful analysis, but he recently went a big-mouth mile too far when he declared that autism was “a fraud, a racket … In 99 percent of the cases, it’s a brat who hasn’t been told to cut the act out. … They don’t have a father around to tell them, ‘Don’t act like a moron. … Act like a man. Don’t sit there crying and screaming, idiot.’ “

2 Comments | special ed | Tagged: , , , | Permalink
Posted by Jay P. Greene

Camp Education

July 21, 2008

“I desire macaroni pictures! And those little shaker things where you put beans inside of paper plates that are glued together! And let us put patterns of glue on the outside of those paper plates so we can then pour glitter on them so they look nice and sparkly!”

As I drop the kids off at sleep-away summer camp, I’ve been thinking about whether school should be more like camp.  At camp the kids learn an enormous amount, including a large amount of traditional academic content.  Two of my children are at a Jewish camp where they learn Hebrew and Judaics in addition to more typical camp activities.  (And no, there is no giant Moses in the shape of the CPU from Tron demanding macaroni pictures).  My oldest goes to a special needs camp that offers an emphasis on independent living skills (just like school) in addition to the usual camp stuff.

They all learn a lot.  But unlike school, the kids love it.  Don’t get me wrong, they like school quite a bit — but they love camp.  They love it even though they are made to do all sorts of challenging or sometimes unpleasant things that they rarely do at home.  They have to do all of the cleaning, they serve and clear all of the meals, and they fold their own clothes.  It can be broiling during the day and freezing at night.  They help tend farm animals.  They climb to the top of a high tower.  They go for long hikes.

The camps my kids go to have very nice facilities and are considered expensive.  Their camps offer activities not usually found at other summer camps, including go-carts, mountain biking, computers, water trampolines, and tennis.  The ratio of counselors to campers at the Jewish camp is less than 5 to 1, and at the special needs camp is about 2 to 1 (including specialists).

How are these camps able to teach kids a lot, get them to work hard, and get the kids to love it, while schools struggle to do any of these things?

What’s more, even these expensive camps are less expensive than the average public school.  The Jewish camp costs $151.92 per day, which given that they are cared for 24 hours per day, comes out to $6.33 per hour.  The average public school, as of 2006-7, cost $10,725 per pupil for 180 days, which works out to $59.98 per day or $8.51 per hour for the 7 hours they are in school.  Even the special needs camp, which seems quite expensive, costs less than the average special education in public schools.  The hourly cost of the special needs camp is $11.02 compared to $16.17 for special education at the average public school.  I also looked up the tuition of a popular Christian camp in the area.  The charge there is only $3.33 per hour.

How do sleep-away camps get kids to work hard, learn a lot, broaden their experiences and love it — all for less than the cost of public schooling?  A big difference is that most of the counselors are young, college kids.  They don’t get paid very much but tend to be enthusiastic, bright, and energetic.  Some will later be doctors or lawyers, but they are happy to be counselors for a few summers in the meantime.  It’s easier to get talented people for low pay for a short time than for an entire career.  Camps always have some wise old-hands to keep the young staff in check and to maintain the norms and mission of the organization, but camps mostly succeed at low cost because of their energetic young counselors.

Could schools be more like camps?  Could we hire a lot of enthusiastic, bright, and energetic teachers fresh out of college, who know full well that most of them will leave in a few years to become lawyers, doctors, or something else?  A few old-hands would stick around to keep the young staff in check and to maintain the norms and missions of the organization.  But schools could potentially attract more talented people as teachers at lower cost if they followed the camp model.  And perhaps schools with a high-turnover, young staff would better connect with students and convey the love of learning and working hard.

I know that current research finds that teachers tend to be less effective in their first few years and that turnover is harmful.  But those are findings about new teachers and high turnover under the current system that rewards teachers for sticking around for 20-25 years.  We can’t simply extrapolate from that to what would happen under a system that attracted a different crop of new teachers and where turnover was effectively encouraged (reform of the pension system and pay scale could move us in that direction).

Maybe the intensity of camp just couldn’t be sustained for an entire school year.  Maybe adding even a little more academic content would ruin the camp magic.  I’m sure many things would go wrong if we tried to make schools more like camps, but I think it’s worth thinking about what we can learn from camps to make schools more effective.

17 Comments | teacher pay, teacher quality | Tagged: , , , | Permalink
Posted by Jay P. Greene

Bigger is Not Better in Education

July 20, 2008

I have a piece in this morning’s Arkansas Democrat Gazette arguing that consolidating school districts in Arkansas to 75 countywide school districts is not a promising reform strategy.  A number of state officials as well as the Dem Gaz have floated the idea of cutting the number of districts to less than one-third of the current number as a way of saving superintendent and football coach salaries while improving the capacity of high schools to offer state-required courses.  I argue that the salary savings will be few, there are better ways to help high schools offer courses (such as with distance ed), and student achievement tends to suffer in larger schools and school districts. 

Now, this doesn’t mean that reconfiguring larger urban high schools into “small” schools within a school, as the Gates Foundation once pushed, is likely to produce much of an improvement either.  The benefits of smaller schools and school districts may be related to the tighter connection they have to their communities and the more competitive market provided by having more districts.  Simply breaking up big high schools may not better connect schools to communities or create more competitive pressure. 

Being able to choose among schools within a district is like being able to choose among the menu items at McDonalds.  It’s nice that you could choose the Filet-O-Fish if you prefer to eat fish, but there is no change in competitive pressure from adding that menu item — all of the money still ends up in the same place.  The same is true for choice within school districts — all of the money still stays with the school district, so their motivation is not significantly altered by your choice among their schools.  We should only expect significant competitive pressure when money leaves one organization and enters another as a result of consumer choice.  School districts are the main organizational unit of education funding.

3 Comments | competitive effects | Tagged: , , , , | Permalink
Posted by Jay P. Greene

Would You Pay $43,479 for a 1971 Impala?

July 19, 2008

Andrew Coulson at Cato does a great job of illustrating how disastrous it is to have had stagnant achievement outcomes for 17 year-old public school students since 1970, while per pupil spending has increased by a factor of 2.3 (adjusted for inflation).  He likens it to paying $43,479 for a 1971 Chevy Impala, which is 2.3 times the $19,011 inflation-adjusted price back then ($3,460 before adjusting for inflation).  Meanwhile, a brand new 2008 Impala sells for $21,975 and comes with features like On-Star, side air bags, and anti-lock brakes that weren’t even imagined in 1971. 

In the automotive industry cars keep getting better with little increase in cost (after inflation), while education has not improved significantly and costs us 2.3 times as much (after inflation).  It isn’t every day that people wish that an industry would be as efficient as car-makers.

4 Comments | competitive effects | Tagged: , , | Permalink
Posted by Jay P. Greene

Being Misquoted

July 17, 2008

Dean Millot has a new post attacking me on the peer review issue that Eduwonkette promotes on her own site.

But Dean Millot is being fundamentally dishonest in that he misquotes me. He says that I argue: “In short, I see no problem with research becoming public with little or no review.”

In fact I wrote: “In short, I see no problem with research initially becoming public with little or no review.” (See here )

The absence of the word “initially” makes quite a difference and sets up the straw man that Millot wishes to knock down. The issue is not whether research can benefit from peer review, but whether it is inappropriate to make it publicly available INITIALLY, before it has received peer review.

Readers may want to wonder about the credibility of Millot’s claim that “One of the reasons I do my best to quote the very words of people I write about in edbizbuzz is that I prefer to fight fair.”

And so much for Eduwonkette’s praise of Millot’s “measured, careful, and thoughtful analysis.”

I’m waiting for the correction and apology from both of them.

3 Comments | journalism, research reports | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

It Never Ends

July 14, 2008

I thought that the exchange with Eduwonkette over the appropriateness of releasing research without peer review had run its course with my last post.  But it seems that it will never end.  Here is her latest post and here is the reply that I posted in her comment section:

Eduwonkette is attempting to change the subject. I’ve never disputed that peer review can help provide additional assurances to readers about quality.  The issue is whether research ought to be available to the public even if it has not been peer reviewed.  In attacking the release of my most recent study Eduwonkette seems to be arguing that it is inappropriate to release research without peer review, at least under certain conditions that she only applies to research whose findings she does not like.  If she were going to be consistent, she would have to criticize anyone who releases working papers of their research, which would be almost everyone doing serious research.

 

What’s more, she is still trapped in a contradiction: she can’t say that we should analyze the motives of people who release research directly to the public when assessing whether it is appropriate, while she prevents analysis of her own motives because she blogs anonymously.  As I have now said several times, either she drops the suggestion that we analyze motives or she drops her role as an anonymous blogger.  If she refuses to resolve this contradiction, Ed Week should stop lending her their reputation by hosting her blog.  Let her be inconsistent in blogging at the expense of her own anonymous persona and not drain the respectability of Ed Week.

 

Lastly, the comparison of the market for education policy information and the market for cars comes from my most recent post in our exchange, but she oddly does not credit me here. (See https://jaypgreene.com/2008/07/12/see-were-in-italy/ )  Her position seems to be that we ought to forbid (or at least shun) the sale of used cars without warranties (translation: research without peer review).  My argument is that used cars without warranties come at a risk but there are compensating benefits.  Similarly, non-peer-reviewed research has its risks but also its benefits.

 

UPDATE — My exchange with Eduwonkette continues although it seems increasingly pointless.  Here is my (slightly edited) last comment on her site:

“Let’s make this very concrete. Was it inappropriate for Marcus Winters and I to release our social promotion findings in 2004 without peer review, or should we have waited until it had been peer-reviewed and published (in various forms) in 2006, 2007, and again in 2008? If the appropriate thing is to wait, would interest groups, editorial boards, and bloggers similarly hold their tongues until the additional evidence came in?  Would policymakers hold off on decisions that might have come out differently if they had the suppressed information?

Would it have been OK to release in 2004 as long as we tried to make it obscure enough so that people were less likely to find it? What if interest groups, bloggers, etc… found our obscure finding and promoted them (as has happened with Jesse Rothstein’s paper)?

And in saying ‘working papers and thinktank reports are released for entirely different functions’ you are repeating your call for an analysis of motives. You’ve said that think tanks want to influence policy (bad motive) while academics are trying to advance knowledge with each other (good motive). But if academics are serving the public good, shouldn’t they ultimately want to influence policy? I am an academic who also releases working papers through a think tank. Does that make my motives good or bad? I think all of this analysis of motives is silly when the real issue is the truth of claims, not why people are making those claims. Calling for an analysis of motives is especially silly for someone who is trying to influence people anonymously. The fact that you are trying to influence people through a blog does not give you a free pass from having to be consistent on this.”

Leave a Comment » | journalism, research reports | Tagged: , , , | Permalink
Posted by Jay P. Greene

Blog Rankings

July 14, 2008

This blog is not yet three months old but I am pleased to report that it is off to a good start.  According to Technorati’s rankings, JayPGreene.com is attracting more readers than the American Federation of Teachers’ blog, Edwize, more than Diane Ravitch and Deborah Meier’s, Bridging Differences hosted by Education Week, more than the Reason Foundation’s Out of Control, and the Center for Education Reform’s Edspresso.  It significantly trails the educouple of Eduwonk and Eduwonkette as well as Cato at Liberty (although that’s not primarily an education blog).  Flypaper, which started about the same time as this blog, is also off to a good start.  The Queen of education blogs seems to be Joanne Jacobs.

Here are the Technorati rankings (as of this morning) of education sites that seem to share some of the same audience as this blog.  By no means is this a comprehensive list of education blogs.  And I have no idea how reliable or meaningful Technorati’s rankings really are.  I’d continue blogging no matter what the rankings were because it’s fun.  I imagine the same is true of most others.

  1. Cato at Liberty               3,662
  2. Joanne Jacobs                3,709
  3. Eduwonkette                27,419
  4. Eduwonk                      30,876
  5. Flypaper                       95,943
  6. Jay P. Greene               104,227
  7. Bridging Differences   107,924
  8. D-Ed Reckoning         107,924
  9. AFT’s Edwize              116,227
  10. Edspresso                  123,039
  11. Out of Control            123,039
  12. Core Knowledge         127,851
  13. Sherman Dorn            151,703
  14. EdBizBuzz                   184,730

8 Comments | blogging | Tagged: , , , , , , , , , , , | Permalink
Posted by Jay P. Greene

AJC Op-Ed

July 13, 2008

I have an op-ed in today’s Atlanta Journal Constitution on the social promotion issue in Georgia.

2 Comments | social promotion | Permalink
Posted by Jay P. Greene

See, we’re in Italy…

July 12, 2008

Stripes

“See, we’re in Italy.  The guy on the top bunk has gotta make the guy on the bottom’s bed all the time.  It’s in the regulations.  If we were in Germany I would have to make yours.  But we’re in Italy, so you’ve
gotta make mine. It’s regulations.”

This is more or less Eduwonkette’s response to my complaint that she can’t argue that the source of information is important in assessing the truth of claims while blogging anonymously.  Her answer is that it’s different for bloggers (in Italy) than for researchers (in Germany).  It’s regulations.

She goes on to describe some differences between different types of information in education policy debates, but it’s not clear why any of those differences would be relevant to whether assessing the source is important for one and not for another.  The closest she comes to explaining why things are meaningfully different is when she says, “And let’s be realistic: an anonymous blogger isn’t shaping public policy.”  So, if information will have no bearing on policy debates, then its source is unimportant.

This would be a consistent argument if she really believed that bloggers had NO influence.  But of course they have at least some influence.  Why else would she and the rest of us be bothering with this?  And if bloggers have some influence, then the same basic principles should apply: either we should analyze the motives of sources of information to assess the truth of claims or we shouldn’t.  I’m in favor of not analyzing motives for anyone since I think that the truth of claims is independent of the motives of the source.  Even bad people can make true arguments.

At the risk of belaboring this issue, maybe I can clarify things by describing the market of ideas in policy debates as being like the market for cars.  We have different levels of confidence in cars that have gone through different processes before being made available for sale.  We could buy a used car from the corner used car dealer with no warranty.  That would be like reading blogs.  We don’t really know whether we are getting a lemon or not, since almost no assurances have been made about quality.  Or we could buy a used car from a larger chain with at least some warranty.  That would be like getting information from newspapers or magazines.  There has been some review and assurance of quality, but we still don’t quite know what we’ll get.  Or we could buy a new car from a major dealer and buy the extended warranty.  That would be like getting information from a peer-reviewed journal.  It may still be a lemon, but we’ve received a lot of assurances that it is not.  And I suppose reading an anonymous blogger is like buying a used car from someone you don’t know in the want ads.  There are trade-offs in getting cars with these different level of assurances about quality, just as there are trade-offs in getting information that has gone through different processes to assure quality. 

Eduwonkette’s argument is essentially that the same rules regarding these trade-offs don’t apply to the market for cars without warranties that do apply to the market for cars with warranties.  My view is that there are only differences in degree, not kind.  Even bad people can sell cars that are good values.

I’ve also noticed that Marc Dean Millot has weighed in on this issue.  He’s just knocking down a straw man.  It is not my position that research doesn’t benefit from peer review.  He can check out my cv to see that I have two dozen peer-reviewed publications, many of which were earlier released directly to the public without review.

I’ve been arguing that the public benefits from seeing research even before it has received peer review because it gets more information faster.  Without the assurances of peer review people will tend to have lower confidence in that research, and their confidence may increase as the research receives those additional assurances.  Millot seems to want to embargo information from the public until it receives peer review.  If he really believes that, then he should criticize every researcher with working papers on the web.  That’s almost everyone doing serious research.

And on his points about ideology tainting research I would suggest that people read Greg Forster’s excellent earlier post on Vouchers: Evidence and Ideology.

1 Comment | journalism, research reports | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

« Previous Entries
Next Entries »