Violating the Denominator Law

December 2, 2008

Sean Corcoran, who is guest blogging for the blogger formerly known as Eduwonkette, may have to go to education research jail because he violated the Denominator Law today.  For those of you unfamiliar with the Denominator Law from my previous post on the topic (and ignorance of the law is no excuse) it is: “No one should be allowed to highlight numerators without also presenting denominators.  That is, it is often misleading to describe a big number without putting that number in perspective.”

So, Sean is all worried about private donations to public schools creating or exacerbating inequities in funding.  He references a report about California (and it had better be peer-reviewed or the blogger formerly known as Eduwonkette will throw a fit) that finds: “contributions to California school foundations rose from $123 million in 1992 to $238 million in 2001.”  He does helpfully add that $238 million only amounts to $40 per pupil.  But he doesn’t fully comply with the Denominator Law because he fails to point out that $238 million only represents .4% of the $52.2 billion in total public school revenue in California in 2001.

It’s not the average amount of private giving in California that really worries him.  What concerns him is that these donations are concentrated in wealthy areas: “Of course—as Brunner and Imazeki point out—these contributions are far from evenly distributed. Donations are strongly related to family income, and in some cases they are quite high, at more than $250 to $500 per student. (You can read about the $3.3 million education foundation in Santa Monica-Malibu Unified School District here).”

Here, your honor, is where he flagrantly breaks the Denominator Law.  He suggests that $250 to $300 per pupil, as illustrated by $3.3 million in private giving to public schools in Santa Monica, is “quite high.”  Without a denominator, it’s hard to judge how high $3.3 million in Santa Monica really is. 

Let me help.  According to the School Matters web site operated by Standard and Poor’s, Santa Monica has 12,191 students.  The private contributions Corcoran mentions amount to $271 per pupil — within his $250 to $300 range.  But total revenue for Santa Monica public schools amounted to $11,062 per pupil as of 2006.  Private contributions of $271 amount to only 2.4% of total revenue — not exactly “quite high.”

And this private giving hardly accounts for resource differences between Santa Monica and the average district in California.  According to School Matters the average district in CA had total revenue of $9,553 as of 2006, $1,509 less than in Santa Monica.  If Santa Monica received $271 in private donations compared to $40 for the average California district, the extra $231 could only account for about 15% of the extra resources Santa Monica possesses. 

If this is the worst case that folks can muster, it hardly seems like private giving is a significant contributor to resource inequities.  We only gain this appropriate perspective when we comply with the Denominator Law — so be sure to follow the law out there.

Bloggers Shouldn’t Have Rapper Names

October 8, 2008

In my last post I described Jennifer Jennings as the blogger formerly known as Eduwonkette.  I had thought we only had to call her Eduwonkette when we didn’t know who she was.  But I guess she continues to go by her rapper name, Eduwonkette.  And Aaron Pallas, an otherwise respectable scholar, continues to call himself Skoolboy — with a k!  And I guess they are both cribbing (in the non-rapper meaning) from Eduwonk, who we’ve always known to be Andy Rotherham.

I find the use of rapper names by bloggers to be downright silly.  It’s especially silly when accompanied by self-aggrandizing cartoons and graphics.  Here at Jay P. Greene’s Blog we’ve gone for a minimalist approach, both out of laziness and an aesthetic vision that tried to put the focus on content.

But if a bunch of other folks are going to continue to call themselves rapper names and have cartoon graphics to represent themselves, maybe I should do the same.  Perhaps I should go by my rapper name — DJ Super-Awesome.  And maybe we should use Thundarr the Barbarian graphics to represent ourselves.  I call the image of Ookla and I’ll let Greg and Matt fight over who gets to be Ariel.

Policymaking By Anecdote

October 7, 2008

Is it good policy to reduce barriers to firing sub-par teachers?

According to Jennifer Jennings, the blogger formerly known as Eduwonkette, the answer is no.  We need to preserve teacher tenure, she argues, because she has found an example of a really great teacher, Art Siebens, who was fired when his DC school was reconstituted.  His case is “haunting for the glimpse it offers into the brave new world of unchecked principal autonomy.”

Well, Siebens wasn’t actually fired.  He wasn’t re-hired at the same school and was instead offered a job teaching a different science course at a different DC public school.  Don’t fret ye of weak hearts — continued employment for teachers is still essentially guaranteed even if not in the school and class of their choosing.

It’s puzzling why Siebens wasn’t re-hired given that he was an award winning teacher with what appears to be a strong record of excellent work.  But the fact that he wasn’t is hardly evidence against DC superintendent Michelle Rhee’s proposal to offer teachers significant pay increases if they give up tenure.  Perhaps there is more to Siebens’ story than is publicly known.  

More importantly, the case of Art Siebens is not evidence against abandoning tenure because it is a single case.  The plural of anecdote is not data.  We shouldn’t make policy by referencing anecdotes.  Instead, we should look “through the lens of social science,” as a wise person once wrote, and consider systematic evidence when formulating education policy. 

The remarkable investigative reporting by Scott Reeder has powerfully documented the problems with teacher tenure.  After filing 1,500 Illinois Freedom of Information Act requests with the state board and all 876 Illinois school districts, Reeder uncovered the following:

1) “Of an estimated 95,500 tenured educators now employed in the state [of Illinois] an average of only seven have their dismissals approved each year by a state hearing officer. Of those seven, only two on average are fired for poor job performance. The remainder is dismissed for issues of misconduct.”

2) “Of Illinois’ 876 school districts only 61, or 7 percent, have ever attempted to fire a tenured faculty member since the teacher evaluation reforms were imposed 18 years ago.”

3) “Of those 61 school districts, only 38 were successful in actually firing a teacher.”

4) “Not only is it exceedingly rare to fire a tenured teacher in Illinois, but it also is extraordinarily expensive. In fact, Illinois school districts that have hired outside lawyers in these cases have spent an average of more than $219,000 in legal fees during the last five years.”

5) “In the last 10 years, about 477,000 evaluations of Illinois tenured teachers have been performed, but only 513 received unsatisfactory evaluations… In other words, only 1 out 930 evaluations result in a tenured teacher receiving an ‘unsatisfactory’ rating.” Conducting those evaluations consumed 2.5 million administrative hours.

OK, so it is next to impossible to fire tenured teachers.  And we also know from systematic evidence that the quality of the teacher is the single most important factor within school control to influence student academic improvement. (See for example the research referenced here.)

Unless we believed that all but .007% of tenured teachers are doing a solid job, the current system is clearly keeping incompetent teachers in the classroom.  Any meaningful reform strategy has to involve getting rid of dud teachers and attracting better teachers as replacements.  Rhee’s proposal to increase pay in exchange for greater flexibility in terminating sub-par teachers seems like a promising idea to do just that.  The higher pay might attract better new people into teaching and the flexibility on termination could remove bad teachers from the classroom. 

Of course, the blogger formerly known as Eduwonkette makes a fair point when she asks, “Why are you confident that principals will always – or even often – pick the ‘best teachers?'”  For a system with reduced tenure protections to work, principals would also have to be properly motivated to distinguish between effective and ineffective teachers.  But this could be done either through meaningful merit evaluation and rewards for principals or through market accountability in choice programs.  If continued employment or pay raises for principals depended upon identifying effective and ineffective teachers, they are unlikely to let talented teachers like Art Siebens go and are likely to get rid of duds.

But we should all accept that any system of hiring and firing teachers will have its injustices.  The status quo tenure system has the injustice of protecting bad teachers in their jobs.  And if cuts have to be made it is newer teachers who have to be let go, even if they are better teachers than their senior colleagues.  A system like Rhee proposes will occasionally mistakenly let go of a good teacher.  But with well-designed incentives for the principals this should be the exception and not the rule.

Besides, we have to ask ourselves:  how many kids do you want to condemn to an ineffective teacher to avoid the possibility of unjustly terminating a good teacher?  If we care more about the kids than the adults in schools, then the injustice to the students should matter much more to us than the possible injustice to a few teachers.

Eduwonkette or Jennifer Jennings, Makes No Difference

August 26, 2008

I’m glad that Eduwonkette decided to end her anonymity and identify herself as Jennifer Jennings, a sociology graduate student at Columbia University.  I’m not glad because I think it was inherently wrong for her to blog anonymously.  As I’ve previously written: “The issue is not who Eduonkette is, but whether she is right or not.  Knowing who she is does not make her evidence or arguments any more or less compelling.” 

The problem was that Eduwonkette did not share my belief in the principle that we should focus on the veracity rather than the source of claims.  She repeatedly emphasized the credibility of the source of information.  Emphasizing the credibility of sources while blogging anonymously, preventing analysis of her own credibility, was logically untenable and had to end.  I wished that she would end this inconsistency by embracing the view that ideas are true or false independent of their sources.  Instead she has resolved her inconsistency by ending her anonymity.

Now that we know that Eduwonkette is Jennifer Jennings we can see another prominent example of the logical inconsistency of blogging anonymously while focusing on the credibility of sources.  In 2005 Jennings published an article in the American Educational Research Journal that argued that accountability systems encouraged schools to focus on the achievement of “bubble” students — those close to an achievement cutoff — at the expense of high and low achieving students.  She arrived at this conclusion after visiting a school in Texas and observing it for a period of time.  She was aided in drawing this conclusion by using jargon like “neoinstituionalism” and “normative isomorphism,” but I kind of zoned out during that part of the article.  I’m guessing that neoinstitutionalism is bad following my theory that anything starting with “neo” is supposed to be bad while anything starting with “post” is supposed to be good.

A few years later Matthew Springer published articles in Education Next and in the Economics of Education Review that empirically examine Jennings’ claim of “educational triage.”  The Education Next piece actually began with a lengthy quote from Jennings’ article as a foil for its findings that NCLB accountability improved the achievement of “bubble” students, but not at the expense of lower and higher achieving students.

Jennings then took-on her critic, Springer, but she did so as the anonymous blogger Eduwonkette, never revealing that she was attacking the person who criticized her own research.  And her first argument against Springer was that his work was published in Education Next and “Education Next is not a scholarly journal.”  Jennings targets the source of the critique of her own work while concealing her identity to prevent analysis of her as a source.  The irony is too rich.

Why Jennings did not just focus on the version of the paper published in Economics of Education Review or the unabridged version linked to on Education Next, , which would have been free of the unscholarly taint she perceives in Education Next,  is unclear.  It was obviously important for her to discredit Springer’s argument against her own study by attacking the credibility of Education Next as the source of Springer’s argument — all the while preventing assessment of her credibility by doing all of this anonymously.

Let me be clear that I have no problem with Jennings defending her own work anonymously.  Her arguments against Springer are true or false regardless of who she says or doesn’t say she is.  My point is that by arguing her own case anonymously, Jennings betrays the principles that she appears to endorse.  Namely, if the source of information is important in assessing claims, it would clearly be inappropriate to attack your critic without revealing who you are. 

Even now that Jennings has revealed her identity, I hope that she abandons her reliance on assessing the source of claims.  Doing so would justify her past actions and help us move forward in analyzing ideas rather than analyzing people and motives.

Voucher Effects on Participants

August 21, 2008

(This is an update of a post I originally wrote on August 21.  I’ve included the new DC voucher findings.)

Here is what I believe is a complete (no cherry-picking) list of analyses taking advantage of random-assignment experiments of the effect of vouchers on participants.  As I’ve previously written, 9 of the 10 analyses show significant, positive effects for at least some subgroups of students.

All of them have been published in peer reviewed journals or were subject to outside peer review by the federal government.

Four of the 10 studies are independent replications of earlier analyses.  Cowen replicates Greene, 2001.  Rouse replicates Greene, Peterson, and Du.  Barnard, et al replicate Peterson and Howell.  And Krueger and Zhu also replicate Peterson and Howell.  All of these independent replications (except for Krueger and Zhu) confirm the basic findings of the original analyses by also finding positive effects.

Anyone interested in a more complete discussion of these 10 analyses and why it is important to focus on the random-assignment studies, should read Patrick Wolf’s article in the BYU Law Review that has been reproduced here.

I’m eager to hear how Leo Casey and Eduwonkette, who’ve accused me of cherry-picking the evidence, respond.

  • These 6 studies conclude that all groups of student participants experienced reading or math achievement gains and/or increased likelihood of graduating from high school as a result of vouchers:

Cowen, Joshua M.  2008. “School Choice as a Latent Variable: Estimating the ‘Complier Average Causal Effect’ of Vouchers in Charlotte.” Policy Studies Journal 36 (2).

Greene, Jay P. 2001. “Vouchers in Charlotte,” Education Matters 1 (2):55-60.

Greene, Jay P., Paul E. Peterson, and Jiangtao Du. 1999. “Effectiveness of School Choice: The Milwaukee Experiment.” Education and Urban Society, 31, January, pp. 190-213.

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E. Peterson. 2002. “School Vouchers and Academic Performance:  Results from Three Randomized Field Trials.” Journal of Policy Analysis and Management, 21, April, pp. 191-217. (Washington, DC: Gains for all participants, almost all were African Americans)

Rouse, Cecilia E. 1998. “Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program,” The Quarterly Journal of Economics, 113(2): 553-602.

Wolf, Patrick, Babette Gutmann, Michael Puma, Brian Kisida, Lou Rizzo, Nada Eissa, and Marsha Silverberg. March 2009.  Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years. U.S. Department of Education, Institute of Education Sciences. Washington, DC: U.S. Government Printing Office. (In the fourth year report the sample size shrunk so that the positive achievement effect barely missed meeting a strict threshold for statistical significance — p < .06 just missing the bar of p < .05.  But this new report was able for the first time to measure the effect of vouchers on the likelihood that students would graduate high school.  As it turns out, vouchers significantly boosted high school graduation rates.  As Paul Peterson points out, this suggests that vouchers boosted both achievement and graduation rates in the 4th year.  Read the 4th year evaluation here.)

  • These 3 studies conclude that at least one important sub-group of student participants experienced achievement gains from the voucher and no subgroup of students was harmed:

Barnard, John, Constantine E. Frangakis, Jennifer L. Hill, and Donald B. Rubin. 2003. “Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City,” Journal of the American Statistical Association 98 (462):299–323. (Gains for African Americans)

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E. Peterson. 2002. “School Vouchers and Academic Performance:  Results from Three Randomized Field Trials.” Journal of Policy Analysis and Management, 21, April, pp. 191-217. (Dayton, Ohio: Gains for African Americans)

Peterson, Paul E., and William G. Howell. 2004. “Efficiency, Bias, and Classification Schemes: A Response to Alan B. Krueger and Pei Zhu.” American Behavioral Scientist, 47(5): 699-717.  (New York City: Gains for African Americans)

This 1 study concludes that no sub-group of student participants experienced achievement gains from the voucher:

Krueger, Alan B., and Pei Zhu. 2004. “Another Look at the New York City School Voucher Experiment,” The American Behavioral Scientist 47 (5):658–698.

(Update: For a review of systemic effect research — how expanded competition affects achievement in traditional public schools — see here.)

Being Misquoted

July 17, 2008

Dean Millot has a new post attacking me on the peer review issue that Eduwonkette promotes on her own site.

But Dean Millot is being fundamentally dishonest in that he misquotes me. He says that I argue: “In short, I see no problem with research becoming public with little or no review.”

In fact I wrote: “In short, I see no problem with research initially becoming public with little or no review.” (See here )

The absence of the word “initially” makes quite a difference and sets up the straw man that Millot wishes to knock down. The issue is not whether research can benefit from peer review, but whether it is inappropriate to make it publicly available INITIALLY, before it has received peer review.

Readers may want to wonder about the credibility of Millot’s claim that “One of the reasons I do my best to quote the very words of people I write about in edbizbuzz is that I prefer to fight fair.”

And so much for Eduwonkette’s praise of Millot’s “measured, careful, and thoughtful analysis.”

I’m waiting for the correction and apology from both of them.

It Never Ends

July 14, 2008

I thought that the exchange with Eduwonkette over the appropriateness of releasing research without peer review had run its course with my last post.  But it seems that it will never end.  Here is her latest post and here is the reply that I posted in her comment section:

Eduwonkette is attempting to change the subject. I’ve never disputed that peer review can help provide additional assurances to readers about quality.  The issue is whether research ought to be available to the public even if it has not been peer reviewed.  In attacking the release of my most recent study Eduwonkette seems to be arguing that it is inappropriate to release research without peer review, at least under certain conditions that she only applies to research whose findings she does not like.  If she were going to be consistent, she would have to criticize anyone who releases working papers of their research, which would be almost everyone doing serious research.


What’s more, she is still trapped in a contradiction: she can’t say that we should analyze the motives of people who release research directly to the public when assessing whether it is appropriate, while she prevents analysis of her own motives because she blogs anonymously.  As I have now said several times, either she drops the suggestion that we analyze motives or she drops her role as an anonymous blogger.  If she refuses to resolve this contradiction, Ed Week should stop lending her their reputation by hosting her blog.  Let her be inconsistent in blogging at the expense of her own anonymous persona and not drain the respectability of Ed Week.


Lastly, the comparison of the market for education policy information and the market for cars comes from my most recent post in our exchange, but she oddly does not credit me here. (See )  Her position seems to be that we ought to forbid (or at least shun) the sale of used cars without warranties (translation: research without peer review).  My argument is that used cars without warranties come at a risk but there are compensating benefits.  Similarly, non-peer-reviewed research has its risks but also its benefits.


UPDATE — My exchange with Eduwonkette continues although it seems increasingly pointless.  Here is my (slightly edited) last comment on her site:

“Let’s make this very concrete. Was it inappropriate for Marcus Winters and I to release our social promotion findings in 2004 without peer review, or should we have waited until it had been peer-reviewed and published (in various forms) in 2006, 2007, and again in 2008? If the appropriate thing is to wait, would interest groups, editorial boards, and bloggers similarly hold their tongues until the additional evidence came in?  Would policymakers hold off on decisions that might have come out differently if they had the suppressed information?

Would it have been OK to release in 2004 as long as we tried to make it obscure enough so that people were less likely to find it? What if interest groups, bloggers, etc… found our obscure finding and promoted them (as has happened with Jesse Rothstein’s paper)?

And in saying ‘working papers and thinktank reports are released for entirely different functions’ you are repeating your call for an analysis of motives. You’ve said that think tanks want to influence policy (bad motive) while academics are trying to advance knowledge with each other (good motive). But if academics are serving the public good, shouldn’t they ultimately want to influence policy? I am an academic who also releases working papers through a think tank. Does that make my motives good or bad? I think all of this analysis of motives is silly when the real issue is the truth of claims, not why people are making those claims. Calling for an analysis of motives is especially silly for someone who is trying to influence people anonymously. The fact that you are trying to influence people through a blog does not give you a free pass from having to be consistent on this.”

Blog Rankings

July 14, 2008

This blog is not yet three months old but I am pleased to report that it is off to a good start.  According to Technorati’s rankings, is attracting more readers than the American Federation of Teachers’ blog, Edwize, more than Diane Ravitch and Deborah Meier’s, Bridging Differences hosted by Education Week, more than the Reason Foundation’s Out of Control, and the Center for Education Reform’s Edspresso.  It significantly trails the educouple of Eduwonk and Eduwonkette as well as Cato at Liberty (although that’s not primarily an education blog).  Flypaper, which started about the same time as this blog, is also off to a good start.  The Queen of education blogs seems to be Joanne Jacobs.

Here are the Technorati rankings (as of this morning) of education sites that seem to share some of the same audience as this blog.  By no means is this a comprehensive list of education blogs.  And I have no idea how reliable or meaningful Technorati’s rankings really are.  I’d continue blogging no matter what the rankings were because it’s fun.  I imagine the same is true of most others.

  1. Cato at Liberty               3,662
  2. Joanne Jacobs                3,709
  3. Eduwonkette                27,419
  4. Eduwonk                      30,876
  5. Flypaper                       95,943
  6. Jay P. Greene               104,227
  7. Bridging Differences   107,924
  8. D-Ed Reckoning         107,924
  9. AFT’s Edwize              116,227
  10. Edspresso                  123,039
  11. Out of Control            123,039
  12. Core Knowledge         127,851
  13. Sherman Dorn            151,703
  14. EdBizBuzz                   184,730

See, we’re in Italy…

July 12, 2008


“See, we’re in Italy.  The guy on the top bunk has gotta make the guy on the bottom’s bed all the time.  It’s in the regulations.  If we were in Germany I would have to make yours.  But we’re in Italy, so you’ve
gotta make mine. It’s regulations.”

This is more or less Eduwonkette’s response to my complaint that she can’t argue that the source of information is important in assessing the truth of claims while blogging anonymously.  Her answer is that it’s different for bloggers (in Italy) than for researchers (in Germany).  It’s regulations.

She goes on to describe some differences between different types of information in education policy debates, but it’s not clear why any of those differences would be relevant to whether assessing the source is important for one and not for another.  The closest she comes to explaining why things are meaningfully different is when she says, “And let’s be realistic: an anonymous blogger isn’t shaping public policy.”  So, if information will have no bearing on policy debates, then its source is unimportant.

This would be a consistent argument if she really believed that bloggers had NO influence.  But of course they have at least some influence.  Why else would she and the rest of us be bothering with this?  And if bloggers have some influence, then the same basic principles should apply: either we should analyze the motives of sources of information to assess the truth of claims or we shouldn’t.  I’m in favor of not analyzing motives for anyone since I think that the truth of claims is independent of the motives of the source.  Even bad people can make true arguments.

At the risk of belaboring this issue, maybe I can clarify things by describing the market of ideas in policy debates as being like the market for cars.  We have different levels of confidence in cars that have gone through different processes before being made available for sale.  We could buy a used car from the corner used car dealer with no warranty.  That would be like reading blogs.  We don’t really know whether we are getting a lemon or not, since almost no assurances have been made about quality.  Or we could buy a used car from a larger chain with at least some warranty.  That would be like getting information from newspapers or magazines.  There has been some review and assurance of quality, but we still don’t quite know what we’ll get.  Or we could buy a new car from a major dealer and buy the extended warranty.  That would be like getting information from a peer-reviewed journal.  It may still be a lemon, but we’ve received a lot of assurances that it is not.  And I suppose reading an anonymous blogger is like buying a used car from someone you don’t know in the want ads.  There are trade-offs in getting cars with these different level of assurances about quality, just as there are trade-offs in getting information that has gone through different processes to assure quality. 

Eduwonkette’s argument is essentially that the same rules regarding these trade-offs don’t apply to the market for cars without warranties that do apply to the market for cars with warranties.  My view is that there are only differences in degree, not kind.  Even bad people can sell cars that are good values.

I’ve also noticed that Marc Dean Millot has weighed in on this issue.  He’s just knocking down a straw man.  It is not my position that research doesn’t benefit from peer review.  He can check out my cv to see that I have two dozen peer-reviewed publications, many of which were earlier released directly to the public without review.

I’ve been arguing that the public benefits from seeing research even before it has received peer review because it gets more information faster.  Without the assurances of peer review people will tend to have lower confidence in that research, and their confidence may increase as the research receives those additional assurances.  Millot seems to want to embargo information from the public until it receives peer review.  If he really believes that, then he should criticize every researcher with working papers on the web.  That’s almost everyone doing serious research.

And on his points about ideology tainting research I would suggest that people read Greg Forster’s excellent earlier post on Vouchers: Evidence and Ideology.

Eduresponses to Edubloggers

July 10, 2008

My recent posts on the release of our new study on the effects of high-stakes testing in Florida and posts here and here on the appropriateness of releasing it before it has appeared in a scholarly journal, have produced a number of reactions.  Let me briefly note and respond to some of those reactions.

First, Eduwonkette, who started this all, has oddly not responded.  This is strange because I caught her in a glaring contradiction: she asserts that the credibility of the source of information is an important part of assessing the truth of a claim yet her anonymity prevents everyone from assessing her credibility.  I prefer that she resolve this contradiction by agreeing with my earlier defense of her anonymity that the truth of a claim is not dependent on who makes it.  But she has to resolve this one way or another — either she ends her anonymity or she drops the argument that we should assess the source when determining truth.

But apparently she doesn’t have to do anything.  Whose reputation suffers if she refuses to be consistent?  Her anonymity is producing just the sort of irresponsibility that Andy Rotherham warned about in the NY Sun and that I acknowledged even as I defended her.  The only reputation that is getting soiled is that of Education Week for agreeing to host her blog anonymously.  If she doesn’t resolve her double-standard by either revising her argument or dropping her anonymity, Education Week should stop hosting her.  They shouldn’t lend their reputation to someone who will tarnish it.

Mike Petrilli over at Flypaper praises our new study on high stakes testing but takes issue with referencing comments by Chester Finn and Diane Ravitch about how high stakes is narrowing the curriculum in the “pre-release spin.”  I agree with him that this study is not “the last word on the ‘narrowing of the curriculum.’”  But to the extent that it shows that another part of the curriculum (science) benefits when stakes are applied only to math and reading, it alleviates the concerns Checker and Diane have expressed. 

As we fully acknowledge in the study, we don’t have evidence on what happens to history, art, or other parts of the curriculum.  And we only have evidence from Florida, so we don’t know if there are different effects in other states.  But the evidence that high stakes in math and reading contribute to learning in science should make us less convinced that all low stakes subjects are harmed.  Perhaps school-wide reforms that flow from high stakes in math and reading produce improvements across the curriculum.  Perhaps improved basic skills in literacy and numeracy have spill-over benefits in history, art, and everything else as students can more effectively read their art texts and analyze data in history.

Andy Rotherham at Eduwonk laments that what I describe as our “caveat emptor market of ideas” doesn’t work very well.  I agree with him that people make plenty of mistakes.  But I also agree with him that “in terms of remedies there is no substitute for smart consumption of information and research…”  There is no Truth Committee that will figure everything out for us.  And any process of reviewing claims before release will make its own errors and will come at some expense of delay.  Think Tank West has added some useful points on this issue.

Sherman Dorn, who rarely has a kind word for me, says: “Jay Greene (one of the Manhattan Institute report’s authors and a key part of the think tank’s stable of writers) replied with probably the best argument against eduwonkette(or any blogger) in favor of using PR firms for unvetted research: as with blogs, publicizing unvetted reports involves a tradeoff between review and publishing speed, a tradeoff that reporters and other readers are aware of.”  He goes on to have a very lengthy discussion of the issue, but I was hypnotized by his rare praise, so I haven’t yet had a chance to take in everything else he said.

%d bloggers like this: