(Guest Post by Matthew Ladner)
I feared that my career had peaked upon receiving a Bunkum Award from the NEA’s rent-a-reactionary academic shop, but today I learned that I can now die happy as the first recipient of a Lifetime Bunkum Award. The prestigious award reads as follows:
NEPC has never bestowed an individual with a Bunkum Award. But we’ve never before had someone campaign for one, and we’ve never before found someone with an individual record of Bunkum-worthy accomplishments that cries out for recognition. This year, however, we are honoring Matthew Ladner, an advisor to former Florida Governor Jeb Bush’s advocacy organization, the “Foundation for Excellence in Education.”Dr. Ladner’s body of Bunk-work is focused on his shameless hawking of what he and the Governor call the “Florida Formula” for educational success. As our reviews have explained, they’d be less deceptive if they were selling prime Florida swampland. One cannot, however, deny Dr. Ladner’s salesmanship: gullible lawmakers throughout the nation have been pulling out their wallets and buying into his evidence-less pitch for flunking of low-scoring third graders and other policies likely to harm many more students than they help. See here and here for more analysis of Dr. Ladner’s body of bunk and its unfortunate reach.Our judges were particularly impressed recently, when Ladner attributed Florida’s “hitting the wall” drop in NAEP scores to a collapse in the housing bubble and other “impossible to say” factors. Bunkums have been awarded for far less impressive an accomplishment than this sort of “heads I win, tails you lose” use of evidence. So Matt Ladner – this Bunkum’s for you.You can watch NEPC’s award ceremony youtube here:
My reaction:
Honestly I can’t take credit for this great honor. It was Governor Bush and his team of fearless reformers who ignored the wailing howls of K-12 reactionaries and forced through a set of reforms that improved Florida education steadily over time. It is they who deserve credit for moving Florida from one of the worst performing states by ignoring the “expertise” of NEPC’s ideological tribe and drove their low-income literacy scores above statewide averages for all students.
My role in all of this has simply been to help document the progress, all of which happened over the howling objections of NEPC’s soul mates. NEPC has mounted a series of feeble attempts to muddy the water. Their first effort completely ignored a peer-reviewed article in the nation’s most influential education policy journal that fell directly on point to concerns raised in the article. Oh and it also contained an appendix that refuted its own central thesis. Undeterred, the next effort a “review” of a Powerpoint presentation that the critic didn’t see. All of this climaxed with sending out one of their scholars to claim that Harry Potter books may have caused the improvement in Florida reading scores. This is, you see, because Harry Potter books are seldom read outside of Florida, and no, I am not making this up. An audience of hundreds witnessed it with their own eyes.
I am thrilled to receive this Lifetime Achievement award. Reformers around the country have begun the process of making K-12 policy based upon things other than the political preferences of the special interests organized around the K-12 status-quo. If this grand undertaking were a play, I would have but a small role in it-this is far, far, far bigger than me and bigger than Florida. Notice for instance that both the Progressive Policy Institute and the Center for American Progress earned NEPC Bunkum Awards this year (congratulations!) which is a probable sign that those groups are doing good work and a certain sign of the political and intellectual isolation of the teacher union left.
I want to thank my family, my teachers and professors, my mentors and all the other people who helped me to win this unique and prestigious award. You know who you are and you hold my deepest appreciation. I want to thank those who fought so hard to produce the gains which NEPC is so desperate to obscure. Most of all I want to thank NEPC for revealing what they fear most, which we can infer from this year’s Bunkum ritual seems to be the success of reformers and their own isolation from their former allies in the morally and intellectually serious left, apparently in that order.
I will now redouble my efforts in the hope of becoming the first winner of a second lifetime Bunkum Award. Otherwise, I will have no worlds yet to conquer.
Jay P. Greene's Blog 
I fear that I am the “Susan Lucci” of Bunkum Awards — passed over for that prestigious honor for 13 straight years now. What, pray tell, must I do to earn this hollowed distinction? The NEPC crowd seems to be motivated by money, judging by the hundreds of thousands of teachers union money that annualy rolls into its coffers after having been laundered by the Great Lakes Center. So maybe I should bribe them!
We here at the LondonTex household are so very proud of you. What an accomplishment!
Minnesota-
Don’t despair! Keep working hard at producing logical arguments with rock solid evidence to back them up, and your day will come. Just keep at it!
Thank you AHL!
Congratulations.
We should all be concerned, even “disturbed”, about the 2011 NAEP scores for 8th grade reading in Florida. From the NAEP site:
“In 2011, Black students had an average score that was 22
points lower than White students. This performance gap
was not significantly different from that in 1998 (28 points).
In 2011, Hispanic students had an average score that was
11 points lower than White students. This performance gap
was not significantly different from that in 1998 (17 points).
In 2011, female students in Florida had an average score
that was higher than male students by 9 points.
In 2011, students who were eligible for free/reduced-price
school lunch, an indicator of low family income, had an
average score that was 19 points lower than students who
were not eligible for free/reduced-price school lunch. This
performance gap was not significantly different from that in
1998 (24 points).
In 2011, the average score of eighth-grade students in Florida was 262. This was not significantly different from the average score of 264 for public school students in the nation.”
RE: Ms. Samuelson’s comments
Looking only the gaps falls into the Simpson’s Paradox problem with NAEP scores (which in this context relates to how overall average NAEP scores have stayed relatively flat while scores for every racial minority have made notable improvement over the years).
As a note, guidance found in all recent NAEP reports says that you need to pay attention to things like student demographic shifts and exclusion rate changes over time when interpreting NAEP results. When you do that, Florida has made progress for its racial groups.
A little work with the NAEP Data Explorer shows Florida can claim that between 1998 and 2011 in Grade 8 NAEP reading for public school students:
White scores increased by 6 points, a statistically significant improvement.
Black scores increased by 12 points, a statistically significant improvement.
Hispanic scores also increased by 12 points, but due to demographically-driven sample size issues for 1998, this was not statistically significant.
Why is the 12-point rise significant for blacks but not Hispanics? It is because over the 13 years between 1998 and 2011 Florida’s Hispanic public school population grew from just 13 percent of the enrollment to 27 percent. The notably smaller enrollment in 1998 led to a very large standard error in Florida’s 1998 Hispanic scores, making it much harder for Florida to get credit for a statistically significant improvement in scores for this racial group.
Florida’s key racial groups have also improved in comparison with national average scores since 1998.
Florida’s whites scored 4 points lower than the national white public school average in 1998, a statistically significant difference. By 2011 Florida was only 2 points behind the national public school white average, a statistical tie.
Florida’s blacks scored a statistically significant 6 points behind the national black public school average in 1998 in Grade 8 NAEP reading. By 2011, they tied the national average.
Florida’s Hispanics scored 6 points above the national average in 1998, and increased that lead by 2 more points in 2011. However, again due to the sampling issue for Hispanics in 1998, the 1998 difference was not statistically significant. By 2011, we can be confident that the score difference is statistically different, favoring Florida.
There is more.
As Dr. Ladner and I discussed earlier, and as he has commented upon in this blog, exclusion rates for students with learning disabilities are problematic for NAEP, especially the reading results. Of note, in 1998 Florida excluded 5 percent of the entire raw student sample NAEP wanted to test due to learning disabilities. That’s not 5 percent of the learning disabled students – it is 5 percent of the entire raw sample of all students initially selected to test. In 2011 Florida dropped that exclusion rate by more than half to only 2 percent.
Including more of these special students tends to depress scores. Florida’s improvements noted above would look even better if the state had maintained the higher levels of exclusion. However, the fact that many more of the state’s learning disabled students were able to sit for the NAEP in 2011 is a testimony to growing effectiveness in reaching those special students. Adding to that impression, after rounding in the scores considered, in 1998 Florida’s learning disabled students scored 3 points above the national public school average for learning disabled students. The difference was not significant. In 2011, Florida’s tested learning disabled population, despite the fact that far fewer of them were excluded, scored 6 points higher than the national average and this was a statistically significant difference.
Do the gaps remain? Yes, they do, just as they do in every other region of the country.
But, that does not mean Florida has not made progress with all racial groups, generally moving from behind or only tied with the nation to now performing equally or better than the nation. It’s just that you have to understand the Simpson’s Paradox problem in order to see it.
So, a good case can be made that Florida has made notable improvement, and the rest of the country should take notice.
For more on the pitfalls of interpreting NAEP, go to freedomkentucky.org and search for the article, “The National Assessment of Educational Progress.”
As the only individual recipient of a Bunkum, this kind of makes you royalty. I hereby dub you Sir Matthew of Bunkum.
AMS-
There isn’t any doubt that Florida is still in need of yet more reform. Although reactionaries fought hard to prevent them, the reforms that could have impacted NAEP scores to date are clearly modest departures from the mid-20th Century factory model of public schooling that has served disadvantaged students so poorly.
Hang in there- the best is yet to come.
Matt,
But the point is that the ed system leaves many students behind, all along the entire spectrum of abilities and talents despite socioeconomic status. Just because a student attends an “A” school in Florida does not mean that he or she is getting a proper education. And frankly, there are just not enough choices (supply) out there to meet the demand. All students/parents must be given the proper opportunity to meet their educational needs.
I could not agree more that far more remains to be done than has been done so far.
You and Paul Peterson should get together and write an ednerd satire newspaper.
Matthew,
I’m not sure why anyone takes Kevin Welner seriously. A cursory glance over his CV and it’s clear he publishes almost exclusively in 3rd tier education journals with the primary thickness of his Vita being padded by chapters in edited book volumes (volumes published no doubt by ideologically sympathetic education faculty). What don’t you see on his CV?
Journal of Policy Analysis and Management
American Economic Review
Economics of Education Review
Education Finance and Policy
What do you see?
A lot of law review articles, Teachers College Record, a few EEPA… not exactly a Caroline Hoxby, Hanushek, Paul Peterson level CV. I mean, even SchoolFinance guy Bruce Baker publishes in better outlets than Welner. Again, why does anyone take him serious?
Reporters aren’t usually familiar with these distinctions. And in any event anyone peddling a welcome story won’t have his credentials scrutinized too closely.
A cursory search of the Welner name came up with a recent co-authored publication in Educational Researcher. ER is a top tier Ed. journal published by AERA. So, Richard, your ad hominem argument is quite easily refuted (even though attacks on a person’s CV seems almost too ridiculous to warrant a reply).
1 publication in ER does not make someone an authority. It’s a joke that with a CV that is padded by chapters in edited volumes and MOSTLY (I didn’t say entirely) I said MOSTLY 2nd and 3rd tier journals ends up with a title such as Welner’s. I ask you: who has a better CV? Paul Peterson or Kevin Welner? Caroline Hoxby or Kevin Welner? Eric Hanushek or Kevin Welner? ER is not even close to being as difficult to publish in as AER, EER, or EFP… not even close. And you’re hanging onto a single co-authored pub from ER to say that Welner is an academic big boy. Puleazzzze. And it’s not ad hominem. Professor’s should be judged by their output. Calling Welner’s output weak based on his publication records is fair game. My point is he gets far more attention in ed policy circles than is warranted by his CV. Heck, I could say the same thing about a supposed Icon like Linda Darling Hammond. She wouldn’t know HLM models from T-Tests… okay that’s a little harsh, but this isn’t about left vs. right, it’s about who publishes in the big boy journals. I don’t agree with Bruce Baker on much, but BB at least publishes in the big boys.
Welner’s problems go far deeper than credentials. I think this quote from Arthur Levine’s review of College of Education dissertations suggests why Welner has a difficult time finding anyone with a sophisticated grasp on research to write reviews for him:
“In general, the research questions were unworthy of a doctoral dissertation, literature reviews were dated and cursory, study designs were seriously flawed, samples were small and particularistic, confounding variables were not taken into account, perceptions were commonly used as proxies for reality, statistical analyses were performed frequently on meaningless data, and conclusions and recommendations were often superficial and without merit.”
Thus when advanced regression discontinuity design studies become available on topics such as the impact of retention, Welner’s reaction is to ignore them while clinging to outdated and unsophisticated studies on out of date policies.
If Welner wants to clean up education research, he should start with his own tribe. Alas the NEA is unlikely to subsidize such an effort, so they are likely to soldier on renting the University of Colorado’s name instead.
[…] they fail to point out in giving Ladner this “award” is, as Ladner points out in this biting response posting, that the NEPC is funded by teachers unions which have a vested interest in opposing any and all […]