PBS News Hour on Ohio 3rd Grade Reading Guarantee

April 22, 2015

(Guest Post by Matthew Ladner)

Balanced piece on Ohio’s reading policies. Interesting that opponents make complaints about retention happening at all and then about the bar being set too low. The firm but incremental approach advocated by Senator Lehner demonstrates both wisdom and resolve in my view.

Advertisements

New Social Promotion Study

July 25, 2012

Marcus Winters and I have a new study on the effects of ending social promotion in Florida that appeared this month in the journal, Education Finance and Policy.  In our earlier published research we observed that retained students made greater academic gains in subsequent grades than did promoted students who were just like them.  But we could only track students for 2 years after the intervention, so we didn’t know if the benefits we observed compounded or faded over time.

In the new study we track students for as many as 5 years after retention.  The benefits of the policy do diminish, but they remain statistically significant and educationally substantial through middle school.  We hope to continue tracking these students through high school, graduation, and even college, but so far it looks like there are enduring benefits to ending social promotion.


Ohio Superintendent to Lawmakers-Please Ignore the Illiterates Behind the Curtain

July 5, 2012

(Guest Post by Matthew Ladner)

Ohio recently passed a law to curtail social promotion from the 3rd grade for students requiring extra help in reading. Not everyone is thrilled as a Ohio Newspaper recently reported “Pike-Delta-York Superintendent Ken Jones thinks the state is overstepping its boundaries with the mandate.   ‘School administrators and parents are smart enough to figure all that out. We don’t need the state coming in and telling us how to operate or telling us how to move kids through our system,’ he said.”

Hmmmm

I’m sure Ohio school administrators are plenty smart, but they might want to go look at some data before making their mind up about the need for this policy. The figure below draws data from the NAEP data explorer, showing the percentage of Ohio students scoring “Below Basic” and “Proficient or Better” on the 2011 4th grade reading NAEP by ethnicity.

Superintendent Jones does seem to have one thing right- he and his fellow Ohio Superintendents apparently know all about “moving kids through the system”- even the students who desperately need more help in mastering basic literacy skills. Note that among Ohio’s Black students that four times as many scored Below Basic as Proficient. The fact that fifty-four percent of Ohio’s Black 4th graders couldn’t read in 2011 hardly constitutes a firm basis for a “steady as she goes!” declaration.

Superintendent Jones may feel confident that he and his compatriots have this whole reading thing figured out, but it is little wonder why Ohio legislators and Governor Kasich saw things a bit differently. If I really wanted to be cruel I would go look up the numbers for Cleveland in the TUDA…err…wait….too late!

More than twice as many White students scoring Below Basic as Proficient,  almost 13 times as many Black students scoring in the illiterate area as Proficient, seven times as many among Hispanics. Little wonder that Ohio lawmakers also decided to depart from the status-quo again and turn the district over to the Mayor.

What makes anyone think that this policy will make things better? Well there are no guarantees that Ohio will implement it as well as Florida, but here is what happened in Florida:

I’m sure that life would be easier for Ohio Superintendents if lawmakers would just keep sending the money to the districts without asking any questions. I’m also however certain that it would not make life for students any easier if Ohio continued to ignore what is plainly a literacy crisis.


Bipartisan Group of NC Legislators Override Veto, Enact K-12 Reform

July 3, 2012

(Guest Post by Matthew Ladner)

The North Carolina legislature voted to override the budget veto of Governor Purdue, enacting significant K-12 reform in the process. Reforms included in the budget include A-F school grading, curtailment of social promotion and merit pay for teachers.

I think that the map of states having adopting A-F school grades now looks like this, although I may have missed a state. The star represents New York City:


Ed Week on 3rd Grade Retention

March 27, 2012

(Guest Post by Matthew Ladner)

Ed Week on the 3rd grade retention debate, including quotes from my Foundation for Excellence in Education colleague Jaryn Emhof and Marcus Winters on his forthcoming research on persistence of the retention effect. I am very proud of our lawmakers in Arizona and especially Governor Brewer and Senator Crandall for taking action to close loopholes in the Arizona law.

Arizona has a sad history of punting on tough reforms, having dummied down the state AIMS test by dropping the cut scores and having delayed the enactment of using the AIMS test as a high school exit exam several times. The 3rd grade retention law that passed in 2010 put the new standard in place for incoming kindergarteners the following year, giving an ease-in adjustment period for the districts. Last year at the Arizona School Boards Association conference, an Arizona Superintendent confided in me that “we’ve found the loophole in the retention law, and we are getting ready to use it.”

Fortunately, Governor Brewer’s team found the loophole as well, and are taking action to close it. This law is going to be a tremendous test of character for the Arizona education community of the sort we have failed in the past. Reading intervention should have the top priority for every dime of federal funding received for K-3 students. All of the Title programs can be used to support early reading intervention, remediation, and professional development. So long as we are really going to see the policy through, I support Governor Brewer’s call for additional state resources without reservation.

The only time Arizona officeholders garner attention seems to be when they do something controversial or downright nutty. Kudos for getting some things right!


Grade Retention is Common Nationally but Effective in Florida

February 28, 2012

(Guest Post by Matthew Ladner)

I came across an interesting study from NCES recently concerning the practice of grade retention that creates yet another hole in NEPC boat regarding their Florida theories. In fact, here is a link to a study from the ASU precursor to NEPC by Columbia Teacher College Professor Chatterji (one of the NEPC critics) from 2003 calling on Florida to “rethink sanction and retention policies in light of new and past research showing that retention does not improve student achievement.” 

Now you can look at the below figure and ask yourself just who needs to reconsider what. The red line is FCAT 1 scores for Black students, the Green line is for Hispanic students, and the blue line is for all students.

The NEPC boat is already sitting on the floor of the ocean, but hey, why not drop a depth charge on it?

The main pet theory of the NEPC squad has been that Florida’s 4th grade NAEP scores have been profoundly warped by the state’s retention policy. This beats the daylights out of their Harry Potter theory, but there still is far less to it than meets the eye. Problems with this theory include a substantial improvement in 4th grade NAEP scores before the retention policy went into place, a substantial decline in retentions since the onset of the policy, and a substantial improvement in 3rd grade reading FCAT scores.  Oh and the advent of mid-year promotions and a few other things which NEPC has been either unable or unwilling to address. The peak of any aging effect would have come in 2005 and declined substantially, and yet Florida’s scores continued to rise.

An implicit assumption of this theory was that Florida is doing far more K-3 retention than other states around the country. After seeing this NCES study, I am no longer certain this is the case, especially now that Florida retention has fallen so substantially. Let’s dig into the data and find out.

State level data on grade level retention is very difficult to come by outside of Florida. However, NCES included a question about retention in their parent survey. Low and behold, 10% of parents in the NCES survey report that their child has been retained for one or more grade in grades K-8, more than 20% of low-income parents.

NCES: Students retained in one or more grade, K-8

So first off, this is quite a bit higher than I would have suspected and the trend has been rising. Given the hostility that many College of Education Professors have towards grade retention, it seems apparent that many of the teachers and administrators that go through their programs are not buying what they are selling on retention.

Now that we have a measure of retention nationally, we should explore the question of how prevalent the practice is in Florida. The Florida Department of Education provides this handy chart for the statewide numbers for retention for students in grades K-12. The technical term to describe this chart is “falling off a cliff.”

So if you rummage around in the spreadsheet provided by the Florida Department of Education on retention by grade level and add a few cells together, you can calculate that the total retention figure in Florida in 2009-2010 for Grades K-8 was 54,843.

That sounds like a lot, until you go over to the NCES Common Core Data (note to Jay, Greg and MWAB- not the academic standards, please call off the cruise missle strike :-) and learn that there were over 1.7 million students in the Florida K-8 system in 2009-10. When you do the math, it turns out that 3.9% of Florida K-8 students were retained during the 2009-2010 school year. What about the peak of Florida retention the year the 3rd grade retention policy took place in 2003-04? The total retention rate for that year was (waaaaait for it…..) 5.5%- a little more than half of the national rate that the NCES found in 2007.

We don’t have national data for K-3 retention, which is what we would need to do an ideal comparison, but the data we do have certainly establishes that there is a substantial amount of retention going on around the country, which will be having some impact on NAEP scores of states across the nation, not just Florida. Unless a state is doing far more than average, it retention is likely to be white noise overall- blips in the error term. Furthermore, it is not clear that Florida was doing more K-3 retention than the national average, even during the peak of the practice in 2003-04.

Mind you that I make no claim that retention is necessarily a good practice overall. I think there have been terrible retention practices, such as the practice of “redshirting” 9th graders in Texas back when the state gave a 10th grade exit exam. Redshirting was a widespread district level practice not mandated by state law and it was truly an awful policy basically designed to get students to drop out of school in 9th grade and thereby inflate the passing rate for the 10th grade exit exam.

There was nothing admirable about Texas redshirting. I would venture to guess that both a casual and a sophisticated analysis of data would have found it associated with higher drop out rates.

The Florida policy however is the opposite of the old Texas practice in that it is designed to set kids up to succeed rather than to fail. Not only have there been bad retention practices, there has also been a great deal of bad research done on retention that lacked the statistical rigor to establish causality. Do cancer drugs kill people, or is it the cancer? Most of the retention research doesn’t allow us to answer that sort of question.

Jay, Marcus Winters and the RAND Corp however have been applying sophisticated regression discontinuity designs to retention policies in Florida and New York City. They have found positive academic results. RAND found no self-esteem harm to students, and that NYC educators have generally positive views of the policy, to boot.

The question is not whether retention is “good” or “bad”- that all depends on how it is used. The evidence on the overall literacy effort in Florida-which includes retention as a centerpiece-is overwhelmingly positive.


2011 NAEP: Florida Finally Hits a Wall

November 3, 2011

(Guest Post by Matthew Ladner)

The Florida Age of Public School Improvement hit a wall in the 2011 NAEP. This should not be terribly surprising, as Florida’s improvement seemed certain to plateau in the absence of additional reforms.

Governor Jeb Bush relentlessly pursued a dual strategy- transparency with teeth from the top down, parental choice from the bottom up. Together these reforms drove improvement in the public schools for a number of years.  Accountability measures included school grading (A-F) and earned promotion in the early grades. Parental choice measures included Opportunity Scholarships for children attending F rated schools, the nation’s first special needs voucher program (McKay Scholarships), the nation’s largest scholarship tax credit program (Step Up for Students), a decent charter school law and the nation’s most robust system of digital learning. Florida lawmakers also attempted to thoughtfully incentivize success.

Governor Bush took office in 1999 and left office in 2007. It would be nice if these efforts could indefinitely push progress forward, but there have been plenty of bumps and problems along the way. In 2006, the Florida Supreme Court rendered a logic-free ruling abolishing Opportunity Scholarships (failing school vouchers) for private schools, and followed that up by ruling against a state authorizer for charter schools. Tax-credits, McKay and digital learning continued to incrementally advance, but not at an earth-shattering rate.

The larger problem may have come in the top down measures. The chart below presents the distribution of district and charter school grades, with one line being the A/B grades and the other D/F grades. The dotted lines represent instances when the state board raised school grading standards.

The setting of these standards represents far more of an art than a science. Set them far too high and disaster follows (this happened in Arizona). Set them too low, and you remove the tension in the system needed to drive improvement. Even after the last increase in grading standards, more than 10 times as many Florida schools received A/B grades as D/F grades.

Florida’s policymakers raised standards four times, and last year (wisely) put in an automatic trigger to raise standards by a preset amount when a certain ratio of schools get A or B grades. In addition, a fresh set of reforms passed the Florida legislature in 2011, revamping teaching and increasing charter school and digital learning options.

Just as it is impossible to exactly pinpoint how much of what caused the gains, it is likewise impossible to say exactly what made them stall. Note however that one of the favorite explanations of the anti-reform crowd, the pre-school, finally saw the advent of children old enough to have participated in the program and age into the 4th grade NAEP sample. I hope that someone is carefully studying variation in participation and corresponding trends in FCAT data, but the results at the aggregate level thus far seem underwhelming.

Plenty of other things, however, have been going on- including the collapse of a housing bubble, cutbacks in public school funding (including of some of the incentive funding programs) and a variety of other very bad things. My advice to Florida policymakers: roll up your sleeves and get back at it. Despite the enormous amount of progress seen on NAEP (and no one loves celebrating it more than me) too great of a gulf lies between a state system awarding ten times as many top grades as low grades but still  suffering from large minorities of students scoring below basic on the NAEP exams.

Governor Bush has consistently said for years that success is never final, and reform is never finished. The 2011 pause in progress demonstrates that he called it correctly.  Moving the needle on student learning on a meaningful scale and at a sustained basis represents one of the greatest public policy challenges of our times. Governor Bush has passed the torch to a new generation of Florida reformers, and they must now find new ways, and fine-tune the old ways, to push academic progress forward.

Edited for typos