(Guest Post by Matthew Ladner)
Balanced piece on Ohio’s reading policies. Interesting that opponents make complaints about retention happening at all and then about the bar being set too low. The firm but incremental approach advocated by Senator Lehner demonstrates both wisdom and resolve in my view.
Leave a Comment » | 
social promotion | Tagged: Ohio Reading Guarantee, Senator Peggy Lehner | 
Permalink
Posted by matthewladner
Marcus Winters and I have a new study on the effects of ending social promotion in Florida that appeared this month in the journal, Education Finance and Policy. In our earlier published research we observed that retained students made greater academic gains in subsequent grades than did promoted students who were just like them. But we could only track students for 2 years after the intervention, so we didn’t know if the benefits we observed compounded or faded over time.
In the new study we track students for as many as 5 years after retention. The benefits of the policy do diminish, but they remain statistically significant and educationally substantial through middle school. We hope to continue tracking these students through high school, graduation, and even college, but so far it looks like there are enduring benefits to ending social promotion.
1 Comment | social promotion | Permalink
Posted by Jay P. Greene
(Guest Post by Matthew Ladner)
Ohio recently passed a law to curtail social promotion from the 3rd grade for students requiring extra help in reading. Not everyone is thrilled as a Ohio Newspaper recently reported “Pike-Delta-York Superintendent Ken Jones thinks the state is overstepping its boundaries with the mandate. ‘School administrators and parents are smart enough to figure all that out. We don’t need the state coming in and telling us how to operate or telling us how to move kids through our system,’ he said.”
Hmmmm
I’m sure Ohio school administrators are plenty smart, but they might want to go look at some data before making their mind up about the need for this policy. The figure below draws data from the NAEP data explorer, showing the percentage of Ohio students scoring “Below Basic” and “Proficient or Better” on the 2011 4th grade reading NAEP by ethnicity.
Superintendent Jones does seem to have one thing right- he and his fellow Ohio Superintendents apparently know all about “moving kids through the system”- even the students who desperately need more help in mastering basic literacy skills. Note that among Ohio’s Black students that four times as many scored Below Basic as Proficient. The fact that fifty-four percent of Ohio’s Black 4th graders couldn’t read in 2011 hardly constitutes a firm basis for a “steady as she goes!” declaration.
Superintendent Jones may feel confident that he and his compatriots have this whole reading thing figured out, but it is little wonder why Ohio legislators and Governor Kasich saw things a bit differently. If I really wanted to be cruel I would go look up the numbers for Cleveland in the TUDA…err…wait….too late!
More than twice as many White students scoring Below Basic as Proficient, almost 13 times as many Black students scoring in the illiterate area as Proficient, seven times as many among Hispanics. Little wonder that Ohio lawmakers also decided to depart from the status-quo again and turn the district over to the Mayor.
What makes anyone think that this policy will make things better? Well there are no guarantees that Ohio will implement it as well as Florida, but here is what happened in Florida:
I’m sure that life would be easier for Ohio Superintendents if lawmakers would just keep sending the money to the districts without asking any questions. I’m also however certain that it would not make life for students any easier if Ohio continued to ignore what is plainly a literacy crisis.
9 Comments | social promotion | Permalink
Posted by matthewladner
(Guest Post by Matthew Ladner)
The North Carolina legislature voted to override the budget veto of Governor Purdue, enacting significant K-12 reform in the process. Reforms included in the budget include A-F school grading, curtailment of social promotion and merit pay for teachers.
I think that the map of states having adopting A-F school grades now looks like this, although I may have missed a state. The star represents New York City:
1 Comment | school accountability, social promotion | Permalink
Posted by matthewladner
(Guest Post by Matthew Ladner)
Ed Week on the 3rd grade retention debate, including quotes from my Foundation for Excellence in Education colleague Jaryn Emhof and Marcus Winters on his forthcoming research on persistence of the retention effect. I am very proud of our lawmakers in Arizona and especially Governor Brewer and Senator Crandall for taking action to close loopholes in the Arizona law.
Arizona has a sad history of punting on tough reforms, having dummied down the state AIMS test by dropping the cut scores and having delayed the enactment of using the AIMS test as a high school exit exam several times. The 3rd grade retention law that passed in 2010 put the new standard in place for incoming kindergarteners the following year, giving an ease-in adjustment period for the districts. Last year at the Arizona School Boards Association conference, an Arizona Superintendent confided in me that “we’ve found the loophole in the retention law, and we are getting ready to use it.”
Fortunately, Governor Brewer’s team found the loophole as well, and are taking action to close it. This law is going to be a tremendous test of character for the Arizona education community of the sort we have failed in the past. Reading intervention should have the top priority for every dime of federal funding received for K-3 students. All of the Title programs can be used to support early reading intervention, remediation, and professional development. So long as we are really going to see the policy through, I support Governor Brewer’s call for additional state resources without reservation.
The only time Arizona officeholders garner attention seems to be when they do something controversial or downright nutty. Kudos for getting some things right!
1 Comment | social promotion | Tagged: 3rd grade retention, Ed Week, Jaryn Emhof, Marcus Winters | Permalink
Posted by matthewladner
(Guest Post by Matthew Ladner)
I came across an interesting study from NCES recently concerning the practice of grade retention that creates yet another hole in NEPC boat regarding their Florida theories. In fact, here is a link to a study from the ASU precursor to NEPC by Columbia Teacher College Professor Chatterji (one of the NEPC critics) from 2003 calling on Florida to “rethink sanction and retention policies in light of new and past research showing that retention does not improve student achievement.”
Now you can look at the below figure and ask yourself just who needs to reconsider what. The red line is FCAT 1 scores for Black students, the Green line is for Hispanic students, and the blue line is for all students.
The NEPC boat is already sitting on the floor of the ocean, but hey, why not drop a depth charge on it?
The main pet theory of the NEPC squad has been that Florida’s 4th grade NAEP scores have been profoundly warped by the state’s retention policy. This beats the daylights out of their Harry Potter theory, but there still is far less to it than meets the eye. Problems with this theory include a substantial improvement in 4th grade NAEP scores before the retention policy went into place, a substantial decline in retentions since the onset of the policy, and a substantial improvement in 3rd grade reading FCAT scores. Oh and the advent of mid-year promotions and a few other things which NEPC has been either unable or unwilling to address. The peak of any aging effect would have come in 2005 and declined substantially, and yet Florida’s scores continued to rise.
An implicit assumption of this theory was that Florida is doing far more K-3 retention than other states around the country. After seeing this NCES study, I am no longer certain this is the case, especially now that Florida retention has fallen so substantially. Let’s dig into the data and find out.
State level data on grade level retention is very difficult to come by outside of Florida. However, NCES included a question about retention in their parent survey. Low and behold, 10% of parents in the NCES survey report that their child has been retained for one or more grade in grades K-8, more than 20% of low-income parents.
NCES: Students retained in one or more grade, K-8
So first off, this is quite a bit higher than I would have suspected and the trend has been rising. Given the hostility that many College of Education Professors have towards grade retention, it seems apparent that many of the teachers and administrators that go through their programs are not buying what they are selling on retention.
Now that we have a measure of retention nationally, we should explore the question of how prevalent the practice is in Florida. The Florida Department of Education provides this handy chart for the statewide numbers for retention for students in grades K-12. The technical term to describe this chart is “falling off a cliff.”
That sounds like a lot, until you go over to the NCES Common Core Data (note to Jay, Greg and MWAB- not the academic standards, please call off the cruise missle strike :-) and learn that there were over 1.7 million students in the Florida K-8 system in 2009-10. When you do the math, it turns out that 3.9% of Florida K-8 students were retained during the 2009-2010 school year. What about the peak of Florida retention the year the 3rd grade retention policy took place in 2003-04? The total retention rate for that year was (waaaaait for it…..) 5.5%- a little more than half of the national rate that the NCES found in 2007.
We don’t have national data for K-3 retention, which is what we would need to do an ideal comparison, but the data we do have certainly establishes that there is a substantial amount of retention going on around the country, which will be having some impact on NAEP scores of states across the nation, not just Florida. Unless a state is doing far more than average, it retention is likely to be white noise overall- blips in the error term. Furthermore, it is not clear that Florida was doing more K-3 retention than the national average, even during the peak of the practice in 2003-04.
Mind you that I make no claim that retention is necessarily a good practice overall. I think there have been terrible retention practices, such as the practice of “redshirting” 9th graders in Texas back when the state gave a 10th grade exit exam. Redshirting was a widespread district level practice not mandated by state law and it was truly an awful policy basically designed to get students to drop out of school in 9th grade and thereby inflate the passing rate for the 10th grade exit exam.
There was nothing admirable about Texas redshirting. I would venture to guess that both a casual and a sophisticated analysis of data would have found it associated with higher drop out rates.
The Florida policy however is the opposite of the old Texas practice in that it is designed to set kids up to succeed rather than to fail. Not only have there been bad retention practices, there has also been a great deal of bad research done on retention that lacked the statistical rigor to establish causality. Do cancer drugs kill people, or is it the cancer? Most of the retention research doesn’t allow us to answer that sort of question.
Jay, Marcus Winters and the RAND Corp however have been applying sophisticated regression discontinuity designs to retention policies in Florida and New York City. They have found positive academic results. RAND found no self-esteem harm to students, and that NYC educators have generally positive views of the policy, to boot.
The question is not whether retention is “good” or “bad”- that all depends on how it is used. The evidence on the overall literacy effort in Florida-which includes retention as a centerpiece-is overwhelmingly positive.
10 Comments | social promotion | Tagged: Florida retention policy, grade retention, Jay Greene, Madhabi Chatterji, Marcus Winters, NEPC, New York City student retention policy, social promotion | Permalink
Posted by matthewladner
(Guest Post by Matthew Ladner)
The Florida Age of Public School Improvement hit a wall in the 2011 NAEP. This should not be terribly surprising, as Florida’s improvement seemed certain to plateau in the absence of additional reforms.
Governor Jeb Bush relentlessly pursued a dual strategy- transparency with teeth from the top down, parental choice from the bottom up. Together these reforms drove improvement in the public schools for a number of years. Accountability measures included school grading (A-F) and earned promotion in the early grades. Parental choice measures included Opportunity Scholarships for children attending F rated schools, the nation’s first special needs voucher program (McKay Scholarships), the nation’s largest scholarship tax credit program (Step Up for Students), a decent charter school law and the nation’s most robust system of digital learning. Florida lawmakers also attempted to thoughtfully incentivize success.
Governor Bush took office in 1999 and left office in 2007. It would be nice if these efforts could indefinitely push progress forward, but there have been plenty of bumps and problems along the way. In 2006, the Florida Supreme Court rendered a logic-free ruling abolishing Opportunity Scholarships (failing school vouchers) for private schools, and followed that up by ruling against a state authorizer for charter schools. Tax-credits, McKay and digital learning continued to incrementally advance, but not at an earth-shattering rate.
The larger problem may have come in the top down measures. The chart below presents the distribution of district and charter school grades, with one line being the A/B grades and the other D/F grades. The dotted lines represent instances when the state board raised school grading standards.
The setting of these standards represents far more of an art than a science. Set them far too high and disaster follows (this happened in Arizona). Set them too low, and you remove the tension in the system needed to drive improvement. Even after the last increase in grading standards, more than 10 times as many Florida schools received A/B grades as D/F grades.
Florida’s policymakers raised standards four times, and last year (wisely) put in an automatic trigger to raise standards by a preset amount when a certain ratio of schools get A or B grades. In addition, a fresh set of reforms passed the Florida legislature in 2011, revamping teaching and increasing charter school and digital learning options.
Just as it is impossible to exactly pinpoint how much of what caused the gains, it is likewise impossible to say exactly what made them stall. Note however that one of the favorite explanations of the anti-reform crowd, the pre-school, finally saw the advent of children old enough to have participated in the program and age into the 4th grade NAEP sample. I hope that someone is carefully studying variation in participation and corresponding trends in FCAT data, but the results at the aggregate level thus far seem underwhelming.
Plenty of other things, however, have been going on- including the collapse of a housing bubble, cutbacks in public school funding (including of some of the incentive funding programs) and a variety of other very bad things. My advice to Florida policymakers: roll up your sleeves and get back at it. Despite the enormous amount of progress seen on NAEP (and no one loves celebrating it more than me) too great of a gulf lies between a state system awarding ten times as many top grades as low grades but still suffering from large minorities of students scoring below basic on the NAEP exams.
Governor Bush has consistently said for years that success is never final, and reform is never finished. The 2011 pause in progress demonstrates that he called it correctly. Moving the needle on student learning on a meaningful scale and at a sustained basis represents one of the greatest public policy challenges of our times. Governor Bush has passed the torch to a new generation of Florida reformers, and they must now find new ways, and fine-tune the old ways, to push academic progress forward.
Edited for typos
4 Comments | personal tax credits/deductions, school accountability, social promotion, vouchers | Tagged: 2011 NAEP, Florida 2011 NAEP scores | Permalink
Posted by matthewladner
(Guest Post by Matthew Ladner)
Oklahoma lawmakers adopted a policy to curtail the social promotion of children failing to acquire basic literacy skills by the end of the 3rd grade. Oklahoma lawmakers adopted a special needs choice program last year, and are considering a scholarship tax credit program and other far-reaching reforms this year. Oklahoma has got major K-12 reform mojo!
The Foundation for Excellence in Education will be releasing a policy brief on retention soon. Congrats to Oklahoma’s K-12 reformers in joining Florida, New York City, Arizona and Indiana in adopting this tough-love policy. Now comes the hard part: policy implementation. The Devil is in the details on this reform, and others have botched it in the past. Yes, I’m looking at you Georgia…
I spent a few days in Oklahoma recently, and their reformers struck me as resolved, fearless and capable. That’s good, because that is what will be needed to see this policy through.
Leave a Comment » | social promotion | Tagged: 3rd grade retention, Oklahoma Social Promotion Ban | Permalink
Posted by matthewladner
Matt Ladner and I have been testifying to state legislatures around the country about the effects of Florida’s policy to end social promotion in 3rd grade. The policy default-retains all 3rd graders who score below a certain threshold on the state’s reading test. There are several exemptions to being retained, but about 59% of low-achieving 3rd graders repeated the grade.
Research that Marcus Winters and I have published in the peer-reviewed journal, Education Finance and Policy, finds significant achievement benefits for students retained under the policy. After two years the retained students outperformed their promoted counterparts by about .46 standard deviations, which is the equivalent of receiving about 6.6 additional months of reading instruction. We compared students who barely performed above the test threshold on the 3rd grade test and were default-promoted to students who performed just below the test threshold. This regression-discontinuity design approximates a random assignment experiment.
When we testify about this research we are now commonly being asked about a “study” from the Miami-Dade School District that claims to find the effect fades after two years. Clearly the opponents of the policy (read: the unions) are arming folks with this to dispute our research findings. When people oppose a policy that is supported by rigorous research it is important that they at least have a fig leaf of research to support their opposition. The Miami-Dade report is that fig leaf. The report concludes:
This study has replicated the procedures of theGreene and Winters (2006) paper evaluat ingFlorida’s test-based promotion policy and hasderived very different judgements. Where theyconcluded that the retention policy led to significant improvements in reading for the retained students,this study finds no ultimate advantages. However,it would be a mistake to interpret this study as somekind of indictment of the Greene and Winters work. Their interpretation was valid for the way the datalooked after two years. The picture is quite different after four years
First, it is important to note that the “study” is actually a 4 page document produced by the internal research department of the Miami-Dade School District. It has no descriptive statistics, no detailed description of the methodology, and virtually no literature review. In short, it is extremely hard to judge the accuracy of a “study” that is little more than two graphs that have never been published, reviewed, or fully-described.
Second, the Miami-Dade internal report only claims to analyze data from the Miami-Dade School District, while our research is based on data from the entire state of Florida. It is perfectly possible that Miami-Dade poorly implemented the policy by doing things like granting the exemptions inappropriately or failing to offer effective reading interventions for students who were retained. Even if Miami-Dade did not have successful results with the program, the entire state did.
Third, it is inaccurate to say that the Miami-Dade “study” replicated our positive findings after two years but that those positive effects later disappeared. Their graphs suggest that there was no positive effect of being retained in Miami-Dade 1 and 2 years after the retention decision, and then they show a positive effect in years 3 and 4, which disappears in year 5. We found a small positive effect after one year that grew into a larger effect after two years.
Our results (even in the first two years) are completely different from those in the Miami-Dade report. It is hard to say whether this is because they only looked at Miami-Dade while we looked at the entire state, or because they did not actually replicate our methodology. Four pages and two graphs do not allow for a lot of nuanced analysis of the findings.
We are in the process of extending our analyses to include additional years, so we may have a better idea of whether the benefits we observed state-wide grow, shrink, or remain constant. In the meantime, the unions have provided their research fig leaf to cover state legislators who oppose the policy regardless of what research finds.
2 Comments | social promotion | Tagged: Miami-Dade, social promotion | Permalink
Posted by Jay P. Greene
Jeb Bush has an op-ed in today’s Wall Street Journal that gets the new year off to the right start. Here’s a taste:
For the last decade, Florida has graded schools on a scale of A to F, based solely on standardized test scores. When we started, many complained that “labeling” a school with an F would demoralize students and do more harm than good. Instead, it energized parents and the community to demand change from the adults running the system. School leadership responded with innovation and a sense of urgency. The number of F schools has since plummeted while the number of A and B schools has quadrupled.
Another reform: Florida ended automatic, “social” promotion for third-grade students who couldn’t read. Again, the opposition to this hard-edged policy was fierce. Holding back illiterate students seemed to generate a far greater outcry than did the disturbing reality that more than 25% of students couldn’t read by the time they entered fourth grade. But today? According to Florida state reading tests, illiteracy in the third grade is down to 16%.
Rewards and consequences work. Florida schools that earn an A or improve by a letter grade are rewarded with cash—up to $100 per pupil annually. If a public school doesn’t measure up, families have an unprecedented array of other options: public school choice, charter schools, vouchers for pre-K students, virtual schools, tax-credit scholarships, and vouchers for students with disabilities.
Choice is the catalytic converter here, accelerating the benefits of other education reforms. Almost 300,000 students opt for one of these alternatives, and research from the Manhattan Institute, Cornell and Harvard shows that Florida’s public schools have improved in the face of competition provided by the many school-choice programs.
Florida’s experience busts the myth that poverty, language barriers, absent parents and broken homes explain failure in school. It is simply not true. Our experience also proves that leadership, courage and an unwavering commitment to reform—not demographics or demagoguery—will determine our destiny as a nation.
6 Comments | charter schools, competitive effects, school accountability, social promotion, special ed, vouchers | Tagged: Jeb Bush | Permalink
Posted by Jay P. Greene
You are currently browsing the archives for the social promotion category.
Jay P. Greene's Blog 