(Guest Post by Matthew Ladner)
I mentioned a couple of weeks ago that the 2011 NAEP would be the first real book on Michelle Rhee’s tenure running DCPS. The 2009 NAEP was a little early, and the 2013 numbers and those going forward will be owned increasingly by those in charge after Rhee, for better or worse.
So this morning I tried to devise a rough and ready analysis that would be informative (if certainly not definitive) and that I could run before making breakfast for the kids (Mrs. Ladner is off on a well-deserved vacation, daddy is gasping for air).
Here is what I came up with: Rhee took command in 2007, so I use the 2007 NAEP scores as the baseline. We all know the level of academic achievement is terrible in DC, but it was when Rhee got there as well, so I decided to focus on growth in scores.
Finally, DC has experienced a good amount of gentrification in recent years, so I chose to focus on the growth of free and reduced lunch eligible children on all four main NAEP exams (4R, 4M, 8R, 8M) for the 2007-2011 period.
Here’s what came out:
That’s pretty close to the top. Rhee’s critics will be quick to note that DC’s gains between 2003 and 2007 were also large. We of course can never know the counterfactual DC’s scores may have been due for a stall, or they may have kept up the same pace whether Rhee had shaken the District up or not. We’ll never know.
The most important point is: DC scores are still a disaster despite the large gains before, during and after Rhee. Rhee has moved on, but the rotten scores are still there.
DC policymakers, in my opinion, should now look to take a deeper dive on reform. Why does the District’s budget continue to swell when the enrollment continues to shrink? If money were allowed to truly follow the child, you’d see an even more robust charter school movement in the District.
When will the District finally clean up the special education disaster? Many blame it on the lawyers, but go and look at the scores in the post below: these guys are shooting fish in a barrel. Special needs vouchers could play an important role in a comprehensive plan to clean up the special needs mess in DC (no litigation, no ultra-Cadillac placements).
While the needle is moving in the right direction in DC, I believe that the Cool Kids came out of the experience sadder, wiser and undeterred. That’s for the best.
Jay P. Greene's Blog 
Matt – you don’t have to be a Rhee critic to note that the scores were rising (and at a faster clip) before she came, you just need to be an objective observer looking at the statistics.
Given that reading scores, which were rising slowly, are now flat, it suggests that reform has not had a positive effect in DC. We may never know if reading scores would have progressed without reform, but we do know they have stalled with reform. This says nothing positive for reform.
We will never know if scores would have progressed faster if reform had not come, replacing so many experienced teachers and principals with inexperienced ones. But if scores had increased significantly, I bet some people would be quick to attribute it to replacing deadwood with enthusiastic newcomers.
NAEP also reports that the achievement gap widened in DC. We will never never know if it would have widened the same, less or more deeply with reform, but we do know reform didn’t help and we know that reformers in DC were dedicated to narrowing the gap, so we could at least say they failed at their mission. They were very vocal about the gap when reform started in DC. Now they are very quiet.
By the way, the scores you note above include DCPS and DC charters which have always been very similar (with a slight pulling ahead for charters this year in the DC-CAS), so it seems like a stretch to say building up charters will help student achievement.
Efavorite-
I wouldn’t rush to draw too many conclusions from merely the 2009 to 2011 numbers. There has been reading progress since 2007, with much more needed.
On the achievement gap front, you are right, but one must be terribly careful. The socio-demographics of DC are changing, and there isn’t much that reformers or anyone else can do about children with highly educated parents moving into the District to take high paying jobs with the federal government. Reformers and DC educators alike should be focused on securing gains for low-performing students and let the gap stuff work itself out.
On your point about charter achievement, very careful study is needed to tease out the impacts of the charters themselves from the impact of transfering schools (negative but temporary) and of attending a startup school still seeking their sea legs (also negative but temporary) in addition to a variety of other factors.
The overall evidence is also strong however that charter schools have a positive impact on nearby district school achievement. In the District, I would dare to guess that this would be far greater in DC if the Congress would stop treating DCPS funding as some sort of entitlement and instead give a reasonable per student funding amount to both charter and district schools.
Sounds like you are making assumptions about the effects of gentrification rather than looking at the stats on the gap.
Many of the gentrifiers are very young, some not married and/or without children old enough for school. They are affecting the black/white population ratio but not achievement gap.
Besides, the numbers of kids don’t make a difference, it’s the disparity in scores that counts and that shouldn’t vary based on a changing # white kids, because the gentrifiers are of the same high SES as those currently here. There are black and mixed race gentrifiers too and if anything, they would narrow the gap, because they are high SES too
meant to say: e will never never know if it would have widened the same, less or more deeply withOUT reform,
IF Rhee really cared about the quality of education, why didn’t she get rid of the lousy Everyday Math in the district?? She was there to break the union and took down good teachers in the process. I despise the Commies at the NEA however we know that the problems in education go WAY beyond the lefties at the NEA. She didn’t address the lousy curriculum forced upon the teachers. We know teachers are graduating with Ed. Degrees that are a joke.
There are GOOD teachers out there trying to do a good job in the classroom but have become the target of Progressive educators because they wont buy into the fads forced upon them: Constructivism, group learning, project based learning, teachers becoming facilitators instead of instructors, etc. That’s the ultimate goal of the National take-over in education. The good traditional teachers will also be eliminated and everyone should be concerned.
Keep in mind that Rhee didn’t come to DC until July of 2007. Any improvement in the 2007 NAEP scores could not have been the results of reform. You should revise your chart based on that. We’ll never know what caused improvement then, but some people think it could have had something to do with the previous superintendent.
When I first started looking closely at DCPS stats, I noticed a leap in the NAEP scores gains between 2003 and 2005. I checked and found that DCPS had four superintendents during that time- including two actings. [Paul Vance (July 2000 to Dec. 2003) Elfreda Massie (acting, Jan-April 2004) Robert Rice (acting, April-Sept. 2004) Clifford Janey (Sept. 2004 to June 2007).
http://voices.washingtonpost.com/dcschools/2010/07/rhees_staying_power.html
No one then or now attributed those gains to superintendent turnover – that wouldn’t make sense. But I know of no effort to find out what was causing the increase in the midst of the turmoil at the top.
Nobody seemed to care about what made the scores go up until Rhee came along and promised huge gains with her reform efforts. The gains haven’t materialized and now you suggest we should move on. Are there no lessons learned?
What about the kids? Do reformers still care about them?
Please show me how you’ve determined what the overall evidence concludes about “nearby” district school achievement and Charters, and keep in mind that the charters and DCPS are in the same district.
Efav-
I used 2007 as the baseline year as there was approximately a zero percent chance that Rhee had much to do with the scores that year. The chart in the post measures only the gain from 07 to 11.
Jay include a link to a meta-analysis of charter school research that a few posts ago. That would be a good place to start if you are interested.
I’m confused – if you’re trying to check the Rhee effect, why start with a year that she had no effect on? Besides, in your post above, you said you chose 2007 because that was the year she took command. In your previous post you said “the 2011 NAEP will constitute the first plausible check on the tenure of Michelle Rhee” and “Assuming normal lags between changes and impacts, I believe that the 2009 NAEP arrived a bit early. I’ll be very interested to see what happens with the 2011 scores.”
So choosing 2007 as a start seems odd, except that scores were higher then, so including them, even though Rhee had nothing to do with them, makes her tenure in DC look better.
It’s really very simple Efave- 2007 was the last “pre-Rhee” NAEP.
The 2007 NAEP scores measure 2005 to 2007, before Rhee arrived in July of 2007.
If you wanted to demonstrate the effects of a new manager of a baseball team, you wouldn’t start measuring using the stats from the last year of his predecessor.
If you wanted to know the GPA of your son’s high school while he was there, you wouldn’t include the year before he arrived.
If you wanted to know the average temperature of your town in July, you wouldn’t include the month of June.
I hope you see the logic of this and are curious enough to redo your chart not using years before reform started in DC.
On the contrary, if you want to know the impact of a new manager in a team, its performance in the year before he started is precisely the baseline you’d use. If I tell you the team went 8-4 in the new manager’s first year, you can’t say whether the new manager made things better or worse unless you know whether 8-4 is better or worse than the previous year’s record.
Efavorite, look up “baseline”
Thanks, I think I got it now.
It would be interesting to see the chart above using 2003-2007 figures. You mentioned the gains then in DC were large. How large and how do they compare to reform gains?
Also, I wonder how the charts would look if you broke them out by grade and subject.
[…] conservative reformer Matthew Ladner once acknowledged “that the 2011 NAEP would be the first real book on Michelle Rhee’s […]