DC District Schools are Improving Fast but not Fast Enough to Catch DC Charters

January 23, 2014

(Guest Post by Matthew Ladner)

When the National Center for Education Statistics first released the 2013 NAEP data, the website refused to cooperate with requests to give charter/district comparisons for the District of Columbia.  This is of especially strong interest given that 43% of DC public school children attend charter schools.

Well lo and behold the NAEP website decided to start cooperating, and the data tells a pretty amazing story: district schools are improving over time in DC, but charters show even stronger growth.

NAEP takes new random samples of students in each testing year, but judges performance consistently across time. Making comparisons between district and charter students isn’t easy.  The percentages of students in special programs for children with disabilities and English Language Learners can potentially impact average scores. So for instance if DC charter schools have fewer children with disabilities enrolled, or fewer ELL students, or fewer low-income children enrolled, they could appear to be doing a better job educating students when the truth could be quite different.

Fortunately NAEP allows us to take these factors into account.  The charts below show NAEP data that gets as close to an “apples to apples” comparison as possible, comparing only the scores of Free and Reduced lunch eligible students in the general education program. Two other sources of bias that could be expected to work against charter schools involve new schools and newly transferred students. Organizations tend to not be at their best during their “shakedown cruise” and schools are no exception.  Also students tend to take a temporary academic hit as they adjust to a new school after transferring.  Charter schools tend to have lots of new schools full of kids who just transferred in-providing a double whammy when looking at any snapshot of performance.

Unfortunately, NAEP does not contain any tools for taking the age or the school or length of enrollment into account. Thus DC charter schools are fighting at a bit of disadvantage, and a very substantial funding disadvantage, in the below charts.

DC charter 1

DC charters may be fighting with one hand tied behind their back, but it did not stop them from scoring a knockout on NAEP. DC charters widened their advantage in the percentage of children scoring “Basic or Better” from 4 points in 2011 to 9 points in 2013.

DC Charter 2

DC district students saw a large improvement in 8th grade reading between the 2011 and 2013 NAEP, but still found themselves trailing the achievement of DC charter students by 5%. In 4th Grade math, district students scored a very large gain, but charter students achieved an even larger improvement.

DC charter 3

On 8th grade math, district students demonstrated impressive gains, but DC charter students were 19% more likely to score “Basic or Better.”

DC charter 4

Hopefully the race to excellence will continue and even accelerate. Meep! Meep!


I’ll Have What Florida Charter Schools are Having

November 7, 2013

(Guest Post by Matthew Ladner)

Florida’s charter schools totally crushed the ball on the 2013 NAEP Reading test- an 11 point gain in 4th grade reading and a 5 point gain in 8th grade reading. As the number of charter schools in the state has gone up, the ability of NAEP to reliably sample them has improved.

Getting about as close to an “apples to apples” as you can get in the NAEP data by comparing only low-income general education students still shows huge gains and a big advantage for Florida charters to Florida district schools.

Florida charter 2013 NAEP

 


DC, Tennessee and Indiana Crush the Ball on 2013 NAEP

November 7, 2013

(Guest Post by Matthew Ladner)

The National Center for Education Statistics released the 2013 NAEP this morning for 4th and 8th grade Reading and Math. I will be crunching the numbers for some time to come, but here is a quick look at the net results by state jurisdiction (combined 2013 scores minus combined 2011 scores).

2013 NAEP Gains

Quick takes: DC and TN crushed the ball with gains four times larger than the national average. These look to be truly historic gains with both DC and TN scoring statistically significantly higher in all four tests.

Indiana scores big with gains almost three times the national average. Florida gets back on track with gains more than twice the national average.

More number crunching to follow but a

BOOOOOOOOOOOOOOOOM!!!!!!!!!

for top gainers DC, TN and IN is already in order.


Some States are Serious about K-12 Reform, Others Shirley

June 19, 2013

(Guest Post by Matthew Ladner)

John Chubb and Constance Clark have a very interesting new study out from Education Sector called The New State Achievement Gap: How NCLB Waivers Could Make it Worse or Better.

Chubb and Clark examine NAEP data and find that states are diverging into leaders and laggards. In the relative blink of an eye between 2003 and 2011 they found the gap between the performance of students in the best and worst performing states grew to 60 percent of the size of the White-Black achievement gap on the combined NAEP exams (4th/8th reading and math).

Note that part of what has happened here is that the White-Black gap shrank a bit. Note however that it is still sickeningly large-keep in mind that 10 points roughly equates to a grade level worth of average progress on NAEP- so 105 points across four tests is quite disgusting. The state achievement gap meanwhile grew steadily.

Chubb and Clark’s paper would have benefitted from examination of the gory details about how some states are playing fast and loose regarding NAEP inclusion standards for special needs and English language learners- especially in the case of Maryland and Kentucky. These details do not however take away the broad point- some states are improving and some are getting left behind.

The study gets even more interesting as the authors compare the NCLB waivers, accountability systems and standards choices of states with strong and weak NAEP gain performances. Included among these is a comparison between Florida and South Carolina. The referee needs to step in and wrap up Maryland before he pummels West Virginia to death. “Self-reflection” for teacher evaluation Mountaineers? Surely you can’t be serious…

In a not-quite-elliptical fashion, Chubb and Clark note a clustering of states with a recent history of weak NAEP gains with unconvincing NCLB waiver promises and the Smarter Balanced Assessment Consortium. I’m shocked…

Chubb and Clark have turned in a very interesting piece- go read it.


Read ‘Em and Weep Edureactionaries

November 14, 2012

(Guest Post by Matthew Ladner)

There is a great deal of interesting material in the Hanushek, Peterson and Woessmann study on international/American state academic achievement. Below however is the chart from the Ed Next version that I found most interesting:

Focusing on the 4th Grade Mathematics exam between 1992 and 2009, the authors found that increasing spending does not have a strong relationship with improved student learning. Par for the course.

Take a close look at the top of the chart however in terms of the states making large gains and how much additional revenue per pupil they spent to get them.

The states showing the top gains (in order) are Maryland, Florida, Delaware and Massachusetts. MD, FL and DE essentially tie with MA slightly behind.

Notice however that the inflation adjusted spending per pupil increase in Florida between 1992 and 2009 was $1,000. In Delaware it was $3,000. Maryland looks near the midpoint between $4,000 and $5,000 so lets roughly call it $4,500. Massachusetts looks to be $5,000.

So Florida managed first class gains with a much smaller increase in funding. If I were to go and look up the numbers, we would find that Florida’s smaller increase also came from a smaller base- MD, DE and MA were all likely to have been outspending Florida in 1992 and then really outspending them in 2009.

It is also worth noting that Florida faces considerably greater demographic challenges than MD, DE or MA- far more free and reduced lunch eligible children, more ELL kids, and to the extent you want to factor race/ethnicity into the equation it is a far more diverse state with a majority-minority student population.

So conflict-adverse state policymakers with extra billions of dollars burning a hole in their pocket and very wealthy and pale complected students should study MD, DE and MA for clues on how to improve their student outcomes.

If however you live in a state with average or above student diversity, real budgetary constraints on the amount you can spend on K-12 and strong competing demands for any additional revenue you are likely to scrape up, you should study Florida. In fact you should study Florida regardless unless you lack the guts for a good tussel.

P.S. Notice that NY and WY both had gigantic spending increases (an inflation adjusted $6k per student) only to achieve average and below-average gains respectively. At least WY is just wasting money they are pumping out of the ground. NY seems intent to drive their citizens out of state. Taxpayers and especially students are the losers in both cases.


So We Meet Again Baumol…

October 12, 2012

(Guest Post by Matthew Ladner)

Part 3 of the Baumol series at the Ed Fly blog is posted, and James Shuls weighs in with an illustration of Baumol from Missouri.


A Closer Look at DC NAEP Scores

January 12, 2012

(Guest Post by Matthew Ladner)

A few months ago, I provided a quick analysis of DCPS NAEP scores under Michelle Rhee. Having looked into the fine details, I believe that I underestimated the positive trend in DCPS reading scores during the 2007-2011 period.

NAEP has long dealt with a tricky issue with varying inclusion rates for special education and English language learners between jurisdictions. In 2011, the NAEP adopted inclusion rate standards for ELL and SD students, and notified readers of jurisdictions that violated those standards in an appendix.

Some states and jurisdictions had far more successful efforts to comply with these efforts than others. As you can see from the figure below, DC would have been far out of compliance with these standards (had they been in place) during the 1990s and (especially) in 2007. In 2007, DCPS had excluded nearly three times as many students as permissible under the 2011 standards.

So in 2007, DCPS officials excluded 14% of students from 4th Grade NAEP testing, and in 2011 that figure fell to 3% (the inclusion for all students standard in 2011 was 95%). In 2007, DCPS stood far out of compliance, but came well within compliance in 2011. This is all well and fine, other than the fact that it complicates our ability to assess the recent history of DC NAEP gains.

In order to get a clearer picture on this, I decided to run 4th Grade NAEP scores for students outside of ELL or special education programs. This should minimize the impact of inclusion policy changes. Examined in this fashion, you get the following results:

Recall that the unadjusted total scores for 4th grade reading jumped from 197 in 2007 to 202 in 2009 but dropped back a point to 201 in 2011. That is a four point gain in four years, which ranks in meh territory. Given Figure 1 above, I am not exactly inclined to trust those scores, and in fact out second table tells quite a different story: general education students in DC made a 10 point gain between 2007 and 2011 on 4th grade reading. Ten points approximately equals a grade level worth of progress, so it is fair to say that DCPS general education 4th graders were reading approximately as well as 2007 general education 5th graders. Ten points ranks as the largest reading gain in the nation during this period for these students. Mind you, a 209 score for non-Ell and non-special ed students is still terribly low. Only gains will get DC out of the cellar, however, and DC banked solid gains during this period.

If you combine 4th and 8th grade reading gains for general education students, and only look at Free and Reduced lunch eligible students for a bit of socio-economic apples to apples, here is what you find:

DC students had the largest general education 4th grade reading gains in the country, and tie for first in the combined 4th and 8th grade reading gains. The District of Columbia, in short, made very substantial reading gains during the 2007-2011 period.


2011 Trial Urban NAEP-Which Districts to Avoid When Reincarnated as a Poor Child…

December 9, 2011

(Guest Post by Matthew Ladner)

…if you want to learn how to read. In the great reincarnation to come, you want to request Tampa, New York City or Miami. You are three or more times more likely to learn to read at a high level than in Fresno. DC has improved but is still horrible.

I haven’t read the appendix about the inclusion/exclusion rates but the burden of proof lies on Kentucky rather than the other way around on that Jefferson County number. Tied with Boston? Color me skeptical.

Everyone in Wisconsin ought to be horrified by the abomination that is the Milwaukee Public Schools. These awful results make me all the more grateful that kids have the possibility of choosing a charter or private school, and the results may have been even worse in the past (can’t track them very far back) but it is time for something far more drastic.

There should be no bullets left in the gun when it comes to Milwaukee. Policymakers should correct the bad joke of an accountability system the state has employed for starters. Lawmakers expanded appropriately expanded choice last year (can’t get too many lifeboats for that sinking ship) but ought to consider a different governance structure as well.

Meh results from former reform luminaries North Carolina and Texas. The low-hanging fruit has been picked off the tree.

Discuss amongst yourselves…


Greeks Bearing Gifts, Bridge Sales in Brooklyn, Confederate Currency and Kentucky NAEP Scores

November 22, 2011

(Guest Post by Matthew Ladner)

So does the data in this chart look fishy to anyone but me? What about after you read this?


Follow

Get every new post delivered to your Inbox.

Join 2,472 other followers