The Prodigal Nerd Returns to Florida

December 3, 2018

(Guest Post by Matthew Ladner)

A bit of personal news- I’ve taken a new gig at the Arizona Chamber of Commerce and Industry and as editor at RedefinED. Over at RedefinED online I offer an introductory post re-introducing myself to my friends in Florida:

Florida is a grandly innovative state with a record in policy implementation that is far greater than average. It’s very hard to appreciate this when you are close to your own inevitable policy and political messiness, but trust me, it is very clear from over here. I’m proud, for instance, that Arizona originated both scholarship tax credits and education savings accounts. Both innovations have been successfully taken to greater scale, however, in Florida – in no small part due to the relentless attention paid to the details of implementation.

You’ve taken crucial first steps towards equalizing opportunity in schooling. The sky not only did not open with a rain of frogs or locusts, you’ve seen real tangible progress. Florida public education, despite much protestation from traditionalists, is not only still there, it is substantially improved.

Funding for public education is guaranteed in the Florida Constitution and is as close to a permanent institution as you get in American society. It’s here to stay. Florida, however, has the chance not just to practice the form of public education, but to fulfill its actual promise. Much divides our society, but Americans still unite on crucial issues, including education. We desperately want an education system that gives students the knowledge, skills and habits needed for success and to responsibly exercise democratic citizenship. We – left, right and center – commonly and fiercely desire a system of schooling which serves as an engine of class mobility. Florida has moved the needle in this direction by setting families free to pursue opportunities that would otherwise be denied to them. More of this is needed and the next step will be to develop a consensus around setting educators free as well.

 

1 Comment | NAEP | Tagged: | Permalink
Posted by matthewladner

Two Minute Hate-Florida Science Edition in Hillbilly 3-D Satanovision!!!!!!!!!!

June 8, 2018

(Guest Post by Matthew Ladner)

Orlando Sentinel columnist Lauren Ritchie displays double-plus good duck-speaking two-minute hate in what has to be one of the most impressive displays of over-confident bias I’ve seen in many, many moons. In fact, this column is nominated for the Two-Minute Hate Hall of Fame! Some of Florida’s media have grown suspicious of science in instruction in Florida’s private choice program. Oh dear. Here’s a couple of excerpts:

Some of these schools — 80 percent describe themselves as “Christian” — use textbooks that claim people lived with dinosaurs. Heck, Noah had a couple in the ark. Some say God saved North America from Catholics and gave them South America instead. Others teach that slaves who “knew Christ” had “more freedom” than nonbelievers who weren’t captive. Babble. Just sheer babble.

The only reason these fringe “Christian schools” are getting away with sucking up millions in education funding is that Florida legislators are afraid of offending them. Elected types are so terrified of the instant howling about “Christians” being “persecuted” that they never seriously considered demanding the course of study in voucher schools meet the same standards taught in public school. They’re just happy to buy votes with millions in cash. Your tax dollars.

Folks, these are neither real schools nor, scholars will argue, are they Christian. They’re just money-making little engines for benighted fraudsters whose only other chance at a paying job is the Sears hardware department.

Do fundamentalists want their kids to learn a bunch of hillbilly science? Handle venomous snakes? Learn that God looks down on Catholics, that America would still have slavery except “some power-hungry individuals stirred up the people”? Knock yourself out. Just don’t expect anyone else to pay for it, and stop calling it “education.” It’s not. It’s more like a 12-year sentence to some anamorphic Sunday school class from hell with no time off for good behavior.

So, there is a problem with all of this. Despite a fog of name-calling (hillbilly from hell!) and questioning of motives (profiteering hillbillies from hell!) Ritchie does not present a whit of evidence that private school students learn less science than public school students.

Fortunately, the National Center for Education Statistics runs a project known as the National Assessment of Educational Progress (NAEP) which can shed light on this subject. Also known as the “Nation’s Report Card” this source has a huge amount of highly respected academic data, including some on Science, in private schools, among parents without college degrees.

NAEP does not provide private school science scores by individual state, but it does by region. We will therefore focus on the South, and the private school scores of students whose parents did not graduate from college. If you are going to find flat-earther profiteering red-neck science with Dinosaurs on Noah’s Ark, or whatever it is Ritchie imagines, this ought to be the place to look.

NAEP tracks four types of Science achievement: Earth Science, Physical Science, Life Science and Overall Science. The 2009 8th grade science exam allowed us to look exactly at what Ritchie denounces: students whose parents did not graduate from college, who attend private schools located in Southern states. How did these kids perform in the various NAEP Science categories compared to their public school peers?

 

You see the same pattern by the way if you compare southern scores for students whose parents did graduate from college in public and private schools in the South. Or if you do these comparisons in Western, Northeast, or Midwestern students. Across four measures of science and four regions of the country and separately for non-college and college educated parents (32 comparisons in all) private school students demonstrated a higher score a mere 32 times out of 32. Most relevant to our present discussion however are the four comparisons in the above chart, among southern students without college educated parents.

Since the author brought up Protestant schools (snake-handling!!!!) it is worth noting that only 15% of Florida tax credit students attend Catholic schools. Unlike other regions of the country, the denominational distribution in American southern states does not lend itself the private school sectors dominated by Catholic schools to the extent seen in other parts of the country. It’s also worth noting that as the state with the (by far) the nation’s largest tax credit program, the nation’s largest voucher program when these data were generated (2009) we can safely assume that Florida kids are over-represented in the Southern sample. Back in 2009, southern private choice programs were few and far between in number and small in size outside of Florida.

In recent years changes in the federal free and reduced lunch program has made it increasingly unreliable as a proxy for family income. This is all the worse among private schools, who often have difficulty accessing the program. We will therefore make use of parental education as an imperfect proxy for family income. Nevertheless, when hunting for hillbilly kids with snake-handling parents suckered by profiteering private school swindlers being taught that the Sun rotates around the Earth in science class, you would be hard-pressed to find a better place to look.

NAEP has private school scores for 8th grade Science from 2011, 2009, 2000 and 1996. The 2011 figures provide no numbers for students whose parents did not college graduate parents attending private school by region, but did provide numbers for college graduate parents in both public and private schools. Private school students had higher scores in three of four regions (one point advantage for public schools in the Northeast). The 2000 and 1996 data do not include a regional variable, but in the national comparisons both students whose parents did not graduate from college attending private schools and college grad private school parents had higher scores than their public school peers in both 2000 and 1996. So I think this basically puts the score at 39 to 1.

Comparisons like the one above do not prove that private schools are better at teaching science than public schools in any way, shape or form. They do however cast substantial doubt on the notion that they do a hugely worse job at teaching science than public schools. If private schools are running profiteering-hillbilly-from-hell science classes, what pray-tell is going on in the public schools?

1 Comment | NAEP, tax-credit scholarships | Tagged: , , | Permalink
Posted by matthewladner

Texas Charter Schools Enroll 85% Minority Students and they CRUSHED the 2017 NAEP

June 4, 2018

(Guest Post by Matthew Ladner)

The Nation’s Report Card (aka the National Assessment of Educational Progress or NAEP) released new student achievement data in April for Math and Reading exams given in 2017. Nationwide, the news was not good, and the same was generally true for scores in Texas. Figure 1 shows NAEP math and reading gains for 8th grade students since 2009 for states.

Texas 8th graders were scoring three points higher on math than 8th graders in 2009, but five points lower in reading. On these tests, 10 points is approximately equal to a grade level worth of average academic progress. Overall, Texas students failed to show much academic improvement during this period.

While Texas districts have floundered, Texas charter schools have flourished academically. Figure 2 presents the same information from states but includes the progress for Texas charter school students along with the statewide averages. At 315,200 students during the 2016-17 school year, Texas charter schools serves more students than 13 state public education systems.

The improvement in scores among Texas charter school students greatly exceed those of any state. Gains however are not the only consideration. Some states like Massachusetts failed to show large academic gains but had high scores in both 2009 and 2017. International comparisons show Massachusetts compares favorably to the top European and Asian school systems, so there is no shame in holding your mud with high scores. The next chart therefore plots 8th grade math gains (from 2009 to 2017) with overall scores (in 2017) for states and Texas charter students to take both improvement over time and overall level of achievement into account:

Texas charter students not only had higher gains than any state, they also demonstrated higher overall scores than most states. Each NAEP exam utilizes a new random sample of students, and the “sampling error” for subgroups exceeds that for states. Such sampling error however should be randomly distributed, meaning that Texas charter scores/gains could be either larger or smaller. The Reading exam however provides an entirely separate sample of students from the math exam, presented in Figure 4 below:

Texas charter students again show both larger gains than any state, and relatively impressive scores, especially when considering student demographics-leading us to our next subject.

Scores and gains this impressive naturally lead to the question of whether changes in student composition drive them. Only a random assignment study can definitively isolate the role of school quality in driving scores and gains, and such studies are impractical for statewide assessments. A Stanford University study utilizing state academic data from 2011 to 2015 using a student matching strategy found evidence of stronger academic growth for Hispanic students attending Texas charters after controlling for a variety of factors.

The demographic distribution of Texas charter schools stood at more than 85% minority in 2016-17 after having become increasingly majority-minority over the previous decade. Hispanics constituted 60 percent of Texas charter students in 2016-17. A comparison of Hispanic 8th grade math scores that accounted for both parental education and special program status (English Language Learners and Special Education) found that Hispanic students attending Texas charter schools outscored all statewide averages for Hispanic students.

This would have to be the case in order to make this happen:

NAEP can’t isolate the role of average school quality in these impressive scores. When however an 85% minority school sector out scores Vermont I’m cautiously optimistic that school quality had something to do with it.  Texas adds ~90,000 new K-12 students per year and the districts seem to be struggling both academically and financially in keeping up with the growth. Expanded opportunities for families to choose the sort of education to suit their needs and aspirations could help address both concerns.

2 Comments | charter schools, NAEP, Tejas Uber Alles | Tagged: , , , | Permalink
Posted by matthewladner

AZ and TX Charter Sectors Tops in 2017 NAEP 8th Grade Math for Hispanic Students

May 22, 2018

(Guest Post by Matthew Ladner)

“The Force is with you young Charter Tejanos but you are not a mathematics Jedi yet. We would be honored if you would join us in CeleNAEPing good times!”

-Darth Cactus

3 Comments | charter schools, NAEP | Tagged: | Permalink
Posted by matthewladner

DCPS: Still the Achievement Gap Capital of the 2017 NAEP

May 8, 2018

(Guest Post by Matthew Ladner)

Look ma! DCPS reform is working great as long as your parents graduated from college. The below charts are district schools only:


That’s a two point decline compared to a 17 point (!) improvement for those scoring at home. What about math?

That is a 10 point improvement for students whose parents did not finish college compared to a 25 point (!) improvement for students whose parents did finish college. You see a similar pattern if you break the numbers down by race/ethnicity. Here is how DCPS looks (absent charter students) compared to the TUDA districts (also district only) for Black students in NAEP 8th grade reading:

Here is how it looks for Hispanic students:

And…here is how it looks for White students:

6 Comments | NAEP | Tagged: | Permalink
Posted by matthewladner

Arizona Charters: I’m Not Left Handed Either

April 18, 2018

(Guest Post by Matthew Ladner)

So Arizona charter schools rocked the 2017 NAEP again. In 8th grade math, Arizona charter students narrowly scored below the top scoring state (Massachusetts) in 2015 and above them in 2017. Arizona charter schools educate a majority minority student body, both overall and specifically in 2017’s 8th grade class. It’s really quite extraordinary to see them in the same academic neighborhood as MA, given their majority-Anglo, high income, spend twice as much per pupil combo status. If MA is a good sport, they would fight left handed- what chance does a majority-minority school system with half the spending per pupil have against the highest performing state education system in the nation many years running?

I decided to dig into the details.

The Free and Reduced Price lunch definition has become fairly sketchy, so in the below comparisons I will make use of parental education as a proxy for socio-economic status. Another source for variation between states and sectors involves special program students, so the below comparisons will focus on general education students (neither ELL nor SPED). The first set of scores are for students whose parents did not finish college and are in the general education program:

Massachusetts is still winning the duel, but not comfortably-switch to the right hand? Note for the record that 10 out of 10 of the top performing states have a majority Anglo student population. In fact you don’t spot a majority minority student population state until Texas pulls in at #19. Did I mention before that Arizona charter schools are majority minority? Oh, yes, well good that again then.

Now let’s run the same numbers for general education students with college graduate parents. This should be the MA right handed fighting btw- far more fancy degrees in Massachusetts than Arizona after all, more graduate degrees as well. Well, Arizona charter kids are not left-handed either:

 

14 Comments | charter schools, NAEP | Tagged: | Permalink
Posted by matthewladner

I Say the Future is Ours if You Can Count

April 13, 2018

(Guest Post by Matthew Ladner)

Yesterday we went over how states have done very little to improve in 8th grade math since 2009, but how many state charter sectors rocked it. Let’s see about reading, starting with the states:

There are more declines in scores than statistically significant gains in that first chart-some states more lost than others, but a lost decade nationally. Next let’s look at statewide charter sectors-these are the states with charter sectors large enough to make the sample in both 2009 and 2017 in 8th grade reading:


Once again, just as with math, Excel had to move the growth axis scale for several state charter sectors. You will appreciate this better when the following chart combines statewide averages and state charter sector averages:

So in this chart ideally you would like to see high scores and high gains, but there is no shame in just very high scores.

Can you dig it?

Can you dig it?!?

CAN YOU DIG IT?!?!?!?!

Leave a Comment » | charter schools, NAEP | Tagged: , | Permalink
Posted by matthewladner

John Rawls Rolls in his Grave Upon Viewing NAEP Trends for IEP Students

April 12, 2018

(Guest Post by Matthew Ladner)

…at least for most states.

2 Comments | NAEP | Tagged: , | Permalink
Posted by matthewladner

Charters CeleNAEP Good Times During “Lost Decade”

April 11, 2018

(Guest Post by Matthew Ladner)

The news overall is grim. You want to be in the top right quadrant of this chart. Some states thankfully did land there-including the state with the largest student population- but many only barely due to small reading gains. If you need the dot size to push you in, it doesn’t count-better luck next NAEP.

You don’t want to land in the top left or in the bottom right, and you most of all don’t want to see your state in the bottom left quadrant (declines in both subjects). Mike Petrilli used the phrase “lost decade” to describe the results. Some states seem far more lost than others, but it is hard to find fault with that assessment overall.

The two main reform strategies employed since the 1990s have involved test-based accountability and increased parental choice. During the era covered by the top chart, the test-based folks swung for the fences by creating a federal incentive for states to adopt a preferred set of academic standards and to pass statewide teacher evaluation systems based upon the scores on those tests. Gigantic investments of political and financial capital supported these policy changes, but it is hard to characterize the results as much more than disappointing.

Now some of you will be thinking around about now “oh yeah but we’ve expanded choice during this period as well!” That is true, and while we have numerous studies establishing positive competitive effects on district schools from choice programs, few states have choice programs going at a scale to place a large amount of pressure on district enrolments. NAEP does however allow us to track state charter sector gains over time. Sixteen state charter sectors had scores for 8th grade math and reading in both 2009 and 2017, allowing the following calculation:

Excel had to change the scale of the axis for the above chart. You may not have noticed. Putting state averages and state charter sector averages into the same chart will help:

Suddenly those statewide gains in Arizona, California and Mississippi (i.e. the good ones) from the first chart don’t seem so impressive eh? I’m thinking out loud here and inviting you along for the ride. Gains aren’t everything, so the next iteration will include achievement and gains by subject area, but for the huge gainer sectors (spoiler alert) they didn’t get that way with low 2017 scores. I could go on about standard errors being bigger for charter sectors and whatnot, but who are you going to believe a boring statistics lecture or your own lying eyes? If someone can explain why random error would systematically dramatically favor charter sectors, I’m all ears and the comment section eagerly awaits your thoughtful challenge.

In fact there is a white lie in the above chart-some of the sectors and states that look meh in this chart immediately above had very high scores in both 2009 and 2017, and while not ideal there is no crime in holding your mud with high scores. Last year my friend Robert Pondiscio convinced me that combining achievement and gains to provide a clearer picture, so here goes for 8th grade math:

In this chart you either want to have large gains, or high scores, or preferably both of these things. I’m for instance not inclined to criticize Idaho charter schools for modest gains given that they outscore Massachusetts and all despite spending about half the amount per pupil.

Well…yup it is officially time:

 

5 Comments | charter schools, NAEP | Tagged: , | Permalink
Posted by matthewladner

Arizona Charters Crushed the Ball Again But They Have Competition Out West

April 10, 2018

(Guest Post by Matthew Ladner)

Arizona charter schools continued to display impressive scores and gains on the 2017 NAEP. Note as always that only a random assignment study properly conducted could hope to isolate the role of school quality in all of this, and that such studies are not only unavailable but also are impractical for a statewide system of schools- not all of which are oversubscribed (a precondition for a random assignment study). Thus the role of average school quality in driving the above results remains a holy mystery- likely to be hotly debated, impossible to be resolved in this world. I and others will be digging into Arizona charter subgroup scores in the days ahead.

Now, behold the nothing burger that has been net American academic progress since 2009 (first set of columns). On the math and reading tests 10 points approximately equals a grade level worth of average academic progress (the science exams have a different scaling). Net American progress equals nothing, nothing, next to nothing a tiny bit of something in 8th grade reading.

The middle blue columns are the statewide numbers for Arizona. As you can see they consistently surpass the American nothing-burger. The final set of gains are the gains for Arizona charter school students.

Some of Arizona’s neighbors also have charter sectors that crushed the ball- starting again with Colorado. I’m happy to say that Nevada’s sector made the minimum reporting requirements for NAEP this year, and the results look good. Give me some time to dig, but a monster story may be California charters:

 

2 Comments | NAEP | Tagged: , | Permalink
Posted by matthewladner

« Previous Entries