The 2016 AZ Merit: Improving but Meh versus Mehssachusetts

September 19, 2016

(Guest Post by Matthew Ladner)

The Arizona Department of Education released 2016 AZMerit data last week, and charter school students show an across the board advantage.

az-merit-2016

Breaking down the data by subgroups consistently show charter advantages as well. Let’s start with Anglos:

azmerit-anglo

Move on to Hispanic students:

az-merit-hispanic

African-American students:

azmerit-black

 

Native American students:

azmerit-native-american

Asian students:

azmerit-asian

Special education students (btw the percentages of special education students among district and charter schools were roughly equivalent among tested students):

azmerit-sped

Economically disadvantaged students, but with an *.  A high percentage of Arizona charter schools do not participate in the federal free and reduced lunch program, and about 85% of alternative (dropout recovery) schools in Arizona are charter schools. Having said that, charter schools score higher again:

amerit-ecodis

This next chart required me to dust off my algebra skills and use the existing data to solve for X:

azmerit-non-disadvantaged

What we can take from this: while differences in student populations explains some of the differences between overall charter and district scores, when the charters lead in each and every subgroup it does not explain anything close to all of the difference. Every difference across every subgroup counts, and in the end the add up to:

az-ma

For those of you squinting at your Ipad, that is the statewide averages for Massachusetts (the highest scoring state) on the left, Arizona charters in the middle, and Arizona Districts.  MA of course is much wealthier and spends a great deal more than AZ, but when you break down the subgroups Arizona charter students outscore like students from Massachusetts, which makes me want to CeleNAEP good times here in the cactus patch:


Scenes from the Great Education Stagnation

October 2, 2015

(Guest Post by Matthew Ladner)

If a picture is worth a thousand words, here are 3,000 for you. First American manufacturing does more with fewer people (HT: AEI’s Mark J. Perry):

Okay good- ready for the next one? Heritage chart showing that the American education massively increases employment relative to the student count:

 

But it’s all fine because the kids are learning so much surrounded by so many adults compared to the past right? Er, no:

 

 


District or Charter Schools in the District of Columbia?

May 18, 2012

(Guest Post by Matthew Ladner)

DC’s NAEP numbers allowed for some additional controls to be introduced when comparing charter and district schools than I was able to do with the Milwaukee comparison. The following chart shows the percentage of general education program students who qualify for a free or reduced lunch scoring “Basic or Better” on the 2011 NAEP exams. Special education students, ELL students and middle/high income students are not included in order to get a quick closer to apples to apples comparison.

Now of course for a real apples to apples you need a random assignment study, but those have been done and find results favorable to charter schools. This chart doesn’t address the topic of valid stastical significance, but rather whether the differences are meaningful.

Considering that charters get far less money that DCPS per pupil and show higher levels of academic achievement, this looks to be a success, albeit both the blue and the red columns leave much to be desired. The red columns leave much more to desired however, especially when you consider that that they are wallowing in money.


Why are we having this fight again?

June 16, 2010

(Guest Post by Matthew Ladner)

Could the adoption of common core standards lead to substantial academic gains, even if somehow developed and kept at a high level in some imaginary Federal Reserve type fortress of political solitude and kept safe from the great national dummy down?

I ran NAEP numbers for all 50 states and the District of Columbia and calculated the total gains on the main NAEP exams (4th and 8th grade Reading and Math) for the period that all states have been taking NAEP (2003-2009). In order to minimize educational and socio-economic differences, I compared the scores of non-special program (ELL, IEP) children eligible for a free or reduced price lunch.

I then ranked those 50 states, and the table below presents the Top 10, along with the total grades by year for the strength of state proficiency standards as measured by Paul Peterson. Peterson judges state assessments by comparing scores on the state exam to those on NAEP.

To my eyes, it looks as though either nothing or next to nothing is going on here. The top three performers (FL, DC and PA) have unremarkable standards vis a vis NAEP.  Russ Whitehurst has written that some commercially available curriculum packages have shown good results in random assignment studies.

Jolly good- I suggest states adopt them rather than this politically naive common core standards effort.

NAEP Gains in 4h and 8th Grade Math and Reading for FRL, Non-IEP, Non-ELL students, 2003-09 for the Top 10 states (FL=1, NY = 10) compared to State Standards Grades by Peterson and Lastron-Adanon
2003 2005 2007 2009
FL C C C+ C

DC

C C
PA C C C C
MA A A A A
VT B- B B+
Hawaii B B+ B+ A
Md C+ C C D+
NV C C C
NJ C C C B
NY C C C+ D

NAEP Gains in 4th and 8th Grade Math and Reading for FRL, Non-IEP, Non-ELL students, 2003-09 for the Top 10 states (FL=1, NY = 10) compared to State Standards Grades by Peterson and Lastron-Adanon
2003 2005 2007 2009
FL C C C+ C

DC

C C
PA C C C C
MA A A A A
VT B- B B+
Hawaii B B+ B+ A
Md C+ C C D+
NV C C C
NJ C C C B
NY C C C+ D

2009 NAEP Reading Scores Released Tomorrow

March 23, 2010

(Guest Post by Matthew Ladner)

Tune in here tomorrow for news and analysis.


PJM on NAEP and NCLB

May 9, 2009

(Guest post by Greg Forster)

Today Pajamas Media runs my column on the latest long-term trend NAEP results and what they say to critics and supporters of NCLB:

The good news for the critics is that the Nation’s Report Card shows reading and math scores still have not substantially changed since 1971.

The good news for supporters is that the Nation’s Report Card shows reading and math scores still have not substantially changed since 1971.

Welcome to the confusing world of education policy!


The Rawlsian Path Out

October 15, 2008

Perhaps children will read better by the light of my Broader/Bolder money bonfire…

(Guest Post by Matthew Ladner)

John Rawls’ influential work A Theory of Justice argued that societal ethics ought to be decided as if we were behind a theoretical “veil of ignorance.”

Behind the veil, no one would be aware of what his or her position would be in a forthcoming society. You would not know whether you would grow up the child of a billionaire or poor in the inner city. The veil creates an incentive to leave a path out of the latter scenario.

Matthew Miller’s reading of Rawls calls for the aim of public policy to be the creation equality of opportunity, rather than equality of condition. But how much of a path do public schools actually represent for disadvantaged children?

Gauging opportunity is tricky, given that we really only have information on the equality of condition. Let’s be brave and speculate. The percentage of free or reduced price lunch students scoring “Advanced” on the 4th grade NAEP reading test in 2007 was — drumroll – two.

That’s right. Two percent- two point zero. While this was double the one point zero from 1998, it is hardly satisfying.

Mind you, total per pupil spending as of 2004-5 was $10,725. So, a child that has been in the average public school in this country since kindergarten will have had somewhere in the neighborhood of $55,000 spent on their education by the time they reach 4th grade.

Might we not get two percent scoring advanced in the complete absence of a public school system? If we spend $55K per kid to improve their opportunities, shouldn’t we be doing a lot better than 2%?

Some of course will grumble that the Advanced level on NAEP is simply too high a bar to expect an economically disadvantaged child to clear. So, let’s go down a level- 15% of free and reduced lunch kids score “Proficient” on 4th grade reading. Add in your two scoring Advanced, and you have 17% of low-income children reading at a high level.

In return for the taxpayer’s $55,000, we get 83% of our low-income children reading at less than a high level. Half of these children score “Below Basic.” Here is NAEP’s definition of Basic in 4th Grade Reading:

Fourth-grade students performing at the Basic level should demonstrate an understanding of the overall meaning of what they read. When reading text appropriate for fourth-graders, they should be able to make relatively obvious connections between the text and their own experiences and extend the ideas in the text by making simple inferences.

That doesn’t sound like a great deal to ask for $55,000 and five years of schooling- but half of the nation’s low-income children didn’t get there in 2007.

Now, we actually have NAEP data for one urban school district- Washington DC. DCPS shells out over $20,000 per year per student. So, a 4th grader has “benefited from” more than $100,000 of funding. Or, at least someone has benefited. DCPS had exactly zero point zero low-income students scoring at the advanced level in 2007. Well, not precisely zero- the NAEP footnotes note that the number “rounds down to zero.”

Six percent of DCPS low-income 4th graders scored Proficient or better, 29% at Basic or better, and 71% at Below Basic.

Regardless of your philosophical viewpoint, these results simply cannot be credibly defended. When the Texas Rangers overpaid for Alex Rodriquez and found themselves in last place, the General Manager noted that he could be in last place with the lowest rather than a sky-high payroll. DCPS is in the same boat. Give me half of the DCPS budget as an incentive program for kids to read books at the library, and I’ll get you more than zero kids scoring as advanced readers.

Education is the key to social mobility. Those who believe in Rawlsian ideals will be facing the reality that they can either have an alliance with the defenders of the status-quo, or they can have an effective K-12 path out of poverty, but they cannot have both.


How Well Aligned Is Kazakhstan to NAEP Standards?

June 25, 2008

(Guest Post by Matthew Ladner)

Recently I appeared on the Horizon public affairs program together with Arizona Superintendent of Public Instruction Tom Horne, to discuss the No Child Left Behind law and our state AIMS test. Superintendent Horne and I have a public disagreement about the relative reliability of NAEP compared to that of the state’s own version of the Terra Nova exam. NAEP finds Arizona consistently below the national average in all subjects and grade levels, while the state’s Terra Nova finds us above the national average in all subjects tested and grade levels. One of these sets of finds is much more consistent with the socio-economic profile of the K-12 population than the other, given that Arizona ranked second to the bottom on Jay and Greg’s Teachability Index study.

During the discussion, Superintendent Horne said the main reason Arizona students perform poorly on the national NAEP test, also known as the Nation’s Report Card, is due to a non-alignment of standards. If, for example, Arizona does not teach the math concepts in fourth grade that appear on the fourth grade math NAEP, one could expect lower average grades.

The explanation seems quite plausible, and doubtlessly there are some states that have better aligned their standards to NAEP than others. But how big a deal is this, in terms of Arizona’s performance? A study by the American Institute of Research shows probably not much.

The study compared international science scores for eighth graders to eighth grade NAEP science scores through an equating proceedure. Singapore came in first, with 55 percent of students ranked as “proficient” or above. Massachusetts was the highest-performing U.S. state, with 41 percent proficient. Just 20 percent of Arizona eighth grades ranked proficient.

Alignment error ought to be much greater between nations than between American states. Perhaps, for example, Norway chooses not to teach science until 9th grade. One would be hard pressed to buy into the notion that countries such as Singapore, Korea, Estonia, Hungary, and Slovakia simply have national standards more closely aligned to the American NAEP test than Arizona.

When we get clobbered in science proficiency by countries like Estonia, we have problems that go much deeper than standards alignment. I could start looking up GDP per capita in Estonia, but that would be cruel. We need to be willing to think outside the box and figure out what other countries are doing right.

Nation (or State) 8th Grade Science Scores

Percent Scoring “Proficient” or Above

Nation (or State) 8th Grade Science Scores

Percent Scoring “Proficient” or Above

Singapore

55

Lithuania

25

Taipei

52

Slovenia

24

South Korea

45

Russia

24

Hong Kong

44

Scotland

24

Japan

42

Belgium

22

Estonia

41

Latvia

21

Massachusetts

41

Malaysia

20

England

38

Arizona

20

Hungary

38

Israel

18

Netherlands

31

Bulgaria

17

Australia

30

Italy

17

Sweden

28

Norway

15

New Zealand

26

Romania

14

Slovakia

26

Serbia

12


%d bloggers like this: