The Rising Tide in the Desert

February 21, 2018

(Guest Post by Matthew Ladner)


Too much of Arizona’s K-12 debate focuses on inputs, too little on outputs. Some districts have been gaining enrollment, others losing enrollment, but this is an entirely secondary concern when compared to the question of whether an increasing number of Arizona students are acquiring the knowledge, skills and habits for success in life.

The above chart from the Center for Student Achievement shows enrollment trends for districts and charters in a number of Arizona districts. Some districts gained students despite the rise of charters (Chandler, Higley, Queen Creek) while others lost enrollment. All of the above Arizona district/charter combos did well to spectacular in Stanford’s Sean F. Reardon’s measurement of academic growth, with the lone exception of Coolidge Unified.

The Scottsdale district/charter combo came in at the 64th percentile, Tucson at the 67th, Queen Creek at the 68th percentile, Deer Valley at the 83rd, Roosevelt at the 89th. All three of the truly spectacular scores (Higley, Chandler and Phoenix Elementary combos at 95, 95 and 99th percentile respectively) came from situations where both the district and charter sectors grew rapidly. Congratulations to the students and educators of these communities are richly deserved.

The growth party did not stop in 2015.  Here are the ELA proficiency trends in AZMerit for all of these districts:

and here is the same chart for math:

Some of these gains are large (see Queen Creek and Scottsdale) others incremental, but every single one of them is moving in the right direction.

Wait- my telepathic powers are picking something up. You were thinking “Ladner are you really going to celebrate Roosevelt going from 17% proficient to 23%?

I’m glad you asked.

Two things- first the AZMerit academic bar is high, and second most of the rest of the country seems mired in academic stagnation. Of course I’m not satisfied with 23% proficiency (#NoAZwe’vegottoWinMOAAARRR!) but I am indeed happy that both low performing and high performing districts show improvement.

Experience is a harsh mistress, and one of the things she teaches the policy analyst is to never rely solely on state test scores. NAEP will release 2017 scores in a few weeks. Let’s see what happens next. In the meantime, the freedom for families to choose between schools and the opportunity for Arizona educators to create new schools according to their vision of excellence seems to be broadly working.


District/Charter Combos that Outperform RSD

February 19, 2018

(Guest Post by Matthew Ladner)

Just out of curiosity, I decided to look into the Reardon data to see how many Arizona district/charter combos outperformed or tied the Recovery School District in academic growth. The above list is by no means exhaustive, more the product of throwing in some district names into the data base before my morning caffeine. The list is not short, and includes several very poor and isolated school districts.

Mind you that no one bombed any of these districts with millions in philanthropy. You can’t go to a bar in Snowflake Arizona and meet a group of six Teach for America students the way you can in the French Quarter. They also managed this somehow without a “harbor-master.” Finally, the Reardon data ends in 2015, and since then the trend in statewide data in Arizona has been positive, but in the RSD not so much. Hopefully Reardon will update his study so we can track this over time.

I want to be careful to note that I regard the Recovery School District to have been a very clever innovation for a district that had almost nothing to leverage but empty school buildings after a hurricane. If that hurricane however had leveled Houston or Dade County I’m afraid that the limited supply of philanthropic dollars and TFA teachers would have been unequal to the (much larger) task. In order to reach scale, we are going to need solutions that do without substantial infusions of outside money, as that is likely to be in increasing short supply.

Having said that, RSD landing in the 92nd percentile in growth in the Reardon data was truly a magnificent accomplishment. The leap however from “well done” to “everyone needs to do this now!!!!” looks very dangerous imo.


Tuscon versus Columbus: Round Two

February 16, 2018

(Guest Post by Matthew Ladner)

Yesterday I presented statewide NAEP information contrasting urban schooling achievement trends in Arizona and Ohio, and specifically in Tucson and Columbus. Columbus is surrounded by suburban districts choosing not to participate in open enrollment (typical I fear) while Tucson is surrounded by suburban districts who do participate in open enrollment-and actively so.

Today I remembered the cool data tool that the NYT developed using Sean F. Reardon’s data.

Let me start by saying that if I had to pick a district to showcase Arizona, it would not be Tucson. While I am fully aware of some outstanding schools in TUSD, the district’s reputation (fairly or not-I am no authority on the subject) usually involves enrollment decline, empty school buildings, union sway in school board elections and controversy over some sort of voluntary “La Raza” curriculum in the high schools. A decade ago you could peer into the state’s AIMS data and watch student cohorts fall further behind as they “progressed” through the system.

Arizona however has been leading the nation in academic gains, and Tucson continues to face steady and considerable competition for students not only from charter schools and private choice programs, but also from nearby suburban districts. It is my contention that this broad competition enables the bottom up accountability that results in Arizona’s average charter school closing after only 4 years despite receiving 15 year charters from the state. Reardon’s data includes both district and charter school trends, but how did Tucson fare between 2010 (3rd grade scores) and 2015 (8th grade scores) in terms of academic growth?

Tucson Unified (and charters operating within district boundaries) scored at the 64th percentile for growth during this period. Columbus Ohio meanwhile also had a charter school law active, but no suburban districts willing to allow transfers, per the Fordham map:

How did Columbus fare in the Reardon data?

Columbus scored in the 22nd percentile in academic growth during this period. The news is also grim in Cleveland, Toledo and Dayton although Cincinnati stands out as the Ohio urban progress champion during this period. Overall however things look like in NAEP for the two states.

Now if you want to see something really cool:

The east-west on these columns indicate the relative wealth of the district, and Phoenix Elementary and charters sit at the tip of the gains spear.

Gender Gaps in College STEM Education: Boys Tend to be Over-Confident in Math and Benefit from It

February 15, 2018

Related image

(Guest Post by Gema Zamarro & Lina M. Anaya)

Employment in the so-called STEM (Science, Technology, Engineering and Mathematics) fields is projected to continue growing according to the Bureau of Labor Statistics. Additionally, wages in STEM field occupations are estimated to be on average nearly double the national average of wages for non-STEM jobs. Despite this promising future, women continue to be under-represented in STEM. Women are less likely to enroll in STEM degrees in college and represent a smaller share of STEM occupations. The question is why? Only after understanding the possible sources of such gender gaps we can have an idea of what can be done about it.

This question has haunted me (Gema) since my daughter, then a kindergartener, came home one day saying a boy in her class told her “girls are not good in math.”  Indeed, researchers have pointed out at gender differences in math performance and math perceived ability as possible drivers of later gender gaps in STEM. I wondered if parents could somehow counter these effects. After all, my previous work indicated that parental occupation type could be important for women’s long term STEM outcomes. In a recent working paper, I partnered with Lina M. Anaya, a Ph.D. student at the Department of Education Reform at the University of Arkansas, and Frank Stafford, Economics professor at the University of Michigan, to try and shed some light on these questions, using data from the Panel Study of Income Dynamics (PSID).

Using information from the multiple supplements of the PSID, we measured gender gaps in performance  on the standardized Woodcock  Johnson Applied problems test (W-J AP) and self-reported perceived math ability, measured on children from PSID families when they were between 6 and 17 years old (on average around 11 years old). Then, we are able to track these children and study their likelihood of majoring in a STEM field in college. We found that boys are more confident in their abilities than is warranted by their performance, while girls are less confident than is warranted by their performance.  But the problem isn’t just one of lack of confidence — boys’ confidence contributes more to their pursuit of STEM majors than girls’ confidence, even if they had the same true ability and same level of confidence.

Our results corroborated significant gender differences in W-J AP test performance and in perceived math ability during childhood. Even after conditioning in a given level of math performance in the W-J AP test, girls reported significantly lower levels of perceived math ability than boys (See Table 1). In the highest percentiles of math performance, 64% of boys reported the highest levels of perceived math ability, as compared to 50% of the girls. Even in the lowest levels of math performance, boys tended to be more optimistic with respect to their math ability, 29% of the boys reported the highest levels of perceived math ability, relative to 17% of the girls doing so. Having a parent with an occupation in STEM helped increase math performance but did not seem to help improve perceived math ability, if anything it seemed that those with parents in science were more pessimistic.

Table 1: Perceived Math Ability by Gender, given W-J AP scores (% of sample)

    Perceived Math Ability
W-J AP (percentile) Gender 1 to 3 4 to 5 6 to 7
0-50 Boys 15.9% 55.1% 29.0%
Girls 18.7% 64.2% 17.1%
51-80 Boys 4.1% 44.3% 51.6%
Girls 6.5% 49.9% 43.6%
81-100 Boys 2.8% 32.7% 64.4%
Girls 4.6% 45.2% 50.2%

Note: Weighted percentages reported using child population weights

Interestingly, girls’ lack of perceived ability seems to be something specific to math and not the result of girls generally reporting lower levels of perceived ability. The PSID also included results in the Woodcock Johnson reading test (W-J reading) and asked kids to report on their perceived ability in reading. We use this information to study perceived ability in reading conditional on performance. As it can be seen in the results in Table 2, gender patterns are very different for reading, a subject where girls, on average, outperform boys. In this case, we observe smaller gender differences of perceived reading ability among those scoring in the higher percentiles of the W-J reading test while girls performing in the lower percentiles report higher levels of perceived ability than boys.

Table 2: Perceived Reading Ability by Gender, given W-J reading scores (% of sample)

    Perceived Reading Ability
W-J Reading (percentile) Gender 1 to 3 4 to 5 6 to 7
0-50 Boys 13.4% 57.8% 28.8%
Girls 9.2% 50.1% 40.7%
51-80 Boys 4.3% 47.1% 48.6%
Girls 3.4% 32.4% 64.1%
81-100 Boys 1.9% 29.5% 68.6%
Girls 1.5% 33.5% 65.0%

Note: Weighted percentages reported using child population weights

Finally, since the PSID tracked these kids, we study to what extent math performance and perceived math ability, during childhood, and parental occupation type are related to the probability of majoring in STEM during college. Overall, as expected, we find that women are less likely to major in STEM in our sample, especially when we look at the so called “hard sciences” fields of engineering, architecture, mathematics and computer sciences. Both higher levels of math performance in the W-J AP test and higher levels of perceived math ability are related to higher probabilities of majoring in a STEM field.

But, here is where it gets interesting, the effects of higher levels of math performance and perceived ability are much bigger for boys than for girls. Performing in the highest percentiles of the W-J AP distribution, as compared to performing in the lowest percentiles, is associated with an increase in the probability of majoring in a “hard sciences” STEM field of about 13 percentage points for boys but only 6 percentage points for girls. Similarly, reporting the highest levels of perceived math ability, as compared to the lowest levels, is associated with an increase in the probability of majoring in a “hard sciences” field of about 7 percentage points for boys but only 2 percentage points for girls. These results suggest a loss of STEM enrollment by otherwise capable women.  And we can’t simply fix the problem by trying to boost women’s confidence in their true abilities, because women’s confidence contributes less to pursuing STEM than men’s confidence.  Perhaps men are rewarded for over-confidence in a way that women are not.

Interestingly, having a parent who works in a STEM occupation could help girls and not so much boys. The probability of majoring in “hard sciences” STEM fields increases by about 14 percentage points for girls when one of the parents works in a science job. For boys the increase of this probability is only 4 percentage points. Whatever the reason, these results suggest that parental occupation type could be an important factor reducing gender differences.

As for the answer I gave to my daughter, I said “It is not true that girls are bad at math. Look at your mother. My job is doing math all day!” I work on the field of applied econometrics and so, I guess that was close enough.

Tucson Arizona versus Columbus Ohio

February 15, 2018

(Guest Post by Matthew Ladner)

Large urban districts in Arizona are surrounded by suburban districts accepting transfers through open enrollment. I fear that Arizona is an outlier in this regard, and that the rest of the country is more like Ohio than Arizona. Fordham produced this deeply revealing open enrollment map of Ohio, showing every major urban center to be surrounded by districts who do not participate in open enrollment. Non-participating districts shaded in dark blue:

Now you will recall a similar map of Arizona, with districts not participating in open enrollment again marked in deep blue:

I believe that open enrollment is a big reason that Arizona has been leading the nation in NAEP gains, and that charter and private choice programs deserve some credit the eagerness with which districts participate. Take a look at Columbus on the above map- a large urban district literally surrounded by districts choosing not to allow open enrollment transfers. Now take a look at the school district map of Pima County. The Tucson Unified School District is surrounded by districts that do participate in open-enrollment- actively.

Tucson is a part of the nation’s second fastest growing state, but Tucson Unified has experienced a steady decline in enrollment. This is in part due to the rise of charter schools- as documented by the Center for Student Achievement:

Several of the districts in the chart above gained enrollment despite the increase in charter school enrollment-Queen Creek, Higley, Chandler and Phoenix Elementary. Notice also that these districts, which run a gamut between suburban and urban Arizona, all have growing charter school sectors.

Urban students in Arizona have the opportunity to attend suburban district schools, while their peers in Ohio (and much of the rest of the country) do not. We sadly do not as yet have district by district data on open-enrollment, but research by a Yale student put the figure at almost a third of district K-8 enrollment in Phoenix area districts had utilized open enrollment. We know for instance that Scottsdale Unified has 4,000 students attending through open enrollment. Anecdotally we know that several of the Tucson area school districts are also very active in open enrollment.

Arizona’s urban students have the opportunity to attend suburban schools, and Ohio’s urban students do not. This is primarily in my view because Arizona charter schools have helped open suburban seats, while Ohio’s choice programs have been overwhelmingly focused on urban students. So let’s check NAEP trends for large city students for all six exams for the entire period with state level data:

I’m confident I know what is going right for Arizona’s students in large cities: opportunity. They have the opportunity to attend their home district, suburban districts, charter schools (lots of them) and private schools. Tucson did not participate in TUDA, but does show positive trends in the state’s AZMerit data. Tucson’s enrollment is declining, but scores are improving and that is without factoring in the scores of kids attending suburban district schools, charter schools or private schools with scholarship assistance.

I’m not nearly as confident that I understand what is going wrong for urban students in Ohio, but this:

…is not working for them at all.

Public Service Announcement

February 14, 2018

(Guest post by Greg Forster)

Citizens! You are hereby notified that Mary and the Witch’s Flower, which I reviewed here, is returning to US theaters for two days only next week, due to overwhelming demand.

If you missed it the first time, attendance is mandatory. Otherwise, attendance is merely meritorious.

A theater near me has brought back Darkest Hour, the surprise hit of the Oscar noms – perhaps one near you has done so as well. As I said before, you should see that one on the big screen, too!

End transmission.

Eden and Burke on DCPS Fraud

February 13, 2018

Behold my BROOM ye mighty and DESPAIR!

(Guest Post by Matthew Ladner)

Jayblog readers of a certain tenure may recall the case being made here that outside of the DC Opportunity Scholarship Program and DC charter schools, there was little to celebrate for disadvantaged kids attending DCPS. Over the last decade of available NAEP data, it seemed clear that advantaged students were primarily driving the overall improvement in scores, with DC charters at least showing much larger rates of improvement for disadvantaged students compared to the national average. DCPS, not so much:

Well it turns out that my view of DCPS as being largely inept outside of educating advantaged kids in carefully guarded pockets of excellence was excessively benign: DCPS also developed a systemic approach to academic fraud.

Prosecutors Eden and Burke hit the pages of National Review yesterday to bring us up to speed on the various forms of metric-driven academic fraud recently uncovered in DCPS. DCPS has been engaging in systemic fraud in order to “improve” graduation rates. DCPS “improved” graduation rates by giving diplomas to huge numbers of ineligible students, and “improved” suspension rates by taking them off the books. The FBI is on the case. It’s not pretty. Money quote from Eden and Burke:

When former D.C. Public Schools chancellor Michelle Rhee assumed leadership, she had a searing critique, and a clear argument: Urban schools were paralyzed by collective-bargaining agreements and inertia, so the best path forward was to have expert-designed systems for a new generation of leaders to implement. The unions, in turn, warned that administrators would weaponize these new systems to force teachers to go along with dishonest schemes that would harm true education reform in the service of posting meaningless numerical improvements.

It turns out both sides had a point.

Meet the new boss, same as the old boss.