Work Hard. Do Your Research. Does KIPP steal the best students?

August 8, 2014

(Guest Post by Collin Hitt)

KIPP schools demand a certain kind of student – a student who is willing to put in long hours and put up with very strict rules. KIPP has been shown to substantially increase student test scores. But critics argue that the culture at KIPP has major effects on recruitment and retainment. KIPP schools attract better students and are more likely to weed out low performing students, the argument goes. If this is true, KIPP students who persist in school are more likely to have a high-achieving peer group – and the effects of simply being in a peer group are really what explain any positive effects at school. A new study from Mathematica destroys this critique.

At its core, the critique of KIPP is a restatement of larger questions facing the charter school sector. Do charter schools cream the best students from nearby schools? And, compared to surrounding schools, are the lowest performing students at charter schools most likely to leave? Two rigorous studies reviewed here at JPGB answer an unequivocal “no.” But KIPP is a crucial case. The average charter school might take all the students it can find and do anything to keep those students. But surely if anybody engineers the makeup of their student body, it would be a school like KIPP, right?

So, does KIPP cream the best students (or at least better-than-average students) from nearby schools? The following chart shows, clearly not.

Entering KIPP students perform the same or worse than students in surrounding schools. But does KIPP then take exceptional efforts to push low performers back into surrounding schools. Again, clearly not.

Students transfer out of KIPP schools at the same rate as surrounding schools. And the students who transfer perform the same on standardized tests. So the only manner that KIPP may in some way create a measurably different peer group is through the quality of students in later grades who replace the KIPP students who transfer out. In this respect, the students who later transfer into KIPP are higher performers on average than students who transfer into district schools, according to the Mathematica study. But this, of all the ways to create a higher-performing peer group, is the least likely to have any meaningful impact on the performance of students who enter KIPP early on. The high performing peer group wouldn’t even be formed until students’ time at KIPP was almost over.

With their typical class, the Mathematica authors give their critics a charitable hearing, in fact constructing the strongest possible case for the peer-effect hypothesis. So, do peer effects explain KIPP’s impact on test scores? From the Mathematica study itself:

“One way to estimate the possible size of peer effects at KIPP is to combine our findings with other research on how peers’ prior scores affect student achievement. Unfortunately, published estimates of the effect of peer ability on student achievement range widely, from close to zero to nearly half a standard deviation impact for each standard deviation of difference in peer achievement. Even if the largest estimates of peer effects are correct, however, the improvement in peers’ prior test scores would appear to benefit KIPP students’ achievement only by about 0.07 to 0.09 standard deviations after four years at KIPP. KIPP’s cumulative impacts in middle school are three times that size, so even the largest estimates of the size of peer effects suggest that they are unlikely to explain more than one-third of the cumulative KIPP impact.

“Moreover, the best available evidence shows that KIPP produces large impacts on students in their first year at a KIPP school—before late-entering students could possibly have any effect. Consequently, the true peer effect resulting from late entrants is likely to be substantially below the back-of-the-envelope estimate of 0.07 to 0.09 standard deviations.”

The peer-group critique of KIPP essentially says this: anybody could get KIPP’s results if they had KIPP’s students. This simply isn’t true. KIPP is getting better results because of the work being done by teachers and staff. Rather than wonder, if only other schools could have students like KIPP’s, perhaps we should wonder why other schools don’t have adults like KIPP’s. (And, for that matter, why don’t other think tanks have scholars like Mathematica’s?)


Welcome Back to School!

August 8, 2014

Lorie Ann Hill

I know it’s early August but in some areas of the country they are already heading back to school.  And in Wagoner, Oklahoma we have a report of a special education teacher who was arrested on the first day back “after she showed up at school under the influence of alcohol and without her pants.

Andy Rotherham dryly observed: “I blame Common Core.

But hasn’t Oklahoma committed to withdrawing from Common Core?  Maybe he means that he blames the absence of Common Core.  I, as a believer in incentives, blame the relatively low cost of booze and the relatively high cost of pants. Or something like that.

FEA: We Love Late Amendments to Omnibus K-12 Bills! No We HATE THEM, Oh, what are WE DOING?!?!?

August 7, 2014

(Guest Post by Matthew Ladner)

So the Florida legislature adopted an $18.4 million dollar ESA program for children with severe disabilities as a late amendment to an omnibus education bill. The Florida Education Association has filed suit against the state, loudly trumpeting its desire to defend due process, the rule of law and the American way.

Joanne McCall, the Vice President of the Florida Education Association wrote the following in a newspaper column titled Lawsuit tackles Legislature’s ‘backdoor’ way of passing bills:

We’re all taught to play by the rules. In a civil society, we rely on rules and procedures and laws as we go about our daily routine. When people break the rules, they’re expected to be held accountable for their actions — whether it’s within your family, on the job or at school, or in our society as a whole. The Legislature is no exception. There are rules and procedures in the Florida Constitution, in Florida statutes and in the House and Senate chambers that set out the right way to do things — such as pass a law.

I have yet to read Rules for Radicals but I gather that it recommends a rather cut-throat ends-justify-the-means casual attitude about the truth. Practitioners should have learned from the Dan Rather implosion over “fake but accurate” however that it is awfully easy for people to check up on things these days, and thus a rather simple matter to unmask shallow, self-serving hypocrisy. Someone may want to write a Saul Alinsky for Dummies updated for the internet age, it might lead to a more honest debate and avoid needless bumbling.

Take the Florida Education Association’s current antics for example. Jon East over at RedefinED for instance found that the Florida Education Association supported a $480,000,000 teacher pay raise through almost an identical legislative process a mere two sessions ago: late amendment attached to an omnibus education bill. It does not take an overly active imagination to think that this is probably not the first such incident employed by the FEA, simply the most recent.

The Florida Education Association was strangely silent concerning procedural preferences when the last-minute amendment to an omnibus education bill netted a $480,000,000 teacher pay raise.

In fact, Florida Education Association President Andy Ford praised Governor Rick Scott for getting ‘er done:

Ford said, “FEA thanks Governor Scott for his efforts to provide an immediate across-the-board pay increase to Florida’s classroom teachers in recognition of their demonstrated performance which has brought Florida’s education system to sixth in the nation.  FEA applauds the infusion of additional resources into public education as was proposed by the Governor.

Ford could have objected to the procedure used to get this teacher pay raise, and even could have filed suit to stop it. Instead he thanked Governor Scott for pulling it off and groused over some of the details of the funding. One year later a remarkably similar legislative procedure creates a $18.4 million program for children with severe disabilities, and the FEA sends their Vice President out into the papers to wax poetic about legislative process:

These laws failed to pass the right way. They went through the legislative process and didn’t get enough votes to be enacted. So legislative leaders came up with a way to circumvent the rules. This was a backdoor way for legislative leaders to enact measures that had already failed. We all have to be accountable for our actions, even the leaders of the Florida Legislature.

So the $480,000,000 question for the FEA: are you willing to give up the half a billion pay increase and everything else that you have passed over the years through late amendments to omnibus education bills to quash an $18.4 million program for children with severe disabilities?

Keeping Score in the Greene-Polikoff Wager

August 7, 2014
The unraveling of Common Core makes this flop the most obviously ill-conceived and doomed-to-fail reform effort since the Annenberg Foundation threw $500 million away in the 1990s.
Morgan responded:
At last count, 1 state out of 45 has repealed the standards.
So we agreed to make a wager:
In ten years, on April 14, 2024, I bet Morgan that fewer than half the states will be in Common Core.  We defined being in Common Core as “shared standards with shared high stakes tests-even if split between 2 tsts.”  Given 51 states and DC, Morgan wins if 26 or more states have shared standards and high stakes tests and I win if the number is 25 or less.  The loser has to buy the winner a beer (or other beverage).
It hasn’t even been four months, but I thought it might be useful to report the current score on our bet.  With the withdrawal of Iowa this week from the Smarter Balanced testing group, there are only 26 states that plan to use one of the two national tests to assess their students during the 2014-15 school year.  It’s true that 35 states remain part of the two testing consortia and some of the 9 states that have delayed implementation of the common tests may begin using one of them in the next few years.  But it’s safe to say that several of those 9 delayed start states will never follow through.  And some of the 26 states actually using a common test in 2015 are already making noises about withdrawing.  See for example reports coming out of Wisconsin and South Carolina.
If one more state that is currently using one of the common tests drops it than decides to follow through on implementation, I will have won the wager.  And we have more than 9 years to see that happen.  Mmmmm.  I’m thinking that a nice Belgian ale would be a delicious prize for victory.

School Breakfast Research

August 6, 2014

Education is dominated by “do-gooderism.”  Everybody wants to help children.  But sometimes the desire to help is seen as sufficient proof that one is actually helping.  And too often little thought is given to how trying to help might do some harm.

Which brings us to school breakfast programs.  Who could be against helping kids by making sure they start their day with a healthy breakfast provided in school?  Well, it is possible that those programs don’t do much good.  And it is possible they do some harm.

Last month Diane Whitmore Schanzenbach and Mary Zaki of Northwestern University released their analysis of a randomized experiment in which access to free school breakfast was expanded.  You can read the abstract and full report on the National Bureau of Economic Research web site.  Schools that offer free breakfast often have low participation rates.  So to learn about how to increase participation 70 matched pairs (or triplets) of schools participated in an experiment in which they could offer universal free school breakfast regardless of individual student eligibility for subsidized meals or breakfast in the classroom (BIC), where all students are given breakfast in their classroom at the start of the school day.

Schanzenback and Zaki conclude:

We find both policies increase the take-up rate of school breakfast, though much of this reflects shifting breakfast consumption from home to school or consumption of multiple breakfasts and relatively little of the increase is from students gaining access to breakfast. We find little evidence of overall improvements in child 24-hour nutritional intake, health, behavior or achievement, with some evidence of health and behavior improvements among specific subpopulations.

Providing breakfast in the classroom, not surprisingly, has a very large effect on whether students participate in the breakfast program because it’s given to every student in the classroom.  Pretty much the only way you could not participate is by not being in school.  Universal breakfast has a more modest effect on increasing participation in the program (about 10 percentage points) because students have to arrive early to get the breakfast.

So if the goal of the program is to have people participate in the program, BIC is a huge success and universal breakfast is a modest success.  But if the point is to increase the amount or quality of calories students consume or to alter their behavior or learning in school, these programs don’t seem to be effective.  Universal breakfast does not even seem to have an effect on whether students eat breakfast or not.  It only shifts whether students eat breakfast at home or at school.  BIC does increase whether students eat breakfast (or have two breakfasts), but has no effect on total caloric intake.  Students just shift their eating so that they have fewer calories at other meals.

But when students eat might affect their health, behavior, and learning outcomes, so the researchers looked at whether the BIC program helped by increasing the likelihood that students would have breakfast even if those calories were offset by a reduction in eating at other times.  Unfortunately it didn’t.  They conclude: “The BIC treatment does not statistically significantly improve any outcome.”

So, expanding access to school breakfast does not seem to have any meaningful benefits.  Where’s the harm?  Leaving aside the cost to taxpayers, the greatest potential harm to these programs is that they alter the relationship between families and their schools by displacing the family’s traditional role of feeding their own children.  Doing so may make the families and students feel more dependent on the government and make school teachers and administrators view families and students as generally incompetent.  The state becomes the new Daddy and the parents become children incapable of providing for themselves or their own children.

This all makes me think of the new book by Jason Riley of the Wall Street Journal: Please Stop Helping Us.  We need to hold in check our desire to do good by remembering first to do no harm.

Arizona charter schools and the new report card rankings

August 6, 2014

(Guest Post by Jonathan Butcher)

The new A-F report card rankings are up for Arizona public schools, and the news is good—if you’re sending your child to a charter school. Last year, 40 percent of Arizona charter schools earned an A, compared to 28 percent of traditional schools.

Now that Arizona has four years’ worth of A-F rankings, a year-to-year comparison of charter and traditional schools reveals that charter schools’ success over time is what we hope would have happened to all public schools: more charters are earning A’s and fewer are earning D’s as the years go by (note: Arizona managed to make it so hard to earn an F that few schools have done so).

2010-11 Arizona A-F Letter Grades, Charter v. Traditional

 Butcher 1

2011-12 Arizona A-F Letter Grades, Charter v. Traditional

Butcher 2


2012-13 Arizona A-F Letter Grades, Charter v. Traditional

Butcher 3

2013-14 Arizona A-F Letter Grades, Charter v. Traditional

Butcher 4

Between the 2010-11 school year and 2012-13 school year, charter schools occupied the two ends of the A-F distribution, with higher percentages of schools earning A’s and D’s than traditional schools.

This year, however, charter schools own the “A” category, while the percent of charters earning D’s has been cut in half—and, for the first time, is lower than the percent of traditional schools earning D’s. True, the percent of traditional schools earning A’s crept up each year, but not as quickly as charter schools. And the percent of traditional schools earning D’s was relatively consistent.

Nothing is held constant here, so I’ll be the first to admit the limits to these charts. Plus, data from the Arizona Board of Regents (ABOR) reports that Arizona high schools are not preparing students for college, so the achievement reflected in these report cards is decidedly less impressive than it should be.

But to the extent that these school grades reflect student success (see here for how the report cards are calculated), charter schools are leading the way—and at a per student cost of $1,500 less than traditional schools. And there’s something to be said for charter schools’ unique designs, whether it’s hybrid classrooms, college prep, or career and technology centers for at-risk students aged 14-21.

Clearly there’s a reason why charters are the fastest-growing sector of the public school system.

You’re Gonna Need A Bigger Boat

August 5, 2014

(Guest Post by Lindsey Burke)

Thousands of families in Florida have applied for a Personal Learning Scholarship Account (PLSA) for their children. Step Up for Students received 1,200 PLSA applications in under a week, and according to Step Up’s Patrick Gibbons, Florida parents had started 2,050 applications as of August 5th. Enrollment in Arizona’s Empower Scholarship Account program has nearly doubled from last year to this year, to about 1,300 students.

School choice is a rising tide that lifts all boats. It looks like with school choice 2.0 – education savings accounts – You’re gonna need a bigger boat.


Get every new post delivered to your Inbox.

Join 2,538 other followers