Work Hard. Do Your Research. Does KIPP steal the best students?

(Guest Post by Collin Hitt)

KIPP schools demand a certain kind of student – a student who is willing to put in long hours and put up with very strict rules. KIPP has been shown to substantially increase student test scores. But critics argue that the culture at KIPP has major effects on recruitment and retainment. KIPP schools attract better students and are more likely to weed out low performing students, the argument goes. If this is true, KIPP students who persist in school are more likely to have a high-achieving peer group – and the effects of simply being in a peer group are really what explain any positive effects at school. A new study from Mathematica destroys this critique.

At its core, the critique of KIPP is a restatement of larger questions facing the charter school sector. Do charter schools cream the best students from nearby schools? And, compared to surrounding schools, are the lowest performing students at charter schools most likely to leave? Two rigorous studies reviewed here at JPGB answer an unequivocal “no.” But KIPP is a crucial case. The average charter school might take all the students it can find and do anything to keep those students. But surely if anybody engineers the makeup of their student body, it would be a school like KIPP, right?

So, does KIPP cream the best students (or at least better-than-average students) from nearby schools? The following chart shows, clearly not.

Entering KIPP students perform the same or worse than students in surrounding schools. But does KIPP then take exceptional efforts to push low performers back into surrounding schools. Again, clearly not.

Students transfer out of KIPP schools at the same rate as surrounding schools. And the students who transfer perform the same on standardized tests. So the only manner that KIPP may in some way create a measurably different peer group is through the quality of students in later grades who replace the KIPP students who transfer out. In this respect, the students who later transfer into KIPP are higher performers on average than students who transfer into district schools, according to the Mathematica study. But this, of all the ways to create a higher-performing peer group, is the least likely to have any meaningful impact on the performance of students who enter KIPP early on. The high performing peer group wouldn’t even be formed until students’ time at KIPP was almost over.

With their typical class, the Mathematica authors give their critics a charitable hearing, in fact constructing the strongest possible case for the peer-effect hypothesis. So, do peer effects explain KIPP’s impact on test scores? From the Mathematica study itself:

“One way to estimate the possible size of peer effects at KIPP is to combine our findings with other research on how peers’ prior scores affect student achievement. Unfortunately, published estimates of the effect of peer ability on student achievement range widely, from close to zero to nearly half a standard deviation impact for each standard deviation of difference in peer achievement. Even if the largest estimates of peer effects are correct, however, the improvement in peers’ prior test scores would appear to benefit KIPP students’ achievement only by about 0.07 to 0.09 standard deviations after four years at KIPP. KIPP’s cumulative impacts in middle school are three times that size, so even the largest estimates of the size of peer effects suggest that they are unlikely to explain more than one-third of the cumulative KIPP impact.

“Moreover, the best available evidence shows that KIPP produces large impacts on students in their first year at a KIPP school—before late-entering students could possibly have any effect. Consequently, the true peer effect resulting from late entrants is likely to be substantially below the back-of-the-envelope estimate of 0.07 to 0.09 standard deviations.”

The peer-group critique of KIPP essentially says this: anybody could get KIPP’s results if they had KIPP’s students. This simply isn’t true. KIPP is getting better results because of the work being done by teachers and staff. Rather than wonder, if only other schools could have students like KIPP’s, perhaps we should wonder why other schools don’t have adults like KIPP’s. (And, for that matter, why don’t other think tanks have scholars like Mathematica’s?)



11 Responses to Work Hard. Do Your Research. Does KIPP steal the best students?

  1. pdexiii says:

    Our charter school is not a KIPP school, but I’ve observed that the students whose parents choose to remove them from our school were not willing to abide by our expectations, or worse their parents weren’t willing to abide by our expectations.

  2. Cris R says:

    It seems that most documentation used by charter school supporters mistakenly (or purposely?) lump all “low income” students together in their research findings. Especially considering many KIPP schools are already located in areas with a high density of low income families, it’s disingenuous to ignore the statistical differences between free lunch and reduced lunch students (one of the ways we have of measuring the level of poverty a student is suffering the effects of) – especially when looking at attrition rates.

  3. allen says:

    The question – Does KIPP steal the best students? – implies some claim on those “best students” the implication being that those “best students” are in some way property of the school district.

    They’re not and the claim, with its necessarily veiled implication of ownership and thus of theft on the part of charters, is typical of the sorts of dishonesty necessary to try to shield school districts from competition.

    Historically, school districts did, for about seven hours a day, command the presence of most kids within their geographical boundaries. The parallels to ownership are obvious to everyone although proponents of the district model recoil from that characterization for political purposes. The preferred portrayal of mandatory attendance was of performing a great boon for the children and society with the mandatory nature of the relationship treated as an unimportant detail.

    But in the face of a rising tide of competitive offerings it’s difficult to leave any option unused and one such option is to carefully assert ownership. To suggest without being too obvious about it that the kids have some responsibility to the school district. To that suggestion I’m guessing most charter school parents will reply with “screw you, you had your chance”.

    • pdexiii says:

      “Screw you, you had your chance” is the sentiment that drives many urban parents to charter schools, and is the 800 lb. Silverback in the room many charter opponents-traditionalists never confront publicly. Many of these parents attended these schools or had friends/relatives attend these schools, and given the opportunity seize something better for their children.
      It is insulting to parents who keep their children in traditional schools when charter-school opponents, especially traditional school educators complain that charters ‘take the best students.’ 1) You’re saying my child is ‘not the best,’ and 2) you’re all but admitting you cannot teach less than the ‘best’ students.
      Using the urban vernacular, “Is you mad, bro?” that these parents are saying you stink at educating my child by choosing another education option? (POTUS voice): “Why they always hatin’?”

      • momof4 says:

        Even worse is the explicit sentiment that parents of the “best” students should send their kids to the district/unionized/neighborhood schools, so they can be role models for less-capable and/or less-motivated kids. I find that idea to be flat-out immoral. No kid should be expected to serve the needs of any other kids, but should be placed in an environment that is best for that child. Period.This issue comes up regularly, in the WaPo, regarding gentrifying areas of DC.

  4. Cris R. says:

    It’s not about taking the “best students.” It’s about excluding the neediest students, and it is immoral.

    • pdexiii says:

      I will use my charter school as an example. How do we exclude the neediest students? I realize you have no insight into how our school operates, but we do not, and cannot by law, use grades or their IEPs as selection criteria. As I said previously, the students who dis-enroll from our school tend to be parents who feel our ‘rules are too much.’
      Those neediest students need something beyond what a traditional school can offer, specifically something more like Hogwarts or Phillips Exeter Academy. The issues they bring originated beyond the classroom, and all the parent training, after school programs, and counseling people propose can’t fix the damage from 11-13 years of instability or chaos. It does no good for a teacher to accommodate per an IEP in class, and that child goes home to a disruptive home environment, inadequate diet, or worse indifferent or abusive adults.
      Some of my colleagues and I have dreamed of opening a charter school that takes only the neediest students: the tired, the poor, the huddled masses, etc. This type of school would cost more than an average school because you’d need 5-star teachers, administrators, and support staff, and lots of them. As opposed to focusing on what school like KIPP, and I guess my school don’t do, more folks should offer alternatives that meet a need KIPP and similar schools don’t.

    • allen says:

      It’s not about either. It’s about being revealed as an inherently inept institution that only occasionally and only locally rises above the mediocrity that’s the best most parents can hope for in their local public schools.

  5. Thats really a informative post about KIPP schools, I didnt even knew a few things, thanks for this post.

  6. PhillipMarlowe says:

    So, has Mathematica researched why KIPP failed and bailed on Cole Middle School in Colorado?

    • buck8823 says:

      Did Mathematica do research on why thousands of public schools fail their communities each year with zero accountability?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s