Florida Scholarships Boost College Enrollment

ednext_20111_figlio_open

(Guest Post by Jason Bedrick)

Florida’s tax-credit scholarship program is the largest in the country, serving more than 100,000 students. Those students tend to be among the most disadvantaged–nearly 70 percent are black or Hispanic and their average household income is only about $25,000.

New research from the Urban Institute finds that participating in the program boosts college enrollment:

Participation in the FTC program increased college enrollment rates by 6 percentage points, or about 15 percent, for students who participated in the FTC program at some point during their education. Of students who entered FTC in elementary or middle school, 45 percent enrolled in college, compared with 39 percent of their non-FTC counterparts. For students who entered FTC during high school, college enrollment rates were 48 percent for FTC students and 42 percent for non-FTC students.

Of course, opponents of choice are straining mighty hard to dismiss these findings.

Samuel Abrams, director of the National Center for the Study of Privatization in Education at Teachers College, Columbia University, criticized the study’s methodology as flawed, saying that students who had the energy and motivation to get accepted and remain at private schools may already have an edge compared to their peers in public schools. Abrams said the American education system must be improved by addressing income inequality, accessible childcare and health care and teacher pay in public schools and not by putting more students in private schools.

“This is a solution for some kids, but it can hurt other kids because it concentrates underperformers in their default neighborhood public school,” Abrams said.

Actually, every claim Abrams made is flatly contradicted by previous research.

False Claim #1: Scholarship Students Were More Advantaged

Annual studies by Dr. David Figlio and later by researchers at Florida State University found that participating students were more disadvantaged before entering the program. The most recent study found:

[C]ompared to eligible non-participant students, new FTC students had poorer test performance both in ELA and math before entering the FTC program.

Contrary to Abrams, the scholarship students did not “have an edge compared to their peers in public schools” — they were behind those peers.

False Claim #2: Scholarships Concentrate Poor Performers in District Schools

As noted above, rather than “concentrate underperformers in their default neighborhood public school,” the program gave the most disadvantaged students the opportunity to attend new schools where they caught up to their peers academically (indeed, the FSU research shows that they were competitive with the national average, outperforming their low-income peers), and then were more likely to go to college.

False Claim #3: Scholarships Hurt Nonparticipants

Abrams claimed that the supposed concentration of underperformers in district schools would then hurt those students, presumably via peer effects (as he alluded on Twitter). However, not only was there no such concentration of underperformers, an earlier study by Dr. Figlio and Dr. Cassandra Hart found in that competition from the choice program improved the performance of district school students. Far from hurting them, as Abrams claims, the research shows that increased choice and competition helped everyone.

And on top of it all, the tax-credit scholarship program achieves all this while saving taxpayers money.

That’s a win-win-win situation if there ever was one.

[Note: This blog post was edited slightly for clarity.]

 

 

Advertisements

16 Responses to Florida Scholarships Boost College Enrollment

  1. Matthew Ladner says:

    BOOM!

  2. I think it is important to note that the Urban Institute study uses a matching design, which does not allow us to have the same confidence in effects as an RCT. While the Urban Institute used a good matching approach that involved matching students by geography as well as several observable characteristics, there is still the real possibility that students seeking a scholarship were different in un-observable ways. Those un-observable differences that motivated them to seek school choice may have made them less likely or more likely to succeed — it’s hard to know.

    Given that I complained about CREDO’s matching design, I want to be consistent in urging caution about this study. See https://jaypgreene.com/2017/08/28/credo-is-not-the-gold-standard/ I do appreciate that at least the Urban Institute researchers provided sufficient caveats in describing their methodology and results and did not misleadingly describe their comparisons as involving “virtual twins.”

  3. Jason Bedrick says:

    ^Duly noted. FTR, this will not be included in EdChoice’s list of gold standard studies as it is only silver standard. As you note, the researchers were careful to note their limitations.

    My main point was to rebut the demonstrably false statements by Abrams. Whether one buys the Urban Institute research or not, it is simply factually incorrect to say that the scholarship recipients were more advantaged than their peers, or that the program results in “concentrating underperformers” in district schools.

    • bkisida says:

      Still, to be consistent I don’t think we can refute #1. I say “we” because to my knowledge, we’ve all been open about our doubts regarding test scores.

      Describing the comparability of the groups at baseline, and having confidence that the outcomes in a matching design are free from selection effects, comes down to the same thing: Do you trust observable characteristics (test scores) as reliable indicators of unobservable characteristics (relative advantage)?

      In this case, one of the most important “matching” variables is a test score. We can’t doubt the importance of test scores one day and hang our hat on them the next.

      Using precise language is one way to circumvent this problem. Like Figlio said, “Compared to eligible non-participant students, new FTC students had poorer test performance both in ELA and math before entering the FTC program.”

      • Good point. In general, I am not a big fan of matching studies.

      • Jason Bedrick says:

        Yes, I’ve certainly expressed doubts about test scores as well. But by practically any observable measure (test scores, race, income/FRL eligibility), these students were less advantaged. If Abrams is going to claim that they were somehow *more* advantaged, then the burden of proof is on him to demonstrate that that’s the case. There’s simply no evidence to support his assertion — and quite a bit of evidence suggesting the opposite.

        And to be clear, that language was not from Figlio but from the more recent FSU team. Figlio wrote: “These differences are large in magnitude and are statistically significant, and indicate that scholarship participants tend to be considerably more disadvantaged and lower-performing upon entering the program than their non-participating counterparts.”

        https://www.stepupforstudents.org/wp-content/uploads/2015/09/2009-uf-report-on-ftc-test-scores.pdf

      • Jason Bedrick says:

        Moreover, what the Figlio/FSU studies are looking at is the difference between scholarship participants and all eligible non-participants. That’s not a matching exercise.

        If the Q is whether, among eligible students, the scholarship program enables or encourages creaming, then the answer is very clearly “no.”

      • Brian Kisida says:

        I agree that the cream-skimming argument is pretty lame and we have a lot of evidence that refutes it. I think we may be just talking past each other. If I’m giving him the benefit of the doubt, think Abrams was making the argument that a matching design cannot overcome selection bias, and proving equivalency with observables (test scores) doesn’t fix it. I think he is right on that. Then again, it does seem like his follow-up point IS about cream-skimming, so I can’t really be sure.

        So just to clarify: 1) We should be skeptical of matching designs that produce equivalency on all measures except selection itself (opting into or out of a program). 2) There is an abundance of evidence that choice does not induce cream-skimming in the aggregate.

        I wouldn’t even consider propensity score matching silver standard. Silver is IV or RD. Bronze is Diff-in-Diff/Panel Data. PSM is a plastic participation trophy.

        Please be nice to me the next time I conduct one : )

      • Greg Forster says:

      • Brian, thank you for weighing in. After all, I never said in the story in question that students using vouchers came from more advantaged homes. I merely echoed what Matthew Chingos and Daniel Kuehn conceded in their report for the Urban Institute. The methodology used in their report, Chingos and Kuehn wrote, “may not eliminate bias from student selection into the program…. Important unmeasured characteristics include student and parental motivation” (pp. 2, 12). This matter aside, it must also be noted that Chingos and Kuehn wrote, “Private schools can continue to use their usual admissions processes to select applicants” (p. 5). As private schools can screen and, later, prune while default neighborhood public schools can’t, the comparison is inherently flawed.

  4. pdexiii says:

    How some people continue to make these boilerplate claims when the facts say the opposite makes me think they also b.s.’d their way to their college degree, and a lot of what else they have earned. A person like Adams would never debate any of the astute contributors to this blog; it would be like Dorothy figuring out who’s behind the curtain.
    #charlatan.

    • Sir, rest assured I am happy to debate people on the other side of this issue and have. Here, in fact, is the link to the account of a debate with Bob Bowdon of ChoiceMedia, podcast included, held this past July and hosted by the Soho Forum, a libertarian organization in New York, sponsored by the libertarian magazine Reason: http://reason.com/blog/2017/07/12/soho-forum-school-choice

    • Jason Bedrick says:

      pdexiii, I highly suggest watching the debate that Prof. Abrams posted between him and Bob Bowdon. It was a model of civility — something we could use a lot more of these days.

      • Thank you for that endorsement, Jason. And please call me Sam. Indeed, there is no way forward without hearing each other with consideration and respect. After all, as I said at the outset of my debate with Bob Bowdon, opposing parties in education policy have much more in common than is typically conceded.

  5. […] AP, U.S. News, Chalkbeat, Education Week, Washington Examiner, The Hill, redefinED, Cato, Fordham, Jay P. Greene’s blog, and Florida outlets here, here, here, and […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s