Texting Nudges Harm Degree Completion

January 17, 2019


When behavioral nudges using text messages became the flavor of the month a few years ago I expressed some serious reservations. In general, I was concerned that nudges substitute the preferences of distant experts for those of people who may understand their own situation better, thereby pushing people to do things against their better judgement.  These interventions may appear successful in the short run, especially when we examine near-term outcomes that are over-aligned with the intervention, but they may harm people over the long run.

In particular, I was responding to texting nudges being advocated by Ben Castleman and others to reduce “summer melt” by getting students to complete the FAFSA and enroll in college.  I wrote:

…even if sending text messages is successful at getting more low-income students to complete the FAFSA and enroll in college in the fall, it is unclear whether this ensures a positive outcome. Students who start college but then fail to finish may be hurt by forsaking employment and other training opportunities and taking on significant debt for a credential they never earn. The students who are accepted to college but then decide not to enroll may have just been deterred by an intimidating form, as Castleman suspects, or they may know things about themselves that made them rationally decide not to pursue a degree they are unlikely to complete. The 160-character solution may unwittingly push students into making decisions that are against their better judgment and end up harming them. Castleman has not reported retention and graduation rates from the texting intervention, so we do not know whether this behavioral nudge is helping or hurting students in the long term.

Well, Ben Castleman and Lindsay Page have finally released results on the longer-term effects of their “summer melt” texting nudge and they are pretty clearly negative.  That is, students randomly assigned to receive texts to remind them to complete the FAFSA while they are seniors in high school are significantly less likely to complete an AA or BA degree than those who were not nudged into completing the forms necessary to get financial aid and enroll in college.  As can be seen in Table 10, the treatment group was 1.7 percentage points less likely to complete a BA degree in 4 years (p< .01).  The treatment group was also .8 percentage points less likely to complete an AA degree after 2 years and 1.0 percentage point less likely to complete an AA after 3 years (p < .1).

Castleman and Page then focus on the subset of subjects who were in the uAspire program and for whom they had outcomes after 6 years.  At the end of 6 years the students who were randomly assigned to receive texting nudges were 2 percentage points less likely to have earned an AA degree (p < .1) and they were no more likely to have earned a BA degree. (See Table 12).

Castleman and Page also report results on whether students already enrolled in college are more likely to complete their degrees if they are randomly assigned to receive texts reminding them to renew the FAFSA so they can continue receiving financial assistance.  As shown in Table 18, students reminded to renew the FAFSA are no more likely to complete an AA or BA degree.

So, the longer-term results of these texting nudges are generally null or negative despite initially encouraging results that the intervention got more students to complete a form and enroll in college.  The problem is likely to be exactly what I suspected.  Students were being pushed into doing things that were against their own better judgement.

Researchers and foundation officials may think everyone should enroll in college but they don’t know each student’s circumstances and are very poorly positioned to know what is best for others.  Students and the advisors (family, counselors, educators, etc…) who know them and with whom they have authentic relationships understand the context better and are more likely to make good decisions.  Technocrats are inclined to manage things from afar, but this approach is very likely to end up hurting students.

The early, positive results for texting nudges received an enormous amount of attention, including an endorsement from Bill Gates and being featured on NPR’s Hidden Brain. Given that we now know that this type of nudge intervention may harm students, I hope there is a comparable amount of attention given to the release of the longer-term negative results.  There is no shame inherent in an intervention failing, but there are serious problems if we only tout temporary successes while ignoring long-run damages.

To their credit, Castleman and Page appear to be turning their attention in their newer research to “higher-touch” interventions that may cost more but may also have a better chance of providing guidance better suited to each student’s situation.  Higher-touch interventions also seem to acknowledge that success for students typically requires much more than a reminder or some information.  To succeed students need character traits that will help them make better decisions for themselves over and over as life presents an endless string of challenges.  To shape character requires human interaction and meaningful relationships.  That’s something that a “bot” or text message simply cannot do.


Oklahoman Op-Ed

January 13, 2019


(Guest post by Greg Forster)

The Oklahoman carries my op-ed on why collective bargaining is a bad deal for teachers:

Teachers are like doctors and lawyers. Standardizing the work they do into a one-size-fits-all mold creates major headaches. But collective bargaining demands standardization, so processes and outputs can be negotiated.

The standardization demanded by collective bargaining is a major factor in all the complaints we’re accustomed to hearing from public school teachers — useless paperwork, unreasonable rules, rigid systems, dysfunctional bureaucracy. In a 2009 study of national data from the U.S. Department of Education, I compared public and private school teachers. The difference in teacher working conditions was dramatic.

Remember, you read it here first!

Celebrity-Worship And Dysfunction in Social Science

January 8, 2019

Image result for celebrity worship


Rick Hess is releasing his EduScholar ranking tomorrow and we should expect a flurry of tweets and even university press releases bragging about people’s position on that measure. Even though Rick’s ranking (for which I serve as a nominating committee member) is a completely made up thing that has never been validated, people within our field act like it is incredibly important and care intensely how they are ranked.  I suspect that Rick knows this and that the ranking is epic-level trolling on his part.

But the field’s over-reaction to this unimportant ranking is a sign of its obsession with celebrity-worship. Even though almost no one outside of a few hundred people within our field cares about who we are or what we do, the folks within the field are crazed with a desire to have status and power in this universe that no one else even notices.  It’s as if we are in high school and everyone is obsessed with being accepted by the small circle of cool kids.  No one outside of your high school knows or cares who the cool kids are, but to those in the high school it feels like the most important thing in the world.  This is pretty much what education policy and other social sciences look like.

This excessive concern with status within our field is both the result of and contributor to a series of problems.  In economics, which seems the most afflicted with celebrity-worship, we see power and status concentrated in a small number of people within a small number of departments.  That small clique effectively controls the top journals in the field, dominates the main professional association, and has disproportionate influence over who is hired and tenured at those few departments.

Not surprisingly, this concentration of unchecked power leads to a variety of abusive behaviors.  People at the top of this status system can more easily maintain their power and help their friends, which is not only grossly unfair but also hinders a truly meritocratic pursuit of the best people, ideas, and research.  In addition, because those at the top are predominantly white and male, this strict status system excludes women, minorities, and all other newcomers who may differ from those with greater power.  And this concentration of unchecked power has also likely contributed to sexual harassment, intellectual theft, exploitation, and generally rude behavior.

Many people are beginning to speak out about these abusive behaviors.  There were several panels at the most recent ASSA meeting to document these issues and discuss what to do about them.  While this is all very encouraging, I fear that people may be missing what I suspect is the heart of the problem.  We can’t fix abusive and anti-intellectual behavior in social science by replacing a male-dominated status hierarchy with a more gender balanced system that still concentrates status and power so severely.  We suffer under a good old boy system, but we would continue to suffer even if they thought they were good, were women, and much younger.  The problem is the unchecked concentration of power and status.

I think we would be much better off if control over the main journals and professional associations was dispersed outside of a small number of people at a small number of institutions.  Boards for journals and professional associations tend to be self-replicating bodies that draw from the same incestuous pool.  They should consider adopting by-laws or at least norms that push them to consider finding new members outside of their familiar, friend and colleague circles.  It would also be helpful for professional associations and journals to adopt real grievance procedures so that intellectually dishonest or personally abusive behavior could be considered with due process and treated with appropriate sanctions.  My personal experience is that this is not happening.

But even more important than changing association and journal rules and procedures, we need to abandon the culture of celebrity worship.  No one in education policy or other social sciences is actually that important.  At most they are King of the Lilliputians.  At worst they are folks who were excluded from the in-groups in high school now taking their revenge by terrorizing those beneath them.  For the most part, no one in the outside world cares about who we are, what journals we publish in, what rankings we get, etc…  None of us are celebrities.

People at the top of our status system only have power because we have given it to them by acting like they are celebrities and that their position really matters.  The solution is the same as when we were in high school.  The only way to avoid being terrorized by the in-group is to stop caring about the in-group.  They just don’t matter.  Form your own chess club, play D&D, and ignore the football team and cheerleaders.

So when Rick’s ranking comes out, have a good laugh and think about how much those who are striving to be at the top of some silly list are wasting their lives. When high school is finished no one will remember or care that they were once really cool.

Collective Bargaining Hurts Teachers

January 8, 2019


(Guest post by Greg Forster)

OCPA carries my article on why collective bargaining hasn’t been a good bargain for K-12 teachers:

I’m not against unions. My wife worked for a union for years, volunteering long hours as an employee advocate in company dispute resolution. She signed up to work for the union when she saw managers mistreating workers, and the company violating its contractual obligations to them. The union was the only effective protection those workers had.

But collective bargaining and representation simply isn’t a good fit for K-12 teachers. Not all types of workers are well-served by unionizing. Doctors and lawyers don’t unionize. The nature of the work they do just doesn’t permit the standardization, controlled processes, and highly specified work outputs that are necessary for collective bargaining to be effective.

Let me know what you think!

Update: Also worth noticing: “our regression results indicate that unionization has a powerful negative influence on educational outcomes.”

Educational Opportunity and the Widow’s Mite

January 2, 2019

(Guest Post by Matthew Ladner)

Over at Chamber Business News today I discuss the above map from the indespensible Garrett Archer showing that areas with large Native American populations alone voted yes on Proposition 305 (marked green). They did so despite the fact that they were already included in the ESA population and despite the fact that a credible case could be made that it was not in their own selfish interest to vote yes. There is a lesson for the rest of us in all of this.

Native Americans were not voting their self-interest in supporting ESA expansion. They were already included in ESA eligibility. Because “No” prevailed, the ESA program reverted to the pre-expansion program, an annual cap on the number of new participants cycles off in 2019. If “Yes” had prevailed an overall cap would have come into effect, and this could have limited Native American participation absent further action. Every tribal community residing student is eligible to participate in the ESA. Nevertheless, it was they that voted “Yes” to expand eligibility- why?

We can’t be sure, but perhaps these communities are better acquainted with the desperation that parents feel when their child is failing to flourish in a school. Perhaps it is more obvious from the tribal community areas that while open enrollment and charter schools are good things, they aren’t a solution for everyone. We parents in Maricopa County have a vast array of district, charter, magnet and private school options. We may have made the mistake of taking choice for granted. It may be the case that other communities have a better grounding in just how vital and precious a thing it is for families to have a chance to find a school that fits their child’s needs and aspirations.

Many of us are very fortunate with regards to the education of our children. We carefully purchase our homes with an eye to attendance boundaries. We use open enrollment, we consider magnet schools, and/or enroll our children in charter schools. Those of us fortunate enough to live in this world would do well to remember that the communities with the fewest of these opportunities voted to expand opportunity further at some risk to themselves.

A Navajo proverb holds “Always assume your guest is tired, cold and hungry, and act accordingly.” Like the widow and her mite, Arizona’s Native American communities offered what little that they had and revealed once again the great nobility of their spirit. This is an example to which all Arizonans should aspire. We should not hoard opportunity, even if it superficially seems to our own advantage to do so. Rather we should provide opportunity to everyone.

Today we are CANCELLING the APOCALYPSE!!!!!!!!!!

December 18, 2018

(Guest Post by Matthew Ladner)

Over at RedefinED I pay homage to the dumbest awesome movie of all time, or is it the most awesome dumb movie of all time? Silly me- IT’S BOTH! Oh and also there might be some discussion of why it is absurd to talk about Florida education in apocalyptic terms.

That’s no moon, that’s a Death Star Bill!

December 11, 2018

(Guest Post by Matthew Ladner)

So my son Ben had an social studies assignment to explain the legislative process. He had to build a powerpoint and then this happened. Yes it went down the y-chromosome. I got a kick out of it, figured some of you might as well.