Jay P. Greene's Blog

My former students, Dan Bowen and Albert Cheng, have a new study that was just published in the Journal of Catholic Education on how religious priming may affect student character or non-cognitive skills.  They find that priming students to think about religion increases students’ willingness to delay gratification as well as their political tolerance.  No effects are observed if students are instead primed to think of secular success.  This work suggests that there may be particular benefits from religiously-based education that are more difficult to produce in a secular context.  Abandoning private religious education for secular charter schools may come at a cost to these character skills.

These results come from an experiment we conducted at the Arkansas School for Mathematics, Sciences and the Arts (ASMSA), a public boarding school in Hot Springs.  In the experiment we randomly assigned 180 students to one of three conditions.  All students were asked to work on a sentence scramble exercise in which there are ten sets of five words.  Students were asked to look at each set of five words, drop one word, and then make a sentence out of the remaining four words.

In 5 of the sentences one word was altered, changing only five of the 50 words across the three conditions.  One group was primed to think about religion by having the words “worship, preacher, heaven, devotion, and commandments” included in the sentence scramble.  A second group was primed to think about secular equivalents: “honor, leader, success, commitment, and expectations.”  And a third group saw neutral words: “eat, path, man, cabbage, and numerous.”

Prior research had found that students primed in this way to think about religion demonstrated higher levels of self-regulation.  The idea of this experiment was to attempt to replicate those previous findings while exploring whether secular equivalents could produce similar effects.  Several observers have noted that KIPP and other high-achieving charter schools appear to simulate the religious rituals of Catholic schools but replace religious rhetoric with talk of secular success and achievement.  The question this study explores is whether talk of secular achievement appears to be as motivational for students as religious rhetoric.

Dan and Albert find that something is lost when we substitute secular aspirations for religious ones.  Students exposed to the religious priming expressed a stronger feeling of religiosity.  So, the priming worked in getting students to think about religion, even though changing only 5 words out of 50 is very subtle and the students were not consciously aware of the nature of the manipulation.

Students exposed to this religious priming experienced an increase in delayed gratification in that they were more willing to receive $6 the following week as compensation for participating in the study rather than $5 right then.  Students in the religious priming condition were also more likely to express political tolerance on the Sullivan scale, which measures people’s willingness to allow disliked groups to engage in political activities, like holding rallies, having books in the library, or running for office.  But students exposed to the secular success priming were no different from the neutral priming control group in that both were less likely to delay gratification or express political tolerance.

This was a small scale experiment and the effects were observed at p<.1, so there are limits to how confident we should be about the results.  But given that the results are consistent with prior research, we should have some concerns about dropping religious school choice in favor of secular charter schools.  This is especially so given that the No Excuses charter model that has become the darling of ed reformers often comes up short at improving later life outcomes, while private school choice programs seem to fare better at improving high school graduation, college enrollment, and even earnings.

This year’s set of Al Copeland Humanitarian Award nominees was particularly strong, making selection of a winner exceptionally difficult.  As Greg noted in a comment, “this is clearly a ‘political’ year for The Al, in the sense that we’re all nominating witnesses against injustice rather than the creative entrepreneurs who usually dominate.”

Well, almost all.  Matt, as is his habit, nominated the entrepreneurs, Tim and Karrie League, who developed the Alamo Draft House chain of movie theaters.  The Alamo Draft House is one of the greatest places on earth.  The theaters carefully select movies, audience activities, food, and drink to create a completely engaging and entertaining experience.  Some people give hundreds of millions of dollars to art museums that fail to package their offerings nearly as well as Tim and Karrie League do.  And the Alamo does it without any donations while making a profit.  Improving the human condition while also making profit is a quintessential characteristic of winners of The Al.  And I almost slected Tim and Karrie League for this honor.

But as Greg said this seems like a political year in which selecting a traditional entrepreneur-type as the winner just didn’t seem right.  All of the other nominees fell in the “witnesses against injustice” category and with so much injustice all around us, I felt like I should choose one of them.  I could have chosen Jason’s excellent nominee, Remy Munasifi, whose musical parodies expose and help rebut oppression, hypocrisy, and other types of foolishness.  I could also have chosen my own nominee, Yair Rosenberg, whose trolling of neo-Nazis and other anti-Semites on Twitter deprives these bullies of the sense of power that drives much of their behavior.

Instead, I have chosen Master Sergeant Roddie Edmonds over Remy Munasifi and Yair Rosenberg because Edmonds was more than a witness to injustice.  He actively took steps, at enormous danger to himself, to promote justice in the world.  By refusing to comply with Nazi orders to separate Jewish POWs and insisting that he and all of the soldiers under his command were Jewish, Edmonds risked being shot to defy the Nazi’s hateful and murderous plans against Jews.

I hesitated for a moment in selecting Edmonds only because The Al does not typically go to people who have been widely recognized elsewhere, like Steve Jobs or John Lasseter, and Edmonds was recently honored as one of the Righteous Among the Nations at Yad Vashem, Israel’s Holocaust museum.  Unfortunately, being honored by Yad Vashem does not constitute being widely recognized, so drawing more attention to Edmonds seems important and fitting.

As much as I love Remy Munasifi and Yair Rosenberg, mocking injustice on the internet just isn’t enough.  As Ken M, last year’s winner of The Al, taught us, social media is a pretty useless forum for trying to improve the world. So, that silly video you shared on Facebook or that sly remark you made on Twitter doesn’t really do much other than amuse you.

There’s nothing wrong with some amusement. After all, that is the Prime Directive of this blog — to amuse ourselves rather than to change the world.  And being amusing is a lot better than those insufferable political rants or self-righteous internet petitions, which are all talk and no action.

If you want to fight injustice you can’t really do much with a blog, Twitter, or Facebook.  You need to find real injustices, not trumped-up (pardon the pun) minor slights like:

And then you need to follow the example of Master Sergeant Roddie Edmonds and take action that might even put yourself at risk.  Evil will always remain in the world, but we will suffer less from it if we have more people like Master Sergeant Roddie Edmonds.

When we examine the results of standardized test scores we typically think we are seeing evidence of what students know.  As it turns out, that is only partially right.  Test scores capture both what students know as well as their willingness to exert effort to show us what they know.

A new paper by my colleagues, Gema Zamarro, Collin Hitt, and Ildefonso Mendez uses multiple, novel techniques to demonstrate that between 32% and 38% of the variation in PISA test performance across countries can be explained by how much effort students are willing to exert rather than what they know.  The implications of this finding for ed reform are huge.  When we see low test score performance we are often misdiagnosing the problem as poor content instruction when it may in fact be insufficient development of student character skills.  If we focus all of our energy on the former without addressing the later, we’ll fail to make as much progress.

So, how do Gema, Collin and Ildefonso know that between 32% and 38% of variation in PISA test performance across countries is explained by effort?  They used three different methods to measure the influence of effort.  First, they took advantage of the fact that the order of questions without the PISA was randomly ordered.  They then compared how well students performed on the first set of items relative to the last.  Because the order of items was randomized the first and last questions were, on average, of equal difficulty.  The decline in getting items correct from the start to the end of the exam is therefore a function of the decline in effort students are willing to exert, not the difficulty of the items.

If you compare performance in the US and Greece (as can be seen in the figure above), students in the two countries do about as well at the beginning of the test.  That means that students in Greece and the US know about the same amount of stuff.  But students in Greece decline much more rapidly across the test, which means that those students are less willing to exert consistent effort.  When we compare PISA results from the US and Greece we wrongly conclude that content instruction in Greece must be much worse.  In reality, Greek students know as much as students in the US but simply exert a lot less effort.

A second way the paper measures effort is by examining responses students gave to a survey that was administered at the same time as the PISA.  Using novel techniques that have been validated in previous research, they measure the extent to which students skip answers (or say “don’t know”) as well as the extent to which students give careless answers as proxies for their effort.  Both skipped answers and careless answers yield very similar results to what they find from the decline across the test.

Some people have expressed skepticism about the focus on “non-cog” or character in education research because they believe that these capture personality traits that are largely inherited and immutable.  This research contradicts that claim.  Unless we think there are big and important genetic differences across countries, the variation in effort across countries has to be explained by factors that are social constructed and, at least in theory, could be changed.  In addition, great work by Gema and Albert Cheng has found that student effort can actually be changed when students are randomly assigned to different teachers who themselves possess different character skills.

The evidence is becoming clear that character matters and is subject to influence by the education system.

(Guest Post by Matthew Ladner)

Over the last couple of weeks we’ve learned a lot about American politics we would have preferred not to know. We can start with highly placed DNC operatives recruiting and training mentally ill homeless people to bait attacks at Republican rallies. Both of these people were caught on tape expounding monologues like a James Bond super-villain explaining their nefarious plot to ruuule ze vorld. One of them was at pains to explain just how good he was to also see to the medical and legal bills of his goons- it was very touching. The phrase “theoretical conversations” has been trotted out in an incredibly lame attempt at spin, and both men have been fired from their positions.

As the week went on, wikileaks revealed additional details like the DNC chair being provided debate questions in advance, reporters pre-clearing stories with the DNC, and more “pay for play” business from the Clinton Foundation that Bob Woodward described as not just unseemly but corrupt.  So the election is rigged right?

Wrong.

Repulsive as these things are, a Presidential election is not Ivy League football. A Presidential election alas more closely resembles SEC football, where the unofficial conference motto should be “If you a’int cheating you a’int tryin!” People age into political awareness and learn about how bad or corrupt things currently stand, and assume that things were much better in the past. Read enough history however and you’ll learn that there is nothing new under the sun. As utterly repulsive as it is to send mentally ill people into rallies in an effort to bait violence, let’s face it there are good ways to deal with this tactic, and bad ways to deal with it. Option one would be to inform the crowd that agitators are in your midst and want to lure you into attacking them, but that you should refuse to take the bait and instead alert security, who can escort them out. Option two would be to encourage people in the rally to punch these people and offer to pay their legal bills…which help me out here but lies in the same ethical neighborhood as the agitators does it not?

As far as I can tell, the election is not even remotely close. If it is remotely close the Clinton campaign team is utterly incompetent in moving large amounts of resources into a style point state like Arizona, which Romney won on his way to a decisive electoral college loss in 2012. There are some national polls that have the race close, but others have Clinton winning in a blowout. I don’t know enough about polling to discern which polls are more credible, but I do know that we elect the President in 50 separate state elections rather than in a national election and that the Clinton campaign is behaving as if the latter scenario is in play. Republicans complain of dirty tricks and MSM bias, both of which are real. Note however that the media today is far more pluralistic than in the past, and that Democrats would be happy to provide their own laundry list of perceived dirty tricks played by Republicans. Don’t complain too much Vanderbilt fan, it’s your choice to play in the SEC so pull up your big boy pants.

I found this piece by David French to be far more disturbing. There is a naive, hopeful part of me that wants to believe that what French describes in this column is another dirty trick by some goon squad black op outfit, or the Russians, or well, anyone other than a disturbingly large portion of my fellow Americans. Sadly at the moment I have no evidence that would support such a belief.

Neither party has taken Uncle Sam’s looming insolvency remotely seriously. To these eyes, one major party fought off a socialist insurgency with dirty tricks by the hair of their chinny chin chins so they could at least stick with a corrupt royalist, who would have lost in landslide if not for the folly of her opponents. The other fell under the sway of a nativist demagogue despite the obvious disaster that would ensue. Personally I’m not anxious to reenact the Spanish Civil War in American politics, and I am not falling for the “you have to support the fascists because the communists are even worse” trick.

Put me down for none of the above.

Massachusetts voters will be deciding in a few weeks whether to expand charter schools in the state. By all rights, the measure should be winning by a landslide.  Rigorous evaluations of existing Boston charters show large test score gains.  Charter supporters are spending millions to blanket the airwaves with ads. And following what appears to be the new ed reform ideal model, Massachusetts charters predominantly serve highly disadvantaged communities, so they have positioned themselves as the progressive promoters of social justice.

Despite all of this, Question 2 is trailing by double digits in recent polls and appears headed for defeat.  Why?  As I’ve written recently, ed reformers appear to have become so obsessed with social justice virtue-signaling that they’ve forgotten how politics actually works.  Narrowly targeting programs toward disadvantaged communities leaves programs politically vulnerable to harmful regulation, restriction, or repeal.  As much as disadvantaged communities desperately need education improvement, they tend to be poorly positioned to advocate for those efforts politically.

If you want to help the poor, you should design programs that include the middle and upper-middle classes.  This is the political genius of Social Security.  It is extremely effective at alleviating poverty among low-income seniors because high income seniors, who tend to be better positioned for political advocacy, also get it.  This is the political genius of many college subsidy efforts — the poor can benefit from them because wealthier families are also eligible.

I understand that Question 2 is attempting to expand charters so that they can include more middle class and upper-middle class families, but those voters are unaware of how charters might benefit them because already existing Massachusetts charters have largely failed to serve them.  And the unions and their local suburban school officials are doing a great job of scaring suburbanites about how a charter expansion might harm the relatively good arrangements they currently enjoy.

Charters in Massachusetts would have been better positioned politically if they had not previously neglected to benefit more middle and upper-middle class families.  Then more politically-advantaged families could have learned about benefits from their own experience and through networks of family and friends.  Ed reform needs to win by convincing middle and upper-middle class families that they can benefit themselves from creating and expanding ed reform programs.  Trying to obtain their support by arguing that poor and minority families who live somewhere else would benefit is a losing strategy.  Most political decisions are driven by crude calculations of self-interest, not high-minded appeals or guilt.

But ed reformers appear to have taken such a strong aversion to the rough political realities of self-interest that guilt is their dominant political message.  For example Richard Whitmire attempted to shame suburban voters, tweeting to an approving chorus of progressive reformers: “All comes down to: Will well-off suburbanites deny better schools to urban parents?”  The fact that Question 2 is trailing significantly in the polls provides the obvious answer to Whitmire’s query — of course suburbanites will deny better schools to urban parents if they think they might lose something and have little to gain.  That’s how politics works.

Until ed reformers temper this antipathy towards more advantaged families, abandon guilt-driven political appeals, and embrace the political realities of self-interest they should expect to continue suffering a series of political defeats.