I’ve written before about the Character Assessment Initiative (Charassein) at the University of Arkansas, headed by my colleague Gema Zamarro. This post provides an update about the strong progress they are making with their research.
First, the paper by Collin Hitt, Julie Trivitt, and Albert Cheng that I’ve been telling you about for more than two years is now being published in the Economics of Education Review. Getting into a top journal takes time, but this paper certainly deserves high placement. In this paper, Hitt, Trivvit, and Cheng demonstrate across several longitudinal data sets that students who are more non-responsive to survey questions (skipping items or saying “don’t know”) have significantly lower educational attainment and fare less well in the labor market, even after controlling for a broad set of background characteristics and cognitive measures. Essentially, this paper validates that item non-response is a useful proxy for character skills (probably conscientiousness) and is predictive of later life outcomes.
Second, Albert Cheng and Gema Zamarro have a new paper that demonstrates that teachers actually alter student character skills. In particular, they look at data from the MET Project in which students were randomly assigned to teachers in the second year of the study. They find that teachers who themselves have weaker character skills during the first year of the study, as measured by non-responsiveness or careless responses on surveys, weaken the character skills of the students experimentally assigned to them during the second year of the study. Conversely, teachers who model higher levels of conscientiousness improve the character skills of their students.
This teacher ability to affect student character skills in not related to their ability to improve math and reading test performance. So teachers who are great at building character skills may not be the same ones who are great at conveying math and reading. Students are more successful when they learn both cognitive and non-cognitive skills. If we focus only on retaining and rewarding teachers or schools that do one, students may miss out on having teachers or schools that are good at the other.
I hope it’s not another two years until you see this new Cheng and Zamarro paper in a top journal. Now that their measure has been validated by the EER piece, future publications should come more quickly. And Albert is himself headed to great things as he completes his Ph.D. and begins a post-doc at Harvard next year. Keep you eye on him and this line of research on using survey responsiveness and carelessness as measures of character skills with strong predictive power for student success.
Jay P. Greene's Blog 
Congratulations to your team for these accomplishments!
I question the assumption that conscientiousness is the most relevant character trait. What kinds of surveys are these? Is there an obligation to answer every question? If not, conscientiousness seems like an inappropriate choice. Courage or even curiosity seem like better fits. Some character schemes talk about “zest” and as corny as that is it seems relevant here.
The lack of connection between academic measures and character measures that have been validated against life outcomes continues to be troubling. Potentially paradigm shifting in unpredictable ways.
Thanks, Greg. I agree that it is not perfectly clear what is being measured by people skipping items on surveys, saying “don’t know” on items that they should know, or answering in careless patterns. It is clear that these are predictive of long-term outcomes. And some other research Gema and her team have done shows that item non-response and careless response are correlated most strongly with conscientiousness when subjects are also given the Big 5 personality test.
But you are entirely right that our understanding of character and how to measure it is still in its infancy. And you are also entirely right that the observation that character is an outcome of education that is largely independent of what is captured by achievement tests has potentially big and unpredictable implications for how we think about education.
How strong is the link between test scores and life outcomes? I know that some link has been established, but how sure are we of it? Are we measuring the wrong things (at least “wrong” for purposes of life outcomes)?
Does character link with attainment? We do know attainment links with life outcomes, don’t we?
That’s the thing. We don’t really have much rigorous evidence linking test score changes to changes in life outcomes. The only study I can think of that does that is Chetty, et al. As I discussed in this post, https://jaypgreene.com/2015/11/14/more-on-the-over-confidence-of-portfolio-management/ , interventions that raise test scores don’t seem to do much to raise attainment or other later life outcomes while other interventions show little to no test score improvements but show large gains in attainment and later life outcomes. I’m about to have a debate with Mike Petrilli on this very topic. My claim is that test scores are not a particularly reliable proxy for teacher, school, or policy quality because of this disconnect between test score changes and later life outcome changes.
Another topic: It will be important for the research in this field not to develop a specialized jargon for talking about character that misaligns with popular meanings. I know science can’t help but develop jargon. But we’d better pay attention and not let our use of terms like “conscientiousness” drift too far away from what the person on the street means by that term. That would run up a big bill to pay later!
This is a good point, but I don’t know what words we should be using and have little control over what everyone else int he field is saying.
Of course you’re not personally to blame for what words people in the field are using. I’m just trying to name the problem so we can all be aware of it. At some point when enough data have emerged there will be a time for reevaluating terms.
Possibly the only thing that a) keeps me teaching 8th grade, and b) keeps me employed, is while my student test scores aren’t great, the subsequent success of my students in HS appears to be outstanding. The evidence is purely anecdotal (Parents and former students coming back yearly to thank me for the hell I put their child/them through), and I don’t know which side of the argument here it supports, but there seems to be a disconnect between test scores and future learning for my students.