(Guest Post by Collin Hitt)
Some teachers are better than others when it comes to raising test scores, which in turn can raise students’ earnings in adulthood. But test scores aren’t everything. A new study looks at whether individual teachers can have similar impacts on suspension rates, school attendance, GPA and even graduation rates. It finds that they can, and do.
To put the non-test score estimates into perspective, having an Algebra or English teacher 20 at the 85th percentile of GPA quality versus one at the 15th percentile would be associated with 0.09 and 0.054 higher GPA, respectively. For both subjects, a teacher at the 85 percentile of ontime grade progression quality versus one at the 15th percentile would be associated with being 5 percentage points (0.14σ) more likely to enroll in 10th grade on time. Given that not enrolling in 10th grade is a strong predictor of dropout, this suggests significant teacher effects on dropout…
That’s from Northwestern’s Kirabo Jackson, His working paper uses state-of-the-art value-added methods to identify North Carolina high school teachers who have significant impacts on test scores. He then uses the same methods to see which teachers have an impact on “non-cognitive” behaviors. One would expect – at least, I expected – that the teachers who raise test scores also raise non-cognitive outcomes. Not so.
For all outcomes, Algebra teachers with higher test score value-added are associated with better non-test score outcomes, but the relationships are weak…This indicates that while teachers who raise test score may also be associated with better non-test-score outcomes, most of effects on non-test score outcomes are unrelated to effects on test scores. The results for cognitive ability are consistent with this…
Results for English teachers follow a similar pattern. English teacher effects on English test scores explain little of the estimated effects on non-test score outcomes…
Because variability in outcomes associated with individual teachers that is unexplained by test scores is not just noise, but is systematically associated with their ability to improve typically unmeasured non-cognitive skills, classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.
So teachers can have large effects on matters that are supposedly out of their hands. Suspensions, absence rates and GPA are functions of a student’s conscientiousness – or more generally, of his character. This study delivers another blow to the cop out lobby.
But it also presents a huge challenge to the proponents of test-based teacher policies. Read this line again: “classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.” This is not a trivial matter. Jackson shows that non-cognitive outcomes are more strongly correlated with life outcomes than are test scores – especially for students with limited cognitive ability. This paper cannot be ignored.
Hat Tip: Joanne Jacobs
[Edited to correct formatting error and typos]
Jay P. Greene's Blog 
[…] The Other (More Important) Value-Added Measure (jaypgreene.com) […]
Could someone tease out the scholar-speak for me here? Is Jackson saying that some teachers increase kids’ learning, but other teachers who don’t perform as well as measured by student test scores do increase kids’ cognitive abilities, so it’s important not to get rid of the latter using value-added teacher evals?
Hi Joy. Jackson finds that certain teachers regularly contribute to student cognitive development, as measured by test-score growth. Teachers also regularly contribute to student “non-cognitive” development, as measured by attendance, attainment, student misbehavior, etc… The cognitive stuff involves knowing academic content. The non-cognitive stuff involves the development of character traits, like persistence, self-control, etc…
Then Jackson finds that the teachers who add the most to cognitive development are not always the teachers who contribute the most to non-cognitive development. In fact, there is relatively little overlap. So, if we build a teacher evaluation system that rewards and retains teachers based on test-score growth, we’ll be rewarding and retaining teachers who contribute to one dimension of student development, but not another. That would be a bad thing, especially given that Jackson finds that non-cognitive qualities are more strongly related to later life success, like income and employment.
The bottom line is that there are multiple dimensions to effective teaching and some teachers are stronger on some dimensions and some are stronger on others. It would be very hard to design a mechanical, top-down evaluation system that would properly identify and reward teachers who were differently effective on these different dimensions.
[…] The Other (More Important) Value-Added Measure (jaypgreene.com) […]