The Other (More Important) Value-Added Measure

February 26, 2013

(Guest Post by Collin Hitt)

Some teachers are better than others when it comes to raising test scores, which in turn can raise students’ earnings in adulthood. But test scores aren’t everything. A new study looks at whether individual teachers can have similar impacts on suspension rates, school attendance, GPA and even graduation rates. It finds that they can, and do.

To put the non-test score estimates into perspective, having an Algebra or English teacher 20 at the 85th percentile of GPA quality versus one at the 15th percentile would be associated with 0.09 and 0.054 higher GPA, respectively. For both subjects, a teacher at the 85 percentile of ontime grade progression quality versus one at the 15th percentile would be associated with being 5 percentage points (0.14σ) more likely to enroll in 10th grade on time. Given that not enrolling in 10th grade is a strong predictor of dropout, this suggests significant teacher effects on dropout…

That’s from Northwestern’s Kirabo Jackson, His working paper uses state-of-the-art value-added methods to identify North Carolina high school teachers who have significant impacts on test scores. He then uses the same methods to see which teachers have an impact on “non-cognitive” behaviors. One would expect – at least, I expected – that the teachers who raise test scores also raise non-cognitive outcomes. Not so.

For all outcomes, Algebra teachers with higher test score value-added are associated with better non-test score outcomes, but the relationships are weak…This indicates that while teachers who raise test score may also be associated with better non-test-score outcomes, most of effects on non-test score outcomes are unrelated to effects on test scores. The results for cognitive ability are consistent with this…

Results for English teachers follow a similar pattern. English teacher effects on English test scores explain little of the estimated effects on non-test score outcomes…

Because variability in outcomes associated with individual teachers that is unexplained by test scores is not just noise, but is systematically associated with their ability to improve typically unmeasured non-cognitive skills, classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.

So teachers can have large effects on matters that are supposedly out of their hands. Suspensions, absence rates and GPA are functions of a student’s conscientiousness – or more generally, of his character. This study delivers another blow to the cop out lobby.

But it also presents a huge challenge to the proponents of test-based teacher policies. Read this line again: “classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.”  This is not a trivial matter. Jackson shows that non-cognitive outcomes are more strongly correlated with life outcomes than are test scores – especially for students with limited cognitive ability. This paper cannot be ignored.

Hat Tip: Joanne Jacobs

[Edited to correct formatting error and typos]


Aerial Inquiry in Texas

February 20, 2013

(Guest Post by Matthew Ladner)

A spirited debate over testing and accountability has been going on in Texas. The Texas Tribune ran the above photograph by Marjorie Kamys Cotera yesterday of the Texas Association of Business posing a rather important question via crop duster.

For those with eyesight as poor as mine, the message asks “Is 37 percent correct on Algebra too hard?” I believe it is, errr, a reference to cut scores…


Weingarten Has a Great Idea!

December 10, 2012

Lisa Simpson keep out sign

(Guest post by Greg Forster)

What a shock – Randi Weingarten wants to solve the teacher quality crisis with higher barriers to entry. Because unions never erect barriers to entry for a profession in order to fatten themselves by exploiting the weak and vulnerable.

Weingarten’s article opens with yet another sign that we’re winning: “Every profession worth its salt goes through such periods of self-examination. That time has come for the teaching profession.” Yes, it sure has!

But you know, maybe this is a good idea. Hey, Randi, how about this: we institute a bar exam for teachers and then anyone who passes the exam is allowed to teach. What do you say to that?


More Reasonable Responses to My WSJ Piece

October 16, 2012

Yesterday I chronicled the unreasonable (and unfortunately predictable)  reaction of the teachers union to my WSJ op-ed suggesting that there were trade-offs between hiring more teachers and quality teachers.  I also received a number of reasonable, but still mistaken, responses attempting to explain the 50% increase in the teaching workforce without improved results by blaming special education and English Language Learners (ELL).  A letter in yesterday’s WSJ succinctly stated the argument:

In 1970 many disabled and mentally handicapped students were denied access to public education. Today these students are guaranteed a public education until the age of 22. Also in 1970, about 5% of the U.S. population was foreign born, compared with about 20% today. Many of these children enter the education system with limited English skills and are provided services to improve their mastery of English. Such services were unheard of in many parts of the country even 20 years ago.

It is obvious from these statistics that many more special-education teachers and English-language specialists are counted in the teaching profession now as compared to 1970. Mr. Greene claims that math and reading scores of 17-year-olds are unchanged since 1970. I would submit that the teaching resources devoted to students, excluding teachers of special education and limited-English speakers, is close to unchanged since 1970.

There is a plausibility to this argument, but special education and ELL can neither account for the 50% increase in teachers nor can they be ignored when considering the stagnation in student achievement.  Special education teachers constitute about 14% of the teaching work force and disabled students constitute about 13% of the student population.  So, if we imagine, as the letter writer does, that many of these disabled students were denied access to public education, then the addition of teachers was roughly commensurate with the addition of disabled students.  Excluding all disabled students and teachers, the reduction in student-teacher ratios between 1970 and 2012 would still have been roughly from 22 to 15.  If you wanted to use as the starting point 1980, 5 years after the start of federally mandated special education, the ratio still drops from 18.6 to 15.2.

But of course not all disabled students were denied access to schools before federal legislation.  Outside of the most severely disabled, the bulk of students now classified as disabled would have been present in school in 1970; they just weren’t being served very well.  So, if we added a large number of special education teachers to better educate students who were always present but who we now consider disabled, it should have resulted in much better outcomes for those students.  But overall outcomes are flat.

There is a disturbing habit among people who make the argument represented in the WSJ letter to act as if special education is a black hole from which no progress can or should be expected.  Yes, they say, we hired more teachers, but that was for more special education students and you couldn’t expect that to result in any progress.  But this is entirely wrong.  Special education can and should result in greater academic achievement, so even teachers added in that category should be contributing to better aggregate outcomes.

All of these arguments also hold true for ELL except that ELL is much smaller and involves fewer teachers than special education.  A critic could note that the world has given the US public education system more ELL students because of higher immigration, although the same cannot really be said of special education.  Other than the exclusion of severely disabled students, whose numbers are quite small, the distribution of disabilities in the public school student population should be roughly the same today as it was back then given that most disabilities are genetic in their origin.  It’s just that we didn’t serve many of those students well in the past and therefore should expect that achievement should be rising as we devote more resources to them.  More teachers should be producing more achievement.

And yes, more ELL students might require more teachers to produce the same achievement.  But in other ways our student population has become easier to educate.  Unless students have become significantly more difficult to educate across all dimensions, it’s not possible to explain away the facts that we have 50% more teachers without any meaningful improvement in outcomes.

Several years ago Greg Forster and I addressed this in our Teachability Index, in which we tracked 16 indicators of the advantages or disadvantaged that students bring to school and found that overall students are somewhat less challenging to educate now than they used to be.  And for a forthcoming book I have updated and improved upon that analysis and still find that students are somewhat easier to educate, so it should not require many more teachers to get the same results.

We can’t blame special ed and ELL to account for the lack of productivity in education as we’ve hired more teachers.  The problem is that we’ve ignored the trade-offs between teacher quantity and teacher quality.


Randi Weingarten and Friends Respond to My WSJ Piece

October 15, 2012

I’ve long argued that the teacher unions are hardly better at running their political interests than they are at running schools.  They compensate for lousy ideas and poorly made arguments with the brute force of mountains of cash and an army of angry teachers.

My view of the teacher unions was confirmed by their mangled reaction to my piece in the Wall Street Journal noting the trade-offs between the number of teachers we hire and their quality.  The boss of the American Federation of Teachers, Randi Weingarten, tweeted her response: “They don’t want to pay teachers comp salaries…”

Now, I should say that anyone who attempts to engage in a substantive debate on Twitter is an idiot and so I fully confess that I was an idiot for trying to do so.  I responded: “we could have increased teacher salaries by 50% instead of increasing their number by that amount.”  And then I reiterated the point: “you seem to prefer having 50% more teachers over 50% higher salaries. Why is that?”

Having raised the issue, Randi Weingarten obviously had not thought through where the argument might go.  She couldn’t offer the obvious answer: “Because the teacher union cares more about power than about teachers, so having 50% more of them gives us a larger army on election day while 50% more pay might create more satisfied professionals who are less dependent on the union.”

No, thinking things through is not exactly the union’s forte.  They are more accustomed to crushing opponents with ad hominem attacks or distracting the audience with emotional and irrelevant appeals.  So, that’s exactly what they did.  Teacher union flak, Caitlin McCarthy, chimed in with: “Jay Greene shld model how an XL class size would work w/ Randi sitting in back taking notes for us. LOL.”  Randi Weingarten agreed with Caitlin McCarthy, adding to the joke: “I wld have to be in front-so I cld see the board.”

I responded that it is obviously possible to have higher student-teacher ratios since we used to have them and without getting worse results: “student teacher ratios from 40 years ago were modeled 40 years ago. If impossible how did they?”

McCarthy replied with a “these go to 11” argument, repeating that I needed to model how it was possible to have higher student-teacher ratios, tweeting: “Jay, again I suggest u actually model this & not just write/imagine it. Practice what u preach.”  This was followed by a series of tweets from McCarthy all of which were based on the notion that only teachers have standing to hold opinions about education policy.  She wrote: ” I understand & respect teaching b/c I walk the walk. I’m not all talk. Model ur ideas, Jay” and “Jay, have you ever subbed in an urban area for a wk? Not being snarky. A legit question.” and “Never take advice from someone who hasn’t been there.”  Randi Weingarten again joined McCarthy in her argument, tweeting: “Good Q Jay-have u ever taught high school in an urban/rural setting.”

I was struck by the anti-intellectualism of their line of argument.  What kind of educator would believe that the only way to know something is by having done it?  If that were true, we should dispense with schools and just have apprenticeships.  I tweeted: “so the only way to know something is to have done it? Shows no faith in abstract learning” and “As an educator you believe in abstract learning, right? Or do we only learn by apprenticeship?”

Mentioning abstract learning to the teacher union’s army of angry teachers must be like waving a red cape in front of a bull.  Caitlin McCarthy charged with all of her bovine might: “I would expect this kind of comment from an ‘abstract thinker’ out of touch w/ reality. Go sub.”

McCarthy threw in some additional ad hominem just to complete her stereotype as a teacher union flak unable or uninterested in discussing the substance of arguments.  She tweeted: “Jay was born circa ’67. He never lived firsthand the schools of yore & has a pol agenda.”  Oh, the substance of my argument can be ignored because I have a political agenda while she and Weingarten have no agenda at all other than their love of children.  And when Texas Parents Union tweeted Randi Weingarten and Caitlin McCarthy “While we wait on @jaypgreene to respond, what is your specific concern with article? Just curious…” McCarthy replied ” Hmm…u link to StudentsFirst & Stand For Children on ur site, so it’s safe to assume u agree w/ Jay?”  Never mind the argument, let’s talk about who you link to and who’s side you’re on.

I would like to think that the anti-intellectual, non-substantive, and ad hominem nature of the teacher union response was simply a function of the stupidity of trying to have an argument on Twitter.  But unfortunately, this is the main way I have seen them argue for more than two decades.  Fortunately for those opposed to the union’s policy agenda, their bullying and mangled arguments only continue to erode their credibility in policy discussions.  As I’ve said before, the teacher unions are already starting to be treated like the Tobacco Institute, a well-financed and well-organized special interest that has no legitimacy in policy debates.

 

 

 


WSJ Op-Ed — We Don’t Need More Teachers

October 9, 2012

image

I have a piece in today’s Wall Street Journal about how the solution to our education problems can’t be found by hiring more teachers.  We need quality over quantity, for which we will have to pay the teachers we do have more.  And more importantly, we need to substitute technology for labor in education like we have in almost every other industry to improve productivity.  But in public education we have been doing the reverse, hiring more, lower quality teachers and failing to develop and implement cost-effective technology.

I know all of this is well-worn territory, but given that both presidential candidates endorsed the idea of hiring more teachers the editors at the WSJ thought it was important to emphasize the point.


Gates Gets Groovy, Invests in Mood Rings

June 19, 2012

Building on their earlier $1.4 million investment in bracelets to measure skin conductivity (sweating) as a proxy for student engagement, the Gates Foundation has decided to embark on a multi-million dollar investment in mood rings.

As you can see from their research results pictured above, the mood ring is capable of identifying a variety of student emotional states that could affect the learning environment.  Teachers need to be particularly wary of the “hungry for waffles” mood because it is sometimes followed by the “flatulence” or “full bladder” mood.

Besides, mood rings are pretty groovy.  And they can’t be any dumber than these Q Sensor bracelets.


Follow

Get every new post delivered to your Inbox.

Join 2,856 other followers