What’s So Funny ‘Bout Peace, Rawls and Understanding Update on RedefinED

November 11, 2013

(Guest Post by Matthew Ladner)

I updated the “forced reincarnation with the chance to pick your state” thought experiment with NAEP 2013 data over at RedefinED.

Bonus Elvis C:


True vs. the Opposite of True at Ed Next

July 10, 2013

xkcd the opposite of true

HT xkcd

(Guest post by Greg Forster)

Education Next hosts a throwdown between Jay and Kate Walsh on the NCTQ teacher training standards. Backfill here.

What strikes me most about the exchange is that Walsh begins her response by essentially giving away the store to Jay:

The idea of encouraging experimentation in the education sector makes sense: if you don’t know what works, let a thousand flowers bloom. And the field of teacher education would appear to be particularly fertile ground. After all, there’s been a common presumption that no one knows what works.

Then, bizarrely, she argues that because Jay is right that we need to have more experimentation and “let a thousand flowers bloom,” we should impose regulations that restrict experimentation and limit how many kinds of flowers are allowed to bloom:

Teacher prep is the Wild West of higher education…This level of disarray raises an important question:  How much experimentation should we tolerate, given what’s at stake?…No doubt there is a difference between the kind of experimentation that Jay is calling for and teacher prep’s current modus operandi of throwing anything against a wall and seeing if it sticks—or worse, not even caring if it sticks, just doing it because a professor has decided he’s right, no matter the evidence to the contrary. But since the field itself is not rigorously gathering data on what works — and the risk for the students of new teachers is so great — it makes sense to establish reasonable guidelines as to what should go into teacher training to ensure, at the very least, that new teachers “do no harm.”

No, that’s the opposite of true. If we don’t know what works because we aren’t collecting data, and our top priority is to do no harm, the very last thing we should do is impose new regulations! The whole point of regulations is to prevent people from doing things that we know do harm. We impose lead regulations on paint manufacturers because we know putting lead in paint does harm. We impose medical trial regulations on medicine companies because we know selling untested medicines does harm. We impose broken glass regulations on fast food restaurants because we know putting broken glass in hamburgers does harm. (At least until you grind it up so fine that it’s no longer sharp, like they do in McDonald’s milkshakes.)

Imposing regulations when you don’t know what works is the quickest path to doing lots and lots of harm – and, by the way, it also prevents you from collecting data to find out what works (which is what we ought to be doing) because you can’t collect data on methods you aren’t allowed to try.


The Other (More Important) Value-Added Measure

February 26, 2013

(Guest Post by Collin Hitt)

Some teachers are better than others when it comes to raising test scores, which in turn can raise students’ earnings in adulthood. But test scores aren’t everything. A new study looks at whether individual teachers can have similar impacts on suspension rates, school attendance, GPA and even graduation rates. It finds that they can, and do.

To put the non-test score estimates into perspective, having an Algebra or English teacher 20 at the 85th percentile of GPA quality versus one at the 15th percentile would be associated with 0.09 and 0.054 higher GPA, respectively. For both subjects, a teacher at the 85 percentile of ontime grade progression quality versus one at the 15th percentile would be associated with being 5 percentage points (0.14σ) more likely to enroll in 10th grade on time. Given that not enrolling in 10th grade is a strong predictor of dropout, this suggests significant teacher effects on dropout…

That’s from Northwestern’s Kirabo Jackson, His working paper uses state-of-the-art value-added methods to identify North Carolina high school teachers who have significant impacts on test scores. He then uses the same methods to see which teachers have an impact on “non-cognitive” behaviors. One would expect – at least, I expected – that the teachers who raise test scores also raise non-cognitive outcomes. Not so.

For all outcomes, Algebra teachers with higher test score value-added are associated with better non-test score outcomes, but the relationships are weak…This indicates that while teachers who raise test score may also be associated with better non-test-score outcomes, most of effects on non-test score outcomes are unrelated to effects on test scores. The results for cognitive ability are consistent with this…

Results for English teachers follow a similar pattern. English teacher effects on English test scores explain little of the estimated effects on non-test score outcomes…

Because variability in outcomes associated with individual teachers that is unexplained by test scores is not just noise, but is systematically associated with their ability to improve typically unmeasured non-cognitive skills, classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.

So teachers can have large effects on matters that are supposedly out of their hands. Suspensions, absence rates and GPA are functions of a student’s conscientiousness – or more generally, of his character. This study delivers another blow to the cop out lobby.

But it also presents a huge challenge to the proponents of test-based teacher policies. Read this line again: “classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa.”  This is not a trivial matter. Jackson shows that non-cognitive outcomes are more strongly correlated with life outcomes than are test scores – especially for students with limited cognitive ability. This paper cannot be ignored.

Hat Tip: Joanne Jacobs

[Edited to correct formatting error and typos]


Aerial Inquiry in Texas

February 20, 2013

(Guest Post by Matthew Ladner)

A spirited debate over testing and accountability has been going on in Texas. The Texas Tribune ran the above photograph by Marjorie Kamys Cotera yesterday of the Texas Association of Business posing a rather important question via crop duster.

For those with eyesight as poor as mine, the message asks “Is 37 percent correct on Algebra too hard?” I believe it is, errr, a reference to cut scores…


Weingarten Has a Great Idea!

December 10, 2012

Lisa Simpson keep out sign

(Guest post by Greg Forster)

What a shock – Randi Weingarten wants to solve the teacher quality crisis with higher barriers to entry. Because unions never erect barriers to entry for a profession in order to fatten themselves by exploiting the weak and vulnerable.

Weingarten’s article opens with yet another sign that we’re winning: “Every profession worth its salt goes through such periods of self-examination. That time has come for the teaching profession.” Yes, it sure has!

But you know, maybe this is a good idea. Hey, Randi, how about this: we institute a bar exam for teachers and then anyone who passes the exam is allowed to teach. What do you say to that?


More Reasonable Responses to My WSJ Piece

October 16, 2012

Yesterday I chronicled the unreasonable (and unfortunately predictable)  reaction of the teachers union to my WSJ op-ed suggesting that there were trade-offs between hiring more teachers and quality teachers.  I also received a number of reasonable, but still mistaken, responses attempting to explain the 50% increase in the teaching workforce without improved results by blaming special education and English Language Learners (ELL).  A letter in yesterday’s WSJ succinctly stated the argument:

In 1970 many disabled and mentally handicapped students were denied access to public education. Today these students are guaranteed a public education until the age of 22. Also in 1970, about 5% of the U.S. population was foreign born, compared with about 20% today. Many of these children enter the education system with limited English skills and are provided services to improve their mastery of English. Such services were unheard of in many parts of the country even 20 years ago.

It is obvious from these statistics that many more special-education teachers and English-language specialists are counted in the teaching profession now as compared to 1970. Mr. Greene claims that math and reading scores of 17-year-olds are unchanged since 1970. I would submit that the teaching resources devoted to students, excluding teachers of special education and limited-English speakers, is close to unchanged since 1970.

There is a plausibility to this argument, but special education and ELL can neither account for the 50% increase in teachers nor can they be ignored when considering the stagnation in student achievement.  Special education teachers constitute about 14% of the teaching work force and disabled students constitute about 13% of the student population.  So, if we imagine, as the letter writer does, that many of these disabled students were denied access to public education, then the addition of teachers was roughly commensurate with the addition of disabled students.  Excluding all disabled students and teachers, the reduction in student-teacher ratios between 1970 and 2012 would still have been roughly from 22 to 15.  If you wanted to use as the starting point 1980, 5 years after the start of federally mandated special education, the ratio still drops from 18.6 to 15.2.

But of course not all disabled students were denied access to schools before federal legislation.  Outside of the most severely disabled, the bulk of students now classified as disabled would have been present in school in 1970; they just weren’t being served very well.  So, if we added a large number of special education teachers to better educate students who were always present but who we now consider disabled, it should have resulted in much better outcomes for those students.  But overall outcomes are flat.

There is a disturbing habit among people who make the argument represented in the WSJ letter to act as if special education is a black hole from which no progress can or should be expected.  Yes, they say, we hired more teachers, but that was for more special education students and you couldn’t expect that to result in any progress.  But this is entirely wrong.  Special education can and should result in greater academic achievement, so even teachers added in that category should be contributing to better aggregate outcomes.

All of these arguments also hold true for ELL except that ELL is much smaller and involves fewer teachers than special education.  A critic could note that the world has given the US public education system more ELL students because of higher immigration, although the same cannot really be said of special education.  Other than the exclusion of severely disabled students, whose numbers are quite small, the distribution of disabilities in the public school student population should be roughly the same today as it was back then given that most disabilities are genetic in their origin.  It’s just that we didn’t serve many of those students well in the past and therefore should expect that achievement should be rising as we devote more resources to them.  More teachers should be producing more achievement.

And yes, more ELL students might require more teachers to produce the same achievement.  But in other ways our student population has become easier to educate.  Unless students have become significantly more difficult to educate across all dimensions, it’s not possible to explain away the facts that we have 50% more teachers without any meaningful improvement in outcomes.

Several years ago Greg Forster and I addressed this in our Teachability Index, in which we tracked 16 indicators of the advantages or disadvantaged that students bring to school and found that overall students are somewhat less challenging to educate now than they used to be.  And for a forthcoming book I have updated and improved upon that analysis and still find that students are somewhat easier to educate, so it should not require many more teachers to get the same results.

We can’t blame special ed and ELL to account for the lack of productivity in education as we’ve hired more teachers.  The problem is that we’ve ignored the trade-offs between teacher quantity and teacher quality.


Randi Weingarten and Friends Respond to My WSJ Piece

October 15, 2012

I’ve long argued that the teacher unions are hardly better at running their political interests than they are at running schools.  They compensate for lousy ideas and poorly made arguments with the brute force of mountains of cash and an army of angry teachers.

My view of the teacher unions was confirmed by their mangled reaction to my piece in the Wall Street Journal noting the trade-offs between the number of teachers we hire and their quality.  The boss of the American Federation of Teachers, Randi Weingarten, tweeted her response: “They don’t want to pay teachers comp salaries…”

Now, I should say that anyone who attempts to engage in a substantive debate on Twitter is an idiot and so I fully confess that I was an idiot for trying to do so.  I responded: “we could have increased teacher salaries by 50% instead of increasing their number by that amount.”  And then I reiterated the point: “you seem to prefer having 50% more teachers over 50% higher salaries. Why is that?”

Having raised the issue, Randi Weingarten obviously had not thought through where the argument might go.  She couldn’t offer the obvious answer: “Because the teacher union cares more about power than about teachers, so having 50% more of them gives us a larger army on election day while 50% more pay might create more satisfied professionals who are less dependent on the union.”

No, thinking things through is not exactly the union’s forte.  They are more accustomed to crushing opponents with ad hominem attacks or distracting the audience with emotional and irrelevant appeals.  So, that’s exactly what they did.  Teacher union flak, Caitlin McCarthy, chimed in with: “Jay Greene shld model how an XL class size would work w/ Randi sitting in back taking notes for us. LOL.”  Randi Weingarten agreed with Caitlin McCarthy, adding to the joke: “I wld have to be in front-so I cld see the board.”

I responded that it is obviously possible to have higher student-teacher ratios since we used to have them and without getting worse results: “student teacher ratios from 40 years ago were modeled 40 years ago. If impossible how did they?”

McCarthy replied with a “these go to 11” argument, repeating that I needed to model how it was possible to have higher student-teacher ratios, tweeting: “Jay, again I suggest u actually model this & not just write/imagine it. Practice what u preach.”  This was followed by a series of tweets from McCarthy all of which were based on the notion that only teachers have standing to hold opinions about education policy.  She wrote: ” I understand & respect teaching b/c I walk the walk. I’m not all talk. Model ur ideas, Jay” and “Jay, have you ever subbed in an urban area for a wk? Not being snarky. A legit question.” and “Never take advice from someone who hasn’t been there.”  Randi Weingarten again joined McCarthy in her argument, tweeting: “Good Q Jay-have u ever taught high school in an urban/rural setting.”

I was struck by the anti-intellectualism of their line of argument.  What kind of educator would believe that the only way to know something is by having done it?  If that were true, we should dispense with schools and just have apprenticeships.  I tweeted: “so the only way to know something is to have done it? Shows no faith in abstract learning” and “As an educator you believe in abstract learning, right? Or do we only learn by apprenticeship?”

Mentioning abstract learning to the teacher union’s army of angry teachers must be like waving a red cape in front of a bull.  Caitlin McCarthy charged with all of her bovine might: “I would expect this kind of comment from an ‘abstract thinker’ out of touch w/ reality. Go sub.”

McCarthy threw in some additional ad hominem just to complete her stereotype as a teacher union flak unable or uninterested in discussing the substance of arguments.  She tweeted: “Jay was born circa ’67. He never lived firsthand the schools of yore & has a pol agenda.”  Oh, the substance of my argument can be ignored because I have a political agenda while she and Weingarten have no agenda at all other than their love of children.  And when Texas Parents Union tweeted Randi Weingarten and Caitlin McCarthy “While we wait on @jaypgreene to respond, what is your specific concern with article? Just curious…” McCarthy replied ” Hmm…u link to StudentsFirst & Stand For Children on ur site, so it’s safe to assume u agree w/ Jay?”  Never mind the argument, let’s talk about who you link to and who’s side you’re on.

I would like to think that the anti-intellectual, non-substantive, and ad hominem nature of the teacher union response was simply a function of the stupidity of trying to have an argument on Twitter.  But unfortunately, this is the main way I have seen them argue for more than two decades.  Fortunately for those opposed to the union’s policy agenda, their bullying and mangled arguments only continue to erode their credibility in policy discussions.  As I’ve said before, the teacher unions are already starting to be treated like the Tobacco Institute, a well-financed and well-organized special interest that has no legitimacy in policy debates.