(Guest Post by Matthew Ladner)
Michael Lewis strikes again with a must read article about Shane Battier, the greatest professional basketball player you’ve never heard of because all he does is help his team win games. The article is Moneyball for the NBA, but with several twists- most prominently some very nasty individual versus team dynamics. In short, in baseball, you essentially can’t aggrandize yourself as a player without also helping your team. If you are getting on base, you are padding your stats and helping your team win.
Not so in basketball, where you can get paid millions for padding your individual stats whether or not you help your team win games. An example raised in the article: NBA players don’t like to heave the ball at the end of the half or game because it lowers their percentage. In short, basketball is fraught with perverse incentives, making it much more like most of real life than baseball. The would be sabremetricians of the NBA have only begun to sort through this quandry.
Battier provides Lewis the perfect lense into this world, as a player that simultaneously has statistics that stink and is one of the most valuable players in the league.
Is there an education angle here? Yes indeed. Battier is what business guys call a “white space” employee. The term refers to the space between boxes on an organizational chart. A white space employee is someone who does whatever it takes to achieve organizational goals and makes the organization work much better as a whole.
As we move into the era of value-added analysis for teacher merit pay, this article provides much food for thought. School leaders must consider carefully what they will reward, and give some consideration to how white space behavior is rewarded. Rewards should not just be based on individual learning gains- reaching school wide goals should also be strongly rewarded. Otherwise my incentive as a math teacher will be to assign six hours of math homework a night- and to hell with everyone else (see Iverson, Allen).
Schools are more complex social organizations than basketball teams, so education sabrematicians have a great of work ahead of them. The good news however is that it can’t be hard to improve a system that generally only rewards teachers for length of service and often meaningless certifications and degrees.
There’s no reward for being a white space player OR a superstar in the current system of teacher compensation-just an old player. Imagine a system of compensation for the NBA in which Larry Bird was still riding the pine on NBA squads and getting paid more money than LeBron, Kobe or Battier. Hall of Fame = National Board Certified, but you no longer want Bird in the game if you want to win.
You wouldn’t need to be Bill James to figure out how to make such a system much more effective. Figuring out the right way to reward all the little invisible things that someone like Shane Battier does to make his team win, well, that’s trickier. Overall we have nowhere to go but up, however. Remember both LeBron and Battier are multi-millionaires, while their equivalents in the teaching world have all too often left the profession in frustration or gone into administration.
Jay P. Greene's Blog 
[…] Bugs & Cranks wrote an interesting post today on The No Stats All StarHere’s a quick excerptIn short, in baseball, you essentially can’t aggrandize yourself as a player without also helping your team. If you are getting on base, you… […]
This is a great post.
This is a great post and reminds me of a piece by Malcolm Gladwell. I make that parallel explicit at http://schoolboardtransparency.org/2009/02/20/pro-sports-statistics-and-how-we-pay-teachers/.
[…] Via NBA forward and former ACC standout Shane Battier (and Michael Lewis) Matt Ladner takes us to white space employees and the opportunities and challenges of rewarding performance in e… […]
This post brings up some extremely important points for policymakers to consider. I think most reasonable people would concur with the statements made by Matthew.
However . . . this is similar to the innumerable articles, posts, speeches, etc. about changing how we compensate teachers. At the end, because the idea is sold as being a part of mom, baseball, apple pie, and the American Way that everyone–especially policymakers–jumps on-board.
The problem arises when policymakers rush such suggestions into actual policies without thinking very carefully about how we can accurately measure value-added and the contributions of white space employees. The measurement problems inherent in these ideas are ginormous, yet policymakers care more about being first on the bandwagon rather than doing things slowly and thoughtfully–in other words, doing things in a way that actually works.
Anyway–your post has good points, but the devil is in the details of how you would measure white space employee contributions other than school-level achievement.
Ed-
No doubt about it- it is tricky. My ideas broadly are to make school wide goals just as important as individual teacher gains, and to retain a principal review as a part of the process.
Of course, principals should also be judged based upon the overall gains of their schools. When they are, principals will have a strong incentive to reward white space employees.
Jeff Pfeffer of Stanford Business School gave some testimony to Congress a couple of years ago about why individual pay-for-performance is very unlikely to be successful, whereas team-based performance management systems work quite well.
http://www.evidence-basedmanagement.com/research_practice/commentary/pfeffer_congressional_testimony_08mar2007.html
Might be useful input to this topic.
Matthew, I read the article on the way home from work, then read your post. I am amazed at what sabremetricians have been able to do with sports. When they’re on their game, they can make statistics exciting like nobody’s business.
Still, two limitations stuck out to me from the article. As you point out in your article, basketball is fraught with perverse incentives. I’d argue that baseball is, too, though not it’s not as obvious. Michael Lewis has written about how, for example, OBP is a far more meaningful stat than batting average, but that’s not what we reward. Also, baseball has white space players just like basketball who make their teams better in less than obvious ways. My point is, wherever there is such a heavy emphasis on stats there will be perverse incentives.
And while these perverse incentives exist, Battier says he uses the stats to prepare because “the numbers don’t lie.” What he really means, I think, is that the RIGHT numbers won’t lie to him. He knows what to look at. It’s critical to measure what matters.
And while Battier has figured out how to the right stats to his advantage, no one has figured out how to measure Battier’s effectiveness. Throughout the whole article, the only way that Lewis can describe what Battier does to help his team win is through qualitative description.
So while I strongly agree with you that it’s vital to find ways to figure out how teachers contribute to the entire school environment, I’m not convinced that we have nowhere to go but up. I imagine that you would liken schoolwide gains for a principal to wins for a coach. But tests can also provide a perverse incentive for schools to emphasize test prep and de-emphasize the arts and gym. I don’t mean to rehash old arguments, but it’s an important one.
The other thing is that I’m not sure if we can really measure what makes teachers great quantitatively. After all, sabremetricians haven’t figured out how to do the same for Shane Battier.
Sam-
You are right, but solutions are being worked out.
Some of the online platforms for school data management are taking value added analysis to a whole new level. Monthly assessments gets the error down, so that you can do the analysis within the span of a single year (rather than the three years used by the early Sanders model).
Better yet, some of these platforms allow teachers to develop their own assessment items in professional learning communities based on state standards.
In other words- if you’ve got state standards, you can do value added with it- on any subject, not just math and reading. Furthermore, you can do it in such a way that the teachers own the assessment items.
Unfortunately, the fear that underlies excuses for avoiding performance metrics rests on the red herring assumption that such metrics, once used, are set in stone. It’s understandable how this kind of thinking can arise among those who have worked in government or under union contracts for too long.
But in the real world, employee evaluations in all non-union dominated industries are in a constant state of flux. They naturally evolve and improve every year. Even the comprehensive 360 degree evaluations so popular today will eventually give way to new innovations. The important thing in education is not to wait until performance metrics are perfect, but just get started so that the refinements can begin to develop.
Okay, but what about the fact that qualitative descriptions are so vital to understanding Battier’s effectiveness? Do you see room for any qualitative component to future teacher evals?
Also, articles like this one make me skeptical of the superiority of employee evals in the “real” (i.e., business) world.
http://www.nydailynews.com/money/2009/02/11/2009-02-11_bonuses_not_my_call_says_bank_of_america.html
Jeff-
That is a very interesting link, and reinforces some of the difficulties. I agree with Daniel that we need to be (hopefully thoughtfully) experimenting with pay for performance schemes and to see where it goes. It may be the case, that as your article suggests, that collective goals should completely predominate. We won’t know until we try.
Much of the potential gain to be had in reforming teaching compensation lies in reforming the compensation of the teaching profession lies in the the quality of candidates that the profession is able to attract. South Korea pays their 15 year veteran teachers about 2.5 national GDP per person, while in the U.S. that figure is 1.12.
South Korea recruits their teachers from the top 5% of university graduates, while we recruit from (on average) the bottom third. South Korea generates the higher pay for teachers with larger class sizes, and often comes out on top of international test score comparisons:
On the question of how you evaluate teacher merit on qualitative issues, the answer is: school choice!
Average score, reading literacy, PISA, 2009:
[United States, Asian students 541]
Korea 539
Finland 536
[United States, white students 525]
Canada 524
I agree, Sam, that a qualitative component should be factored and weighted in addition to quantitative data in any evaluation for teachers. That’s what 360 degree evaluations currently do in other professions, as well as in many charter schools and most private schools — quite similar to the concepts Matthew is proposing. Ultimately, the exact mix of variables considered should be feedback-driven joint decisions by those closest to the teachers, and critically important, stakeholders with skin in the game. Through a variety of channels, this would include not only principals and education boards, but parents and taxpayers too.
As a founding board member of a charter high school a few years ago, we learned a few things through experimentation. This more direct and personally connected level of accountability I’m speaking of is what creates what I would call a “culture of responsiveness” found in these school workplaces. Without it, the school environment can drift toward the culture found at your local Department of Motor Vehicles.
[…] Thinks Anew Posted in February 26th, 2009 by admin in Teacher Magazine No Tags Am I dreaming? Does Jay Greene now believe that standardized test scores (and value-added analyses) alone cannot be used to judge […]
[…] the teachers’ own class? This essential question was brilliantly posed by Matthew Ladner at Jay Greene’s blog last […]
[…] performance pay really improve teaching? Check out Mathew Ladner’s blog post “The No Stats All Star” connecting Teaching to success in the N.B.A., by discussing the idea of a “white space […]
Matthew L –
There’s a lot more research on the relative merits of performance pay than you imagine. This is from another Pfeffer piece:
“Merit pay for teachers is an idea that is almost 100 years old, and that has been subjected to much research. In 1918, ‘48 percent of U.S. school districts sampled in one study used compensation systems that they called merit pay.’ The evidence shows that merit pay plans seldom last longer than five years and consistently fail to improve student performance.
“Merit pay also backfires because it sends the message that all that matters is students’ test scores. Economists Brian Jacob and Steven Levitt have shown that the greater the incentive pay for enhancing students’ scores, the more teachers and students cheat. Anthony Bryk, a prestigious educational researcher, tells us that the same problems emerged when merit pay systems were implemented in the 1980s. Bryk observes, ‘It is like policymakers suffer from amnesia.’”
http://www.ssireview.org/site/printer/act_on_facts_not_faith/
Matthew wrote:
Of course, principals should also be judged based upon the overall gains of their schools. When they are, principals will have a strong incentive to reward white space employees.
This is the most interesting element in this conversation to me, simply because it rings true. If principals are held accountable for gains, they’re going to have to work a whole lot harder than they currently are to figure out which “white space” employees are having the most impact, aren’t they?
In my experience, this has always been a barrier—many principals rarely understand the impact of individual faculty members beyond the superficial “this guy’s a team player” conclusions that they draw based on gut reactions.
Now for the real question: Do most principals have the skill and ability to identify white space impacts?
I’ve certainly never worked for a statistician!
Bill
[…] The No Stats All Star raises a key point in evaluating teachers…and basketball players. […]
[…] in the classroom ever to stumble out of a right-wing chop shop—the other day and came across this interesting bit by guest blogger Matthew […]