1. Currently, the role of test scores in teacher hiring, pay, promotions, and firing is zero. How then is it “over-stressed” as an “evaluation factor”?

2. Why is it “patently unfair” to evaluate professionals based on whether their work produces the result it’s supposed to, rather than on their “behaviors”? Do we evaluate professionals in any other field this way?

3. You write that the public has not been supportive of higher funding because of the lack of a relationship between higher funding and better quality, “as revealed by a number of studies.” What measure of “quality” did those studies use? If the measure of quality was not quantitative, how did those studies measure the relationship? If it was quantitative, is it fair to measure quality using quantitative outcome measurements when evaluating the relationship between spending and quality, but “patently unfair” to measure quality that way when rewarding teachers?

4. You say that evaluators reach better agreement about the highest and lowest performers than about middle performers. How do you determine whether their agreement in these highest and lowest cases really aligns with teacher quality? If the evaluators are the only “fair” measure of quality, how do you evaluate their evaluations?

5. Since the evaluators are all themselves part of the same dysfunctional system we’re trying to improve, why should we trust their evaluations as the only permissible standard by which to improve it?

6. What reason do we have to believe that students, parents, administrators and teachers are all equally knowledgeable about what behaviors are associated with quality? In the absence of rigorous empirical research using quantitative measures, what reason do we have to believe that they can connect outcomes to specific teacher behaviors at all? If administrators and teachers already knew which behaviors worked, wouldn’t they already be promoting those behaviors?

7. Why is it “patently unfair” to evaluate teachers based on quantitative measures of the outcomes they produce, but fair to evaluate them based on whether their behavior conforms to a set of standards constructed from a poll of students, parents, administrators and teachers?

8. If students, parents, administrators and teachers fail to agree in the poll on which behaviors are desirable, who gets to decide? And to whom are those decision-makers ultimately accountable?

9. You write that when teachers are evaluated, evaluators may investigate whether the evaluatees are accurately characterizing their own behaviors “if this seems necessary.” Would the default practice be to simply evaluate teachers based on their own characterizations of their behaviors? What would constitute sufficient grounds for it “seeming necessary” to check their accuracy, and how would the check be conducted?

10. You want an evaluator to be “more of a consellor and a coach than one who appears to sit in arbitrary judgment.” Will the evaluator have the power of passing judgment on the merits of the teacher, or not?

A primary reason the public has not been more supportive of higher funding for education has been the poor relationship between better funding and higher educational quality as revealed by a number of studies.

Use of an appraisal system based upon the following guidelines should go a long way toward turning things around.

Those associated with schools, need to fairly identify true “stars” and “inadequate performers” as one of the bases for:

justifying good pay for outstanding teachers,

providing for self-guidance on the part of newcomers and present staff,

and providing an important basis for terminating those who cannot, or will not, measure up.

Research findings show that evaluators achieve much better agreement about who are Stars and Inadequate Performers than they do about who are Average, Above-Average, and Below-Average performers. Yet, placing individuals in the middle-three categories is a time-consuming, often arbitrary, and resentment-causing activity that most evaluators dislike having to do. Also, clearly, an average performer in a superior organization deserves much more recognition than an average performer in an inferior one. No wonder that many teachers and their unions oppose conventional merit-rating systems!

To avoid a popularity contest, assure greater fairness, and provide for constructive self-guidance, there should be behavioral documentation for both Star and Inadequate Performer nominations via the Critical Incident Technique.
To lay the groundwork for this, students, parents, veteran administrators, and experienced teachers should be polled at to what specific, observable behaviors they associate with outstanding and inadequate performance for each important aspect of a teacher’s job.

Then, required behavioral documentation for Star and Inadequate-Performer nominations from fellow teachers, adminstrators, students, and parents should be based upon the most agreed-upon behaviors, and the agreed-to relative weights that should be assigned to these.

The results of this analysis can also constructively guide the initial training and subsequent selection of teachers, as well as, provide a much-needed, qualifying context for the currently over-stressed evaluation factor of test-score-improvement.

This approach also sets the stage for more productive review sessions between the rater and ratee. Since the ratee has a sound basis for self-rating, the session should start with the rater asking “How do you rate yourself for this past period through the presentation of relevant, supporting behaviors?” No rater can be all-knowing, so if behaviors are mentioned that she or he is not aware of, the rater can postpone giving his or her evaluation to provide time to check out the validity of the assertions, if this seems necessary.

A sound behavioral basis for rating also facilitates the use of motivational goal setting during the review session. For example, if the ratee wants to be a Star, what specific behavioral goals does she or he plan to adopt by such and such a time? If stardom is not the goal, which specific, Inadequate Performer behaviors will he or she need to avoid?
This approach permits a rater to be more of a counselor and coach, than one who appears to sit in arbitrary judgment.

For discussion of relevant research and related citations, see: “Improving Performance Appraisal Systems” by William M. Fox, NATIONAL PRODUCTIVITY REVIEW, Winter 1987-88, pages 20-27.

William Fox
gryfox@bellsouth.net
Professor Emeritus
Department of Management
University of Florida
(352) 376-9786

I wonder if Aristotle would say that the structure of teacher pay is the “formal cause” of low teacher quality, while lack of competition is the “efficient cause.” (The “final cause” would be the mystique of public education, I suppose.)

There is no way to solve teacher pay within this system. Seniority-based systems lead to the problems listed, not allowing the best people to be rewarded.

Conversely, the lack of external competition, and the bureaucratic system of management make corruption and favoritism a significant risk for “merit” based systems. For every alternative pay system, there is way to work it, and a way that it can become counter productive with unintended consequences.

Now if there was competition, schools that used ineffective remuneration policies would loose their best staff, and customers (parents and children) would go elsewhere. Then we wouldn’t be arguing about it, the market would be busy fixing it!