Where the Teacher Unions will go

Here's a little bit from Professor Gary Anderson of New York University that I found at the Huffington Post.   Where might the future of education and unions lie?   According to Anderson, with Peer Asssitance and Review (PAR):

So teachers unions must continue to defend teachers' wages, benefits, and pensions, but they will be vulnerable to attack if they allow themselves to be defined as roadblocks to innovation and protectors of bad teachers. Perhaps more ominous, many working class Americans who failed to protect their private sector unions are now turning on teachers, whom they view as overpaid and with fat pensions.

One approach that has begun to change the image of teachers unions is Peer-Assisted Review or PAR. It involves peer evaluation of teachers that addresses teacher induction and development as well as due process issues. The peer evaluator and teacher meet with a board made up of district administrators and union officials who ultimately decide on the outcome of the peer-evaluation. This system not only allows for a more collegial form of evaluation and support, but also involves both the district and the union in decisions about teacher quality. This system has also been shown to be more effective at identifying incompetent teachers and either making them better or moving them out of the system. More in depth information on PAR can be found in Jennifer Goldstein's book, Peer Review and Teacher Leadership: Linking Professionalism and Accountability, and on the Project on the Next Generation of Teachers.

PAR may be the future, but the steps taken so far in Montgomery County fall woefully short of an effective system.   The foundation is in place, but we have not taken the next steps to strengthen it.   The best way to stregthen it would be for Montgomery County co cut administrators out of the evaluative process altogether.   They are unable to offer supports that teachers need.   They are unable "to show" teachers how to teach.      They are unable to use the PAR process as it was intended, because they find it too tedius to place teachers in the system.   The system needs to be implemented by teachers from beginning to end.  It needs to be  more complete and continuous.  Trust me when I say this: if teachers were in charge, the evaluation process would be far more effective than it is now.

New Evaluation?

Want to really close the achievement gap?

Want real school reform?

Identify ineffective teachers (a small minority)  and replace them,   due process provided, with effective and  instrinsically motiavated teachers (a large majority).

Is that what happened at Broad Acres Elementary School?

The Board of Education has the power to do this.  From the MCEA teacher contract:

 Process for Changing the Evaluation System: The current unit member evaluation system, including the instrument and the teacher evaluation system booklet of the Montgomery County Public Schools, Rockville, Maryland, (also known as the Professional Growth System Handbook) shall not be changed without following the procedures set out below:
1. Preceding the proposed implementation of any proposed changes, the Board shall notify MCEA of its desire to change the evaluation system.
2. Thereafter, the parties shall confer in good faith over the content of any proposed changes in the evaluation system, until agreement is reached, or until 90 days following receipt by MCEA of notification that the Board desires to change the evaluation system. The conferring teams shall be headed by the chief negotiator for each party.
3. If no agreement is reached within 90 days following receipt by MCEA of notification that the Board desires to change the evaluation system, the Board may unilaterally implement changes in the evaluation system.

We need Board members who are willing to question what's best for education in Montgomery County- not rubber stamp the latest program.   Build and attract the greatest professional staff of teachers in the United States- and watch real reform happen.   Not a pipe dream.   A decision.

The Game of Teacher Evaluations

Game theory is a way to help us understand how rational individuals and groups make decisions. With broad applications from sociology to economics, I thought I'd give its application a try in the world of teacher evaluations, specifically, to help explain why so few teachers in MCPS are placed into Montgomery County's Peer Assistance and Review program, a program to support struggling teachers who show improvement and dismiss struggling teachers who do not.


The following application of game theory is based on several assumptions. And admittedly, assumptions, like the housing market will always go up, can be dangerous. I also take a complex issue, which involves many variables, and make it simple. But my hope is that even a crude analysis can provide some insights into the problem with evaluation systems in education as they exist today. One important side note: while game theory may help explain why most people act a certain way most of the time, it certainly cannot explain why all people choose to act in the way they do. Now to the good stuff.

In game theory, a payoff matrix is used to help understand the possible outcomes that individuals or groups face given two choices. The figure above is such a matrix. It works under the presumption that people will consider their payoffs when making decisions, which in turn helps us understand why many rational people, faced with the same set of circumstances, make the same decision. The figure above, is the payoff matrix for the choices faced by evaluators and ineffective teachers, to work hard (max effort), or to take it easy (min effort).

Let us first consider the evaluator, the person who is primarily responsible for determining whether a teacher receives a satisfactory or unsatisfactory evaluation. The evaluator has two choices. The first is to exert maximum effort, and thus proceed to document and place considerable time and effort in the process of placing the teacher into the PAR system. The second choice, is to exert minimum effort. In this scenario the evaluator does not place the struggling teacher in the PAR system, but enjoys the added benefit of more time to address other issues. In the payoff matrix, we assume that a rational actor is interested in making their life simpler, and thus minimizing effort. I assign no value to exerting maximum effort, but a (+1) value to exerting minimum effort. I assign additional value to removal of the struggling teacher. If the evaluator is able to remove the teacher through placement into the PAR system, I assign a (+1) value, but if the teacher is not removed, I assign a value of zero. This leaves the following payout structure for the evaluator:


0 = Considerable effort , but no teacher removal

1 = Minimum effort, and no teacher removal

1 = Considerable effort, and teacher removal

2 = Minimum effort, and teacher removal



The ineffective teacher has slightly different payouts, but the same choices. Like the evaluator, the ineffective teacher may exert either maximum or minimum effort, but would prefer to minimize effort. Like the evaluator, I assign a (+1) value to minimizing effort and no value to maximizing effort. On the other hand, the teacher has an incentive to keep his or her job. When he or she keeps their job and thus receives scheduled step increases, I assign a value of (+1). However, because of the negative consequences associated with losing one's job, I assign a (-1) value to this outcome. This leaves the teacher with the following possible payouts:

-1= Considerable effort, and loss of job

0 = Considerable effort, and same rate of pay

0 = Minimum effort, and loss of job

1 = Minimum effort, and same rate of pay

Analysis of the payoff matrix allows us to understand why so few teachers end up dismissed from their job, and why a new system is needed that changes the payout structure for both teachers and evaluators.

Evaluators would like to remove ineffective teachers, but they understand this requires considerable effort. Let's assume evaluators don't mind this extra effort, but only if this effort pays off in the removal of the ineffective teacher. Once the evaluator places the teacher in the PAR system however, the tenured teacher has two options. The first option is to continue to place minimum effort into his or her job. However, already in the PAR system, this option will result in his or her dismissal. There is now a substantial carrot for the ineffective teacher to put in more effort in order to avoid dismissal. The result of the evaluator exerting maximum effort, is thus a payout value of zero for the evaluator, who faces a now motivated teacher willling to do what is necessary to keep their job.

Faced with the likely consequence that a teacher will not be removed even if he or she is placed in the PAR system, the rational evaluator is more likely to put forth minimum effort. In other words, the evaluator will not take the time consuming steps of putting the ineffective teacher in PAR. In this scenario, the ineffective teacher is not placed in the PAR system, but the evaluator does not have the headache associated with documenting the placement. The evaluator's life has become substantially easier. The rational ineffective teacher now has the option to either exert maximum effort to become a better teacher or to exert minimum effort, but devoid of consequences for exerting minimum effort, will likely make this choice.

In game theory, we call the above solution, the solution in which each participant acts in their own self-interest (regardless of the decision made by their "opponent"), the Nash equilibrium. In our game, this means both the evaluator and the ineffective teacher will choose to put forth minimum effort. This of course, has a negative consequence for the state of education as a whole, despite the "positive" outcome associated with the decisions made by the individual.

When teachers and administrators have incentives to do just enough to get by, something is broken. And while no one should believe all teachers are just out for survivial, we must create incentives for performances we value.

Race to the Top (RTTP) may very well give states and districts the opportunity to change this payout structure for both teachers and evaluators. Any effective evaluation system, must not only have negative consequences of failure, but positive consequences of success. And my next entry will try to explain how such a pay-off matrix will look and feel.

More on the evaluation system

State Senator Paul Pinsky wrote this op-ed piece in the Baltimore Sun. Of course, at the bottom of this op-ed is the little disclaimer that Paul Pinsky is actually an employee of MCEA. I can't even fathom that our great state has employees of interest groups working as lawmakers, but perhaps that's another matter. What I really don't understand is the the attitude of MCEA on this matter. When the legislation was worded that student achievement data had to be a "significant" factor in evaluations MCEA just celebrated that no change was ncessary. Now that Nancy Grasmick is suggesting that the MCPS evaluation system might need to be changed after all, the union is up at arms. The current evaluation system is NOT perfect. It's not! Why wouldn't the MCEA want to reevaluate their system if it possibly means it could improve? What they really want to change is nothing.

Make Administrators Accountable for Student Achievement

Recent debate about Race to the Top funds has centered around whether teachers should be held accountable for student achievement. But I have not seen much discussion of holding administrators accountable for the student achievement that occurs in their school. This might be even more important than holding teachers accountable. If principals were held accountable, they would actually have an incentive to get rid of ineffective teachers. Currently, principals fail to get rid of most ineffective teachers because its too much of a pain... guilt, documentation, time, effort, and for what? It's easier just to let them slide by. After all, principals are not punished when an ineffective teacher continues to teach in their building. But what if they were? What if their salaries were determined by the amount of learning that occured in their building? I'd bet you'd see alot more ineffective teachers dismissed than we do now.

And again, "Why aren't more teachers in PAR?"

This is from a 2004 report from the Montgomery County shared accountability office:

Referral Rate for Experienced Teachers

Since the program began in the 2000-2001 school year, 315 tenured MCPS teachers have been referred to PAR.3 District officials have raised the question about why this number is “so low.” Principals were asked precisely this. The answers, varied and instructive, fall into the six categories of response, described below:

Unwillingness to give up on teachers—The majority of principals are interested in helping teachers to improve their practice. They believe, “If I just try a little harder, work a little more with this teacher” things will get better. In MCPS, placing an experienced teacher in PAR sends a signal that the teacher is at risk of being terminated.4 One principal’s comments sum up the views of many: “It’s gut-wrenching. These [teachers] are not strangers. You know their stories. They’re not evil people.”

Concern that referral to PAR reflects negatively on the principal—A number of principals expressed concern that referring an experienced teacher to PAR reflects negatively on them as principals. The teacher being referred is often someone the principal has selected, or helped to select, for the position at the school. Some significant fraction of principals believe that if they refer teachers to PAR, it indicates, “we made a mistake selecting that teacher in the first place” and that “mistake” will translate into something negative on the principal’s evaluation.

Not wanting to roil school waters—Some principals say that knowledge that an experienced teacher has been referred to PAR creates tension at the school. “Other teachers become nervous, or think I’m picking on the teacher.” As a result, some principals admit, they do not refer teachers who ought to be referred to PAR so as to keep peace among the faculty.

The time factor—A number of principals admit that they refer fewer tenured teachers to PAR than might benefit from the assistance because of the time it takes for them to prepare a submission to the PAR Panel. Principals are required to complete two formal observations (down from three). Each of these, by some estimates, requires as long as three hours of the principal’s time. “Sometimes I just miss the deadline because I don’t get the work done in time,” commented one principal whose remarks spoke for a number of colleagues.

Holdover concerns about MCEA—A few principals report they are reluctant to refer experienced teachers to PAR because they are worried, based on history, that “MCEA will make things difficult”. None of the principals interviewed could point to an actual instance in which, since the inception of PAR, the union has attempted to stymie the process. Nevertheless, there is an anticipatory reaction among principals that seems to make some of them more reluctant than they otherwise might be to
recommend tenured teachers to PAR.

Newness of the system, newness of many principals—Finally, PAR is a relatively new system for MCPS. Given the staggered implementation of the PGS, some principals have had a year or two of experience with PAR, but fully one-third of principals began to implement the system just this school year. Compounding the newness of the system is the newness of many principals. Familiarity and comfort with the process likely will encourage greater use of it.

Again, the last pagagraph is simply not supported by the data. The time factor rings home to me. Although, I might call it the "it's a pain" factor.

I will tell you this, and let there be no doubt, if principals were paid extra for great assessment data, more teachers would be on PAR.

Why aren't more teachers on PAR?

More from the same article:

Julie Sanders, a 7th grade teacher in Montgomery County, is a strong believer in the idea behind PAR, which she calls a “get-well plan” for teachers, but she isn’t convinced that it adequately captures everyone who needs help.

“It would overwhelm the system,” Ms. Sanders said. “I think [the PAR panelists]
probably need to get rid of a lot more people than they actually do.”

And because of rigorously enforced timelines and the extensive documentation required to refer a teacher to peer assistance and review, some principals continue to use the “excessing” process to rid their buildings of poor-quality instructors, Ms. Sanders said. (Teachers who are removed from schools as a result of program changes, but still are employed by the district, are deemed “excessed.”)

That is a place where administrators need to be held accountable on making better use of the system, said Ms. Lawrence, the Toledo union president.

“It isn’t an easy thing to say to a teacher, ‘You have performance problems and you need to be referred to assistance,’ ” she said. “But that’s part of what being a manager means.”

Phillip Gainous, the vice president of the Montgomery County Association of Administrators and Supervisory Personnel and the co-chairman of the Montgomery County district’s PAR panel, thinks that the referral process is gradually improving. Principals are gradually coming to view PAR not as a hammer, he said, but as a genuine route to improvement.

This last paragraph is certainly not in line with the data that shows fewer and fewer tenured teachers being referred to the PAR program.

PAR data

Found all the data on the Peer Assistance and Review evaluation system on this page. A couple of thoughts: 1) Why are so few tenured teachers referred (I'll dig up some answers)? 2) Should we even spend these considerable resources on struggling teachers? For instance, what if Consulting Teachers (professionals in charge of evaluating the new and under performing teachers on PAR) were helping the mediocre teachers become master teachers instead of making ineffective teachers marginally less so? 3) We have a stick in place (though perhaps under utilized), but what is the carrot for the vast majority of Montgomery County teachers? Race to the Top seems to address this through some kind of incentivized pay system. But if 95% of teachers are not in need of the "stick" after year 1, the county and MCEA would do well to introduce some carrots.

The Montgomery County Public Schools evaluation system.

So I was looking at Montgomery County's "model" evaluation system here. Some very interesting stuff, with some good comparisons to other systems that have Peer Assistance and Review programs. I'm looking for more data, but here is some provided in the Education Week article. Please note that all novice (read: new) teachers are put into the PAR program, while tenured teachers must be referred by adminstrators.

MONTGOMERY COUNTY, MD.

Year: 2007-08
Total novices: 446
Nonrenewed: 16 (3.6%)
Granted 2nd year of PAR: 32 (7.2%)
Resigned:
N/A


MONTGOMERY COUNTY, MD.

(teachers on staff: 9,371)
Year: 2007-08
Teachers in intervention: 9
Dismissed: 7
Resigned/retired: 0
Extra year of intervention: 2
Successful return
to classroom: 0

Nine tenured teachers referred to the system for the 07-08 school year. None of them deemed worthy of returning to the normal professional growth system. Seven dismissed. This system cannot be valuable if so few tenured teachers are referred, and those that are referred are dismissed. How many teachers would have been dismissed if that number were 900 rather than 9? How many teachers could benefit from being in the PAR program?