How much is a good teacher worth?

I was forwarded this article from the Huffington Post on the value of a good teacher.   I've been doing alot of thinking about this lately.   What is the value?    And does it do us any good to think of teachers in these market based terms?

How much is a good teacher worth? Some would say they're priceless, but recent findings in the National Bureau of Economic Research's The Economic Value of Higher Teacher Quality, is a bit more exact. The report, written by Eric A. Hanushek, suggests that quality teachers with 20 students are worth $400,000 more in the future earnings of their students than an average teacher, annually.

Should teachers think of their value in this way?   Should the public?  It's nice to think I might impart a little value on my students' lives.   However, I've never really considered it to be a pecuniary one.

Dan Ariely, a behavioral economist has spent some time considering these questions in one of his books, Predictably Irrational.   In chapter four on social norms, he considers the problem when social norms and market norms cross paths.   It's  a fascinating read that has me thinking about what I do at work and why I do it.   A summary of the chapter is below:

 
I know that there has been a lot of work put in to change the way society thinks of teachers.  Mostly it's been a market based approach.   Teachers after all, have long been undervalued in the market.   So to attack this inequity it makes sense to demand more compensation.   I appreciate that this path has lead to more pay for teachers.   I wouldn't be one if that were not the case.  However, I think there may also be some negative consequences of this framing.   Teachers are not quite revered in American society.   Was this different 30 years ago?   100 years ago?   I don't know.    Would the public think differently about teachers if they weren't constantly asked to think about teachers' salaries?   What about teachers?  I do not think we want teachers thinking about their hourly or minutely contributions in terms of it's market value.   This leads teachers to come late and leave early.  But I think these are important questions to consider when thinking about evaluation systems that will create a better work place environment.

New Evaluation? NO. NO! NO!!!

MCEA (and then MSEA) says no to new evaluations required by the new state regulations.   They say yes only if they don't have to change.    I"ll concede the MCEA point on too much testing when MCEA and MCPS admit they can do a better job than they currently do identifying ineffective teachers.   If they can't do it better, the testing provides an objective meausure for evaluators to move in that direction.

A plea to slowdown

Valerie Strauss is laying on her criticism of Race to the Top even thicker on her Washington Post blog.



And while I admit to being a proponent of reform, I can't help but begin to think that Strauss is right about one thing; this all seems too rushed. She writes:

To make today’s deadline, many of the participating states rushed major education bills through their legislatures to meet the contest’s requirements and engaged in furious negotiations with unions against artificially set deadlines.

Unfortunately, this statement is too true. Despite MSDE claims to the contrary, its clear to me that stakeholders have been brushed aside in efforts to create quick change. I'm sure there's enough blame to go around, but my own school district and union have yet to say "boo" when it comes to where they stand on the issue of reform. I'm sure that statement is on the way though.

Struass goes even further:

[Legislatures] passed laws allowing more charter schools to open -- even though studies show that charter schools on average are no better than regular public schools -- and tying teacher compensation to standardized test scores, even though the tests aren't designed to assess teachers.

Academics call this a part of the construct validity of a test. And it has always been a kind of stumbling block when it comes to the use of high stakes tests. For instance, it might be said of a national algebra test that the test can effectively measure the knowledge skills and abilities of an algebra student. But that same test cannot and should not be used to measure whether a student should graduate from high school. Why not? Because that was not the design of the test when it was created. In other words, this test may be a valid measure of a student's math knowledge, but we cannot make generalizations about this test's ability to determine whether someone should graduate.

Likewise, that same Algebra test cannot and should not be used to measure whether a teacher is necessarily an effective teacher. This is not what this particular test was designed to capture.

This does not mean that student work, or student performance on tests cannot be used to help us evaluate teachers. However, it does mean we need to be congnizant of identifying teacher effectiveness solely or primarly based on student performance data that relies on these tests. And I'm not sure whether these conversations are going on as we speed on with the RttT process.

What we really need first is a comprehensive evaluation system for teachers. And I believe Race to the Top has the cornerstones of a reform system that most hard working teachers are dying to get behind. RttT demands that we need to more carefully identify which teachers (and principals) are strongest, and which teachers are weakest, and which teachers are somewhere in between. RttT also demands that teachers and principals be rewarded when they show they are capable. And RttT demands that we dismiss those teachers that are unable to show improvement, rather than those teachers who have been teachers the shortest length of time. These are real reforms for education; and reforms that will benefit our students around the country if we take the time to get it right.

My blogging friends over in California have put some real time and effort into this question of evaluation. And I applaud their efforts, despite the fact I believe evaluation systems can and should include input from students, parents, and colleagues (both master and non-master teachers).

But with that said, I'll leave you with InterAct's basic tenents for comprehensive teacher evaluation:

Here are the principles on which improved evaluations should be constructed:
-Teacher evaluation should be based on professional standards.
-Teacher evaluation should include performance assessments to guide a path of professional learning throughout a teacher’s career.
-The design of a new evaluation system should build on successful, innovative practices in current use.
-Evaluations should consider teacher practice and performance, as well as an array of student outcomes for teams of teachers as well as individual teachers.
-Evaluation should be frequent and conducted by expert evaluators, including teachers who have demonstrated expertise in working with their peers.
-Evaluation leading to permanent status (“tenure”) must be more intensive and must include more extensive evidence of quality teaching.
-Evaluation should be accompanied by useful feedback, connected to professional development opportunities, and reviewed by evaluation teams or an oversight body to ensure fairness, consistency, and reliability.

The Game of Teacher Evaluations

Game theory is a way to help us understand how rational individuals and groups make decisions. With broad applications from sociology to economics, I thought I'd give its application a try in the world of teacher evaluations, specifically, to help explain why so few teachers in MCPS are placed into Montgomery County's Peer Assistance and Review program, a program to support struggling teachers who show improvement and dismiss struggling teachers who do not.


The following application of game theory is based on several assumptions. And admittedly, assumptions, like the housing market will always go up, can be dangerous. I also take a complex issue, which involves many variables, and make it simple. But my hope is that even a crude analysis can provide some insights into the problem with evaluation systems in education as they exist today. One important side note: while game theory may help explain why most people act a certain way most of the time, it certainly cannot explain why all people choose to act in the way they do. Now to the good stuff.

In game theory, a payoff matrix is used to help understand the possible outcomes that individuals or groups face given two choices. The figure above is such a matrix. It works under the presumption that people will consider their payoffs when making decisions, which in turn helps us understand why many rational people, faced with the same set of circumstances, make the same decision. The figure above, is the payoff matrix for the choices faced by evaluators and ineffective teachers, to work hard (max effort), or to take it easy (min effort).

Let us first consider the evaluator, the person who is primarily responsible for determining whether a teacher receives a satisfactory or unsatisfactory evaluation. The evaluator has two choices. The first is to exert maximum effort, and thus proceed to document and place considerable time and effort in the process of placing the teacher into the PAR system. The second choice, is to exert minimum effort. In this scenario the evaluator does not place the struggling teacher in the PAR system, but enjoys the added benefit of more time to address other issues. In the payoff matrix, we assume that a rational actor is interested in making their life simpler, and thus minimizing effort. I assign no value to exerting maximum effort, but a (+1) value to exerting minimum effort. I assign additional value to removal of the struggling teacher. If the evaluator is able to remove the teacher through placement into the PAR system, I assign a (+1) value, but if the teacher is not removed, I assign a value of zero. This leaves the following payout structure for the evaluator:


0 = Considerable effort , but no teacher removal

1 = Minimum effort, and no teacher removal

1 = Considerable effort, and teacher removal

2 = Minimum effort, and teacher removal



The ineffective teacher has slightly different payouts, but the same choices. Like the evaluator, the ineffective teacher may exert either maximum or minimum effort, but would prefer to minimize effort. Like the evaluator, I assign a (+1) value to minimizing effort and no value to maximizing effort. On the other hand, the teacher has an incentive to keep his or her job. When he or she keeps their job and thus receives scheduled step increases, I assign a value of (+1). However, because of the negative consequences associated with losing one's job, I assign a (-1) value to this outcome. This leaves the teacher with the following possible payouts:

-1= Considerable effort, and loss of job

0 = Considerable effort, and same rate of pay

0 = Minimum effort, and loss of job

1 = Minimum effort, and same rate of pay

Analysis of the payoff matrix allows us to understand why so few teachers end up dismissed from their job, and why a new system is needed that changes the payout structure for both teachers and evaluators.

Evaluators would like to remove ineffective teachers, but they understand this requires considerable effort. Let's assume evaluators don't mind this extra effort, but only if this effort pays off in the removal of the ineffective teacher. Once the evaluator places the teacher in the PAR system however, the tenured teacher has two options. The first option is to continue to place minimum effort into his or her job. However, already in the PAR system, this option will result in his or her dismissal. There is now a substantial carrot for the ineffective teacher to put in more effort in order to avoid dismissal. The result of the evaluator exerting maximum effort, is thus a payout value of zero for the evaluator, who faces a now motivated teacher willling to do what is necessary to keep their job.

Faced with the likely consequence that a teacher will not be removed even if he or she is placed in the PAR system, the rational evaluator is more likely to put forth minimum effort. In other words, the evaluator will not take the time consuming steps of putting the ineffective teacher in PAR. In this scenario, the ineffective teacher is not placed in the PAR system, but the evaluator does not have the headache associated with documenting the placement. The evaluator's life has become substantially easier. The rational ineffective teacher now has the option to either exert maximum effort to become a better teacher or to exert minimum effort, but devoid of consequences for exerting minimum effort, will likely make this choice.

In game theory, we call the above solution, the solution in which each participant acts in their own self-interest (regardless of the decision made by their "opponent"), the Nash equilibrium. In our game, this means both the evaluator and the ineffective teacher will choose to put forth minimum effort. This of course, has a negative consequence for the state of education as a whole, despite the "positive" outcome associated with the decisions made by the individual.

When teachers and administrators have incentives to do just enough to get by, something is broken. And while no one should believe all teachers are just out for survivial, we must create incentives for performances we value.

Race to the Top (RTTP) may very well give states and districts the opportunity to change this payout structure for both teachers and evaluators. Any effective evaluation system, must not only have negative consequences of failure, but positive consequences of success. And my next entry will try to explain how such a pay-off matrix will look and feel.

Readers Respond: Baltimore Sun

Some of my thoughts on the hand-wringing going on over teacher evaluations in Montgomery County. Seems like the pendulum is in motion in other states, and it'd be best for MCPS and MCEA to get on board or be left behind.

Furloughs to come

Teacher furloughs, that is. Couple that with a wage freeze and it looks like MCPS teachers are in for a pay decrease. I do find it kind of interesting how the state budgets seem to lag the rest of the economy. After all, the economy has actually been growing for six months, and the states are just now getting slammed by smaller revenues.

Can't say I'm thrilled but I'm happy that the county council is taking a stand. The following seems to be the key:

“Years of spending without regard for sustainability coupled with a global economic meltdown got us into this situation. Since 2006, a new County Executive working with this Council has steadily decreased the growth in the spending. This year we will actually spend less than we did last year. Going forward, we must insure that spending is sustainable over the short and long term,” continued Elrich.

I'm not sure I completely understand why there should be a big "to do" every year over the budget. If there were a list of pre-recongnized cuts that were initiated were the county to experience a short-fall in revenues in any given year, we could all just go about our business rather than yelling and screaming and lobbying. Something tells me the quote above is lip-service, though. Sure hope I'm wrong.

Now we'll just have to wait and see how the Montgomery County School Board decides to proceed. The Parents' Coalition seems to think this is all one big waste.

And again, "Why aren't more teachers in PAR?"

This is from a 2004 report from the Montgomery County shared accountability office:

Referral Rate for Experienced Teachers

Since the program began in the 2000-2001 school year, 315 tenured MCPS teachers have been referred to PAR.3 District officials have raised the question about why this number is “so low.” Principals were asked precisely this. The answers, varied and instructive, fall into the six categories of response, described below:

Unwillingness to give up on teachers—The majority of principals are interested in helping teachers to improve their practice. They believe, “If I just try a little harder, work a little more with this teacher” things will get better. In MCPS, placing an experienced teacher in PAR sends a signal that the teacher is at risk of being terminated.4 One principal’s comments sum up the views of many: “It’s gut-wrenching. These [teachers] are not strangers. You know their stories. They’re not evil people.”

Concern that referral to PAR reflects negatively on the principal—A number of principals expressed concern that referring an experienced teacher to PAR reflects negatively on them as principals. The teacher being referred is often someone the principal has selected, or helped to select, for the position at the school. Some significant fraction of principals believe that if they refer teachers to PAR, it indicates, “we made a mistake selecting that teacher in the first place” and that “mistake” will translate into something negative on the principal’s evaluation.

Not wanting to roil school waters—Some principals say that knowledge that an experienced teacher has been referred to PAR creates tension at the school. “Other teachers become nervous, or think I’m picking on the teacher.” As a result, some principals admit, they do not refer teachers who ought to be referred to PAR so as to keep peace among the faculty.

The time factor—A number of principals admit that they refer fewer tenured teachers to PAR than might benefit from the assistance because of the time it takes for them to prepare a submission to the PAR Panel. Principals are required to complete two formal observations (down from three). Each of these, by some estimates, requires as long as three hours of the principal’s time. “Sometimes I just miss the deadline because I don’t get the work done in time,” commented one principal whose remarks spoke for a number of colleagues.

Holdover concerns about MCEA—A few principals report they are reluctant to refer experienced teachers to PAR because they are worried, based on history, that “MCEA will make things difficult”. None of the principals interviewed could point to an actual instance in which, since the inception of PAR, the union has attempted to stymie the process. Nevertheless, there is an anticipatory reaction among principals that seems to make some of them more reluctant than they otherwise might be to
recommend tenured teachers to PAR.

Newness of the system, newness of many principals—Finally, PAR is a relatively new system for MCPS. Given the staggered implementation of the PGS, some principals have had a year or two of experience with PAR, but fully one-third of principals began to implement the system just this school year. Compounding the newness of the system is the newness of many principals. Familiarity and comfort with the process likely will encourage greater use of it.

Again, the last pagagraph is simply not supported by the data. The time factor rings home to me. Although, I might call it the "it's a pain" factor.

I will tell you this, and let there be no doubt, if principals were paid extra for great assessment data, more teachers would be on PAR.

Why aren't more teachers on PAR?

More from the same article:

Julie Sanders, a 7th grade teacher in Montgomery County, is a strong believer in the idea behind PAR, which she calls a “get-well plan” for teachers, but she isn’t convinced that it adequately captures everyone who needs help.

“It would overwhelm the system,” Ms. Sanders said. “I think [the PAR panelists]
probably need to get rid of a lot more people than they actually do.”

And because of rigorously enforced timelines and the extensive documentation required to refer a teacher to peer assistance and review, some principals continue to use the “excessing” process to rid their buildings of poor-quality instructors, Ms. Sanders said. (Teachers who are removed from schools as a result of program changes, but still are employed by the district, are deemed “excessed.”)

That is a place where administrators need to be held accountable on making better use of the system, said Ms. Lawrence, the Toledo union president.

“It isn’t an easy thing to say to a teacher, ‘You have performance problems and you need to be referred to assistance,’ ” she said. “But that’s part of what being a manager means.”

Phillip Gainous, the vice president of the Montgomery County Association of Administrators and Supervisory Personnel and the co-chairman of the Montgomery County district’s PAR panel, thinks that the referral process is gradually improving. Principals are gradually coming to view PAR not as a hammer, he said, but as a genuine route to improvement.

This last paragraph is certainly not in line with the data that shows fewer and fewer tenured teachers being referred to the PAR program.

PAR data

Found all the data on the Peer Assistance and Review evaluation system on this page. A couple of thoughts: 1) Why are so few tenured teachers referred (I'll dig up some answers)? 2) Should we even spend these considerable resources on struggling teachers? For instance, what if Consulting Teachers (professionals in charge of evaluating the new and under performing teachers on PAR) were helping the mediocre teachers become master teachers instead of making ineffective teachers marginally less so? 3) We have a stick in place (though perhaps under utilized), but what is the carrot for the vast majority of Montgomery County teachers? Race to the Top seems to address this through some kind of incentivized pay system. But if 95% of teachers are not in need of the "stick" after year 1, the county and MCEA would do well to introduce some carrots.

MCEA

It's time to devote some time to improving the lone voice of Montgomery County teacher's in Maryland. This blog will call to account the policies, bad and good, of MCEA.