My note to MSDE

 Montgomery County Education Association (MCEA) recently celebrated a decision by a committee of the Maryland Genearl Assembly (chaired by an employee of MCEA) to block a  Maryland State Board of Education regulation calling for student achievement data to be 50% of teacher evaluations in Maryland.   The Maryland state law called for student achievement data to be a "significant" factor.    I agree with my union that 50% is too much.    I disagree that our current system is the 7th wonder of teacher evaluation systems.  Here's what I sent to state board members.


Dear Board member,



I am teacher in Montgomery County who has spent a considerable amount of time researching evaluation systems.. I take issue with my union on a number of issues, which I detail at improvingmcps.blogspot.com. However, I implore you to consider the following regarding evaluation systems:

1) Any new evaluation system should seek to more readily identify ineffective teachers. 2) Value Added Modeling, though limited, is particularly useful in identifying those teachers who are ineffective, but only when used over a 3 year period. 3) Conclusions reached by student achievement data, due to reliability concerns, should be corroborated by other evidence of teacher quality. 4) An evaluation that relies heavily on flawed student achievement data is just as arbitrary as a system pays teachers based on seniority. 5) New evaluation systems should promote rewards based on teacher quality, teacher responsibilities, and areas of need. In our current system, new teachers often carry the heaviest load(no classroom, most teacher preparations, toughest population of students) but are paid the least.

Please do not hesitate to contact me if you have any questions.



Sincerely,





Mike McCabe

Old news from Baltimore

I've been having some conversations with union leadership. Mostly, I've become very discouraged with the way MCEA and MCPS have reacted to changes coming around the bend. Will MCPS be left behind? I read things like the quote below from a Sun editorial way too often.

Although some Maryland school districts -- including Montgomery County, the state's largest -- have resisted modifying the way teachers are evaluated, Baltimore City embraced the concept because it could bring millions of new dollars to its cash-starved school system.

Which leaves me to wonder whether my original decision to come to Montgomery County will eventually become an impetus to leave. Montgomery County Public Schools has long been the highest paying school district in the state. Can we manage to maintain that edge in the current political and economic climate?

I don't believe we can when I continue to read posts like this one from Tom Israel, exective director of MCEA. The language of MCEA and MCPS must change, and change now. Money. Real money, is on the line. Hopefully, it's not already too late.

Doug Prouty

Commander in Chief of MCEA, Doug Prouty, made a speech (hail youtube) about the problems associated with Maryland's Race to the Top application.

Same mind-boggling crap
1) "I guarantee you, that every single teacher, every single lesson, he or she is thinking about what went well, what could have been improved, and they're working on that every single day in the classroom (Are you kidding? This has got to be a bad joke. Unless of course, you talk to every single teacher after every single lesson.)
2) The PAR system we created in 2000 is the bomb (because no one gets dismissed)
3) Be afraid of standardized testing.

A plea to slowdown

Valerie Strauss is laying on her criticism of Race to the Top even thicker on her Washington Post blog.



And while I admit to being a proponent of reform, I can't help but begin to think that Strauss is right about one thing; this all seems too rushed. She writes:

To make today’s deadline, many of the participating states rushed major education bills through their legislatures to meet the contest’s requirements and engaged in furious negotiations with unions against artificially set deadlines.

Unfortunately, this statement is too true. Despite MSDE claims to the contrary, its clear to me that stakeholders have been brushed aside in efforts to create quick change. I'm sure there's enough blame to go around, but my own school district and union have yet to say "boo" when it comes to where they stand on the issue of reform. I'm sure that statement is on the way though.

Struass goes even further:

[Legislatures] passed laws allowing more charter schools to open -- even though studies show that charter schools on average are no better than regular public schools -- and tying teacher compensation to standardized test scores, even though the tests aren't designed to assess teachers.

Academics call this a part of the construct validity of a test. And it has always been a kind of stumbling block when it comes to the use of high stakes tests. For instance, it might be said of a national algebra test that the test can effectively measure the knowledge skills and abilities of an algebra student. But that same test cannot and should not be used to measure whether a student should graduate from high school. Why not? Because that was not the design of the test when it was created. In other words, this test may be a valid measure of a student's math knowledge, but we cannot make generalizations about this test's ability to determine whether someone should graduate.

Likewise, that same Algebra test cannot and should not be used to measure whether a teacher is necessarily an effective teacher. This is not what this particular test was designed to capture.

This does not mean that student work, or student performance on tests cannot be used to help us evaluate teachers. However, it does mean we need to be congnizant of identifying teacher effectiveness solely or primarly based on student performance data that relies on these tests. And I'm not sure whether these conversations are going on as we speed on with the RttT process.

What we really need first is a comprehensive evaluation system for teachers. And I believe Race to the Top has the cornerstones of a reform system that most hard working teachers are dying to get behind. RttT demands that we need to more carefully identify which teachers (and principals) are strongest, and which teachers are weakest, and which teachers are somewhere in between. RttT also demands that teachers and principals be rewarded when they show they are capable. And RttT demands that we dismiss those teachers that are unable to show improvement, rather than those teachers who have been teachers the shortest length of time. These are real reforms for education; and reforms that will benefit our students around the country if we take the time to get it right.

My blogging friends over in California have put some real time and effort into this question of evaluation. And I applaud their efforts, despite the fact I believe evaluation systems can and should include input from students, parents, and colleagues (both master and non-master teachers).

But with that said, I'll leave you with InterAct's basic tenents for comprehensive teacher evaluation:

Here are the principles on which improved evaluations should be constructed:
-Teacher evaluation should be based on professional standards.
-Teacher evaluation should include performance assessments to guide a path of professional learning throughout a teacher’s career.
-The design of a new evaluation system should build on successful, innovative practices in current use.
-Evaluations should consider teacher practice and performance, as well as an array of student outcomes for teams of teachers as well as individual teachers.
-Evaluation should be frequent and conducted by expert evaluators, including teachers who have demonstrated expertise in working with their peers.
-Evaluation leading to permanent status (“tenure”) must be more intensive and must include more extensive evidence of quality teaching.
-Evaluation should be accompanied by useful feedback, connected to professional development opportunities, and reviewed by evaluation teams or an oversight body to ensure fairness, consistency, and reliability.

Making data count

Enjoyed this little piece from New York Times Magazine about the shortfalls of data. It's a few weeks old, but thought of it now because Valerie Strauss blogged about Chicago today. Chicago. Former home of Arne Duncan. And supposedly the sucess story behind Race to the Top. Or maybe there isn't a success story behind it.

In the inbox

The Baltimore Sun said this in their thursday editorial:

Maryland hasn't even taken up some of the more sweeping reforms that other states already have adopted, such as creating greater opportunities for charter schools or alternative certification for teachers. But the Montgomery County school district and several local unions are balking at even the relatively modest changes — particularly, tying teacher evaluations to student test scores — that the General Assembly approved this year.

To which I had a reply in my inbox:

A modest change??? That would change everything, IMHO.

Let there be no doubt, as far as institutional change goes, what the state of Maryland is asking of localities is substantial. But I remember reading somewhere, or maybe I was talking about it with some collleagues, that perhaps what MCPS really has up its sleave is its own RttT application, which would likely come about in a sort of 3rd round of applications.

Maybe the key statement is later down in the Sun editorial:

Montgomery County may turn up its nose at the new standards, believing that its system is better than anything the state could possibly come up with, but what it's objecting to really amounts to ceding control over 30 percent of the evaluation.

I've never been much of a power guy; but I suppose this could be the issue. It'd be a lot easier if we made decisions based on the best ideas. But I suppose that's the same reason why I blog about evaluation instead of making decisions about it.

Teachers' Unions' Last Stand

Great article in the New York Times magazine entitled The Teachers' Unions' Last Stand. A must but lengthy read on Teachers' Unions and Race to the Top. Read it critically, and then read this blog by Valerie Strauss of the Washington Post. I'm definitely skeptical of any claim that charter schools, by the mere act of being charter schools, are any better or worse than public schools.

The Game of Teacher Evaluations

Game theory is a way to help us understand how rational individuals and groups make decisions. With broad applications from sociology to economics, I thought I'd give its application a try in the world of teacher evaluations, specifically, to help explain why so few teachers in MCPS are placed into Montgomery County's Peer Assistance and Review program, a program to support struggling teachers who show improvement and dismiss struggling teachers who do not.


The following application of game theory is based on several assumptions. And admittedly, assumptions, like the housing market will always go up, can be dangerous. I also take a complex issue, which involves many variables, and make it simple. But my hope is that even a crude analysis can provide some insights into the problem with evaluation systems in education as they exist today. One important side note: while game theory may help explain why most people act a certain way most of the time, it certainly cannot explain why all people choose to act in the way they do. Now to the good stuff.

In game theory, a payoff matrix is used to help understand the possible outcomes that individuals or groups face given two choices. The figure above is such a matrix. It works under the presumption that people will consider their payoffs when making decisions, which in turn helps us understand why many rational people, faced with the same set of circumstances, make the same decision. The figure above, is the payoff matrix for the choices faced by evaluators and ineffective teachers, to work hard (max effort), or to take it easy (min effort).

Let us first consider the evaluator, the person who is primarily responsible for determining whether a teacher receives a satisfactory or unsatisfactory evaluation. The evaluator has two choices. The first is to exert maximum effort, and thus proceed to document and place considerable time and effort in the process of placing the teacher into the PAR system. The second choice, is to exert minimum effort. In this scenario the evaluator does not place the struggling teacher in the PAR system, but enjoys the added benefit of more time to address other issues. In the payoff matrix, we assume that a rational actor is interested in making their life simpler, and thus minimizing effort. I assign no value to exerting maximum effort, but a (+1) value to exerting minimum effort. I assign additional value to removal of the struggling teacher. If the evaluator is able to remove the teacher through placement into the PAR system, I assign a (+1) value, but if the teacher is not removed, I assign a value of zero. This leaves the following payout structure for the evaluator:


0 = Considerable effort , but no teacher removal

1 = Minimum effort, and no teacher removal

1 = Considerable effort, and teacher removal

2 = Minimum effort, and teacher removal



The ineffective teacher has slightly different payouts, but the same choices. Like the evaluator, the ineffective teacher may exert either maximum or minimum effort, but would prefer to minimize effort. Like the evaluator, I assign a (+1) value to minimizing effort and no value to maximizing effort. On the other hand, the teacher has an incentive to keep his or her job. When he or she keeps their job and thus receives scheduled step increases, I assign a value of (+1). However, because of the negative consequences associated with losing one's job, I assign a (-1) value to this outcome. This leaves the teacher with the following possible payouts:

-1= Considerable effort, and loss of job

0 = Considerable effort, and same rate of pay

0 = Minimum effort, and loss of job

1 = Minimum effort, and same rate of pay

Analysis of the payoff matrix allows us to understand why so few teachers end up dismissed from their job, and why a new system is needed that changes the payout structure for both teachers and evaluators.

Evaluators would like to remove ineffective teachers, but they understand this requires considerable effort. Let's assume evaluators don't mind this extra effort, but only if this effort pays off in the removal of the ineffective teacher. Once the evaluator places the teacher in the PAR system however, the tenured teacher has two options. The first option is to continue to place minimum effort into his or her job. However, already in the PAR system, this option will result in his or her dismissal. There is now a substantial carrot for the ineffective teacher to put in more effort in order to avoid dismissal. The result of the evaluator exerting maximum effort, is thus a payout value of zero for the evaluator, who faces a now motivated teacher willling to do what is necessary to keep their job.

Faced with the likely consequence that a teacher will not be removed even if he or she is placed in the PAR system, the rational evaluator is more likely to put forth minimum effort. In other words, the evaluator will not take the time consuming steps of putting the ineffective teacher in PAR. In this scenario, the ineffective teacher is not placed in the PAR system, but the evaluator does not have the headache associated with documenting the placement. The evaluator's life has become substantially easier. The rational ineffective teacher now has the option to either exert maximum effort to become a better teacher or to exert minimum effort, but devoid of consequences for exerting minimum effort, will likely make this choice.

In game theory, we call the above solution, the solution in which each participant acts in their own self-interest (regardless of the decision made by their "opponent"), the Nash equilibrium. In our game, this means both the evaluator and the ineffective teacher will choose to put forth minimum effort. This of course, has a negative consequence for the state of education as a whole, despite the "positive" outcome associated with the decisions made by the individual.

When teachers and administrators have incentives to do just enough to get by, something is broken. And while no one should believe all teachers are just out for survivial, we must create incentives for performances we value.

Race to the Top (RTTP) may very well give states and districts the opportunity to change this payout structure for both teachers and evaluators. Any effective evaluation system, must not only have negative consequences of failure, but positive consequences of success. And my next entry will try to explain how such a pay-off matrix will look and feel.

MCPS: a summary

So it's all quite fascinating, really. Teacher pay in Montgomery County is frozen. The County is fighting itself over furloughs. Montgomery County teachers are now voting on a contract that freezes pay for at least one additional year after that. And now the Obama administration could be sending money to help. Meanwhile, the state of Maryland wants to become another Race to the Top state, which the state's largest county, Montgomery, is happy to support so long as they don't actually have to change.

My head hurts.

Did MCPS get the memo? About the change? Notice what New Haven, CT titles this page -school reform documents. That's more catchy than school stay the same documents.

Make Administrators Accountable for Student Achievement

Recent debate about Race to the Top funds has centered around whether teachers should be held accountable for student achievement. But I have not seen much discussion of holding administrators accountable for the student achievement that occurs in their school. This might be even more important than holding teachers accountable. If principals were held accountable, they would actually have an incentive to get rid of ineffective teachers. Currently, principals fail to get rid of most ineffective teachers because its too much of a pain... guilt, documentation, time, effort, and for what? It's easier just to let them slide by. After all, principals are not punished when an ineffective teacher continues to teach in their building. But what if they were? What if their salaries were determined by the amount of learning that occured in their building? I'd bet you'd see alot more ineffective teachers dismissed than we do now.

Teacher evaluations to be 50% student data?

Looks like Nancy Grasmick is acting unilaterally to make student growth 50% of teacher evaluations rather than simply a "significant" portion, as required by the new state law. Interesting, because MCPS and the MCEA seemed to "slap five" after the Maryland state law was passed, arguing that they've already got it covered. It seems this is the states's shot off the bow of that celebration, and will force MCPS to go back to the drawing board.

If this is the state's way of calling out MCPS, then I'm all for it. It's time MCPS and MCEA engage in real meaningful reform that incentivizes the teaching profession. Come up with a system that rewards teachers for doing their job well. Reward teachers who work hard, not those who just get by. There is room to create this reform, even within or with additions to the current PGS and PAR system, but only if they engage the process. MCPS should lead and initiate like places such as New Haven, Connecticut are already doing. The alternative is to continue to make arguments for the status quo. Figure out how teachers can track their performance using a value-added approach to student performance. Figure out how to do the same for administrators who could potentially have more power under new system. It's high time we improved the profession once and for all.

From the MCPS page

Their title: MCPS, MCEA Agree on Use of Student Performance Data in Evaluations


My title: Weast and Prouty agree state law doesn't require any change! Rejoice.

Weast on Race to the Top

So I read this, and from what I understand, Jerry Weast's main point is that MCPS already does evaluation of teachers so well that it would have to be wary of changing anything that is already so good. He was almost dubious that Maryland could have a better idea than his. Ok then.