The failure of targeting data

I wrote last week about the dangers of targeting data rather than the underlying conditions that serve as the cause for the data.   Jay Mathews of the Washington Post then wrote about this policy at West Springfield High in Virginia.  The school provides some type of review session for students in need remediation.  That sounds benign enough- except that while a small percentage of students attend these sessions, the rest of the school is left to litterally, do whatever they please:

Fairfax high schools have different names and schedules for the periods. At West Springfield, two 45-minute sessions a week are used to help the 10 percent or so of students in danger of flunking Virginia's Standards of Learning exams.

The decision made above is not one that was made in a vaccuum.   Rather, this is symptamatic of the way data drives narrowly tailored decision making across the country.   Often, this type of decision making drives data driven results that are celebrated as successes.   Yet little is done to consider  the cost of such decision- making.   Are the students at West Springfield High School better off for a policy where 90% of the student body sits idle?   I am inclined to think the answer is no.

The value of Value-Added

Here's a study from the Brookings Institution you won't find on the MCEA blog.  Jay Mathews brought it to my attention and he does a good job summarizing the study which focuses on the reliability concerns associated with value-added modeling.   The main idea is that just because a teacher's "percentile rank" might fluctuate year to year- does not mean it is a worthless piece of information.   They compare a teacher's percentile value-added rank to a baseball player's batting average which has similar year to year fluctuations.   Would you make decisions about the current year based on the batting average from last year?   Well, you'd certainly keep it in mind.  From the executive summary:

We believe that whenever human resource actions are based on evaluations of teachers they will benefit from incorporating all the best available information, which includes value-added measures. Not only do teachers typically receive scant feedback on their past performance in raising test scores, the information they usually receive on the average test scores or proficiency of their students can be misleading or demoralizing. High test scores or a high proficiency rate may be more informative of who their students are than how they were taught. Low test scores might mask the incredible progress the teachers made. Teachers and their mentors and principals stand to gain vast new insight if they could see the teachers’ performance placed in context of other teachers with students just like their own, drawn from a much larger population than a single school. This is the promise of value-added analysis. It is not a perfect system of measurement, but it can complement observational measures, parent feedback, and personal reflections on teaching far better than any available alternative. It can be used to help guide resources to where they are needed most, to identify teachers’ strengths and weaknesses, and to put a spotlight on the critical role of teachers in learning.

Randi Weingarten - The Hero of Reform

Valerie Struass recently printed this defense of Randi Weingarten, president of the American Federation of Teachers, and newly cast villain of   'Waiting for Superman.'     But as Joe Williams points out in Strauss' column, this could not be further from the truth.  From Denver to Pittsburgh to New York City to Baltimore, the American Federation of Teachers has been at the forefront of public school system reform.   This is hardly the face of an obstructionist.   As Williams notes:

Randi Weingarten is the last person you could possibly describe as hiding in a cave, plotting to destroy America. She has appeared on so many panels and television programs as part of the WFS roll-out – and she’s taken quite a public beating in many of them – that ‘cowardly terrorist’ is the last phrase you’d use to describe her. (You’ll notice that the NEA, which unlike the AFT has been totally absent from just about any real reform discussion in the last few years, hardly even appears in WFS. Surely because Randi granted access and the NEA didn’t.)

It is a shame that Weingarten has not gotten more credit for the role she has played in advancing these reforms.   Jay Mathews takes time to celebrate her role for the recently negotiated  (but as of yet unapproved) contract in Baltimore City.   And celebrate we should, for if the current reforms prove successful in anyway, Weingarten will not be the villain, but the hero of this story.

Newsweek: "America's Best High Schools"

Newsweek published their list of "America's best high schools." Ha ha ha ha ha. Don't get me wrong, its kind of a neat metric, but a list of America's best schools? That's like saying I'm the best AP Government teacher because all of my students took the AP Government exam.

How about when we come up with a more helpful way to evaluate teachers, we also come up with a better way to evaluate schools! This can be a metric. Not the only one.

Great conversation

Worth the price of admission (free) to read the comments on Jay Mathews' blog today about how we should evaluate teachers. Any comprehensive evaluation system should have multiple data points that corroborate claims of teacher performance. Practicality aside, I appreciated the following comments:

A list, off of the top of my head, that I would think should be considered are...

1. Several short evaluations by outside evaluators.
2. Several informal evalutations by administrators in the school. (Note: These need to be more realistic then IMPACT, which while a good start is unrealistic in what is expected in 30 minutes AND there is way too much room for intepretation)
3. Student input. Seriously, students know a frightening amount about who are and are not the good teachers. You would have to find a way to separate teachersstudents like vs. those they learn from.
4. Objective measures of professionalism... Does the teacher respond to parents? Does the teacher get to school on time? Does the student engage in professional development and collaboration with their co-workers?
5. What does a teacher do for a school besides his/her direct teaching. Do they run
a club, support tutoring after school, etc...
6. Test Scores- I guess I should list these, because it's going to come up. However, without some method of measuring growth by student, rather then generically, it is just a waste of time at best, and discourages the best teachers from teaching the weakest students at worst.

In other news, MCEA teachers approved the new contract. I voted against it- on principle only.

And finally, the Balitmore Sun blasted MCPS. The more and more I read the more and more I wonder about what is going on behind the scenes. I just don't get the sense that MCPS or MCEA are having any conversations about reform. So I'm just left scratching my head.

It's all in the data.

I've been emailing Jay Mathews of the Washington Post about this quote from his May 21st blog:

I asked some Washington area school districts, nationally regarded for their high performance and sophisticated management, what percentage of their teachers were rated satisfactory in the last evaluation. I subtracted the number they gave me from 100 percent to calculate what portion of their teaching ranks could be considered ineffective. The results:Fairfax County 0.9 percent, Montgomery County 5 percent, Loudoun County 1 percent, City of Falls Church 0.45 percent, Prince William County 1.7 percent. I could not find a school district in the area that admitted to having more than 5 percent unsatisfactory teachers. Most said the figure was closer to 1 percent. Experts tell me this is common throughout the land. I don't believe the numbers.

It turns out that Jay may have made a small but understandable, logical misstep. He assumed that if 95% of Montgomery County teachers are currently rated satisfactory, then 5% must currently be rated unsatisfactory. Turns out, that's not quite right. I went to an MCPS annual report and found something different. It seems that 404 of those teachers (amongst the 5%) were novice teachers. They had not yet been evaluated. The other 67 teachers (of that same 5%) were underperforming tenured teachers. Math is not my speciality, but if it is correct then if 471 is 5%, 67 must be about .7%. This means that in 2009, over 99% of Montgomery County teachers were evaluated as satisfactory. That rates right up there with just about every other county in Maryland.

Meanwhile you've got the Baltimore Sun reporting that MCPS will not sign on to Maryland's Race to the Top application because of this same evaluation system:

Montgomery's Weast told the board Tuesday that the county will not sign the application because it does not want to give up its own teacher evaluation system. In an interview, he said that the county believes its evaluation system is better than others and that the state has not fully thought through the process for many of the reforms.

Or from Weast's own letter to the Baltimore Sun in which he claims,

MCPS has a comprehensive school reform plan that is working and is mirrored in the right-minded principles promoted by President Barack Obama and Education Secretary Arne Duncan as part of the Race to the Top competition. This isn't educational theory; this is real reform, with real results.

It's almost laughable that in a 2004 report (which incidentally, was taken off the interweb by MCPS, but graciously stockpiled here by the Parents Coalition) MCPS was asking itself why so few teachers were referred to PAR, but by 2009 was concluding:

Increases in the number of teachers provided with PAR panel support indicate greater action on the part of principals and supervisors to implement the Teachers Professional Growth System.

A conclusion apparently based on the following data on the number of under performing, tenured teachers referred to PAR:














I'm not sure I'd have come to the same conclusions. But I do have my own reasons for why so few teachers receive an unsatisfactory rating. One thing should be clear, MCPS, like the rest of the state, needs new evaluation methods and new ways to remove ineffective teachers. I still do not understand the reluctance by MCPS officials to embrace a change that could keep MCPS's innovative teacher evaluation system in place, but with added reforms that make it substantially more robust. That seems to be a change everyone should be able to get behind.