Starr vs. Weast: The line in the sand

Joshua Starr, the new superintendent of Montgomery County Public Schools, seems to get it.     From the Gazette:

I’m struck by how outcomes-based the organization has been — and I am a believer in outcomes. You have to be accountable and responsible for outcomes. I learned after a few months that there was an orientation toward: Get the result. The shift that I am trying to make is to have the conversation about how we are getting the result. I still expect the result. In fact, I want a better result. ... So it goes back to how do we do it. ... If we orient ourselves to what do we do every day... to make sure that our kids are fully engaged in high quality instruction, what does that mean with our daily work, with teachers, with principals, with kids? And that is the change. I am not big into initiatives. I am not big into naming things and rolling out the banner and saying here is the big thing we are going to do now. It just doesn’t work. What works is having great instructional leaders leading in every building, having good curriculum and professional development and helping teachers and other folks in the system get better at what they do every day with the kids. That is what works...

This is the ideology behind real progress.   Almost makes me wish I still taught in Montgomery County.

A sophisticated decision: MCPS creates record SAT scores

The message in 2009 was much more sophisticated than in 2003.  This time, principals would not be asked how they could decrease participation on the SAT in order to raise the averages.  This time, MCPS would just ask a simple question:  "which test is best for you?"      It was so benign, yet so brilliant.   Have a meeting with the administration of every high school, and ask them to communicate to parents and students that they had a choice:  take the ACT or the SAT.   Just use criteria like the one below to help students make their decision:SAT ACT Comparison Checklist

Nothing to it.  This SAT/ACT action plan would then be "customized to meet the needs of individual schools."

Clarksburg High School created a page.

Walter Johnson had a page.

Northwood did this.

Damascus High School made a Power Point.

The plan was perfect.  In 2010, twenty-four of twenty five high schools in Montgomery County saw their percentage of SAT test takers drop.  Participation dropped, average scores rose, and Dr. Weast could continue to promote the MCPS brand name while building his consulting career in- get this- helping other districts use data to increase student achievement.

Want to know what is wrong with our educational system?   Folks- MCPS is a national model for its use of data.   But no one cares about what the data is designed to measure.   We just care about moving the data- which is no way to run an educational system.

An SAT meeting with results that matter

Dr. Jerry Weast likely learned a very important lesson in 2003.   Messaging is important.    By 2009 the message was far more refined, but the results would be no less contrived.  Every high school in Montgomery County would be invited to attend "Leadership Training" on the SAT, ACT, and Accuplacer tests- the three college placement exams.   What would go on at this meeting?  It would  certainly not be a repeat of 2003.

SAT ACT Accuplacer Confirmation


However, the results would be instantaneous, and just in time for Jerry Weast's retirement.   The year after the "Leadership Training" in 2009, 483 fewer students in Montgomery County Public Schools would take the SAT.  Of those 483 students, a disproportionate number would be minority students.   Hispanic participation would drop by 18% in 2010, African American participation would drop by 12%, while white participation would drop by a relatively modest 4%.

 The MCPS Office of Shared Accountability described the decline in this way:

For the MCPS class of 2010, slight declines in SAT participation were balanced by record increases in the number of students who took the ACT in lieu of the SAT.

This was the full extent of the MCPS analysis.    There would be no discussion of the who, why, or how.  There would be no detailed analysis of the massive drop in participation.  The analysis simply claimed that the 483 student decline in SAT participation was balanced by a 267 student increase in those students who only took the ACT.    An odd balancing act to say the least.

But what remained were legitimate questions.   Were the record SAT scores in 2010 worthy of celebration?  Was it a sign that all that had been done in Montgomery County Public Schools over the tenure of Jerry Weast had finally paid dividends?  Or was it as current Board of Education member and then principal, Michael Durso, explained in 2003, a way to make it on 60 Minutes.   Sometimes, the simple answer is the more likely one.

The Montgomery County Public Schools' Fudge Factor: Record SAT scores

Fudging data is now the culture of schools across the United States.   It is not a conspiracy theory.  It is the way that districts and the principals within those districts that "honor" the data, attempt to make themselves look great.   And as a general rule of thumb, the better the"fudger," the greater the reputation that accompanies it.  Let's just take former superintendent of Montgomery County Public Schools, Jerry Weast, for example, the man Jay Matthews once called one of the top ten superintendents in the country.  In 2010, the year he retired, he claimed as one of his top ten achievements as a superintendent, record SAT scores.   Of course, these SAT scores were accompanied by record drops in SAT participation.   Think this was just  an accident?   Or could this be a similar but more sophisticated version of what occurred in 2003, when a Montgomery County Public School's newspaper, Blair High School's SilverChips, reported that principals were actually told by Weast to contrive their SAT scores through participation rates:

At an Oct 2 meeting with high school principals, Weast suggested, according to Principal Phillip Gainous,that students who could not help out SAT scores [for MCPS] and were not ready to take the SATs should be discouraged from taking them," said Gainous.  

Both Principal Daniel Shea of Quince Orchard High School and Principal Michael Durso of Springbrook High School corroborated this statement. According to Shea, Weast wanted schools to *examine the level of students taking the SATs to appropriately limit exposure." The superintendent*s message was clear, said Gainous: *We were told to do it. And the expectation was that we would all go back and do it. 

Elimination of low-scoring students from the general test-taking pool will automatically boost average SAT scores, explained Durso. If we have a certain number of students not take the SATs, then we*ll be on Sixty Minutes because of our [high] scores," he claimed.  

"It's morally wrong. It's illegal," said Blair PTSA President Valerie Ervin, who works for the County Council, regarding Weast*s suggestion.

In 2010, the media picked up the the story and touted  Montgomery County's record, just the way Jerry Weast knew they would. But perhaps the record was not quite what it appeared to be on first glance.   Perhaps we were closer to what Valerie Ervin concluded- something morally wrong.

How records are made

The MCPS press release was straight to the point.  The results were nothing short of miraculous:

Students in the Montgomery County Public Schools Class of 2010 produced the district’s highest-ever composite score on the SAT and dramatically outperformed their state and national peers, according to data released today by the College Board...

...African-American and Hispanic test-takers posted the strongest gains in MCPS, improving on last year’s average composite scores by 49 and 54 points, respectively. Those increases outpaced the growth in scores for Asian students (up 21 points) and White students (up 15 points), and further narrowed the district’s racial and ethnic gap in SAT performance.

It was an exclamation point on the tenure of Superintendent Jerry Weast.   What was at one time falling SAT scores, had turned on a dime, and turned hard.   The proof was in the preverbial pudding.   The accountability department published the remarkable data.   

From: http://www.montgomeryschoolsmd.org/info/pdf/Class2010SAT-MCPS-MD-Nation-9-13-10-final.pdf

The achievement gap was closing on perhaps the most educationally important data point there was, the SAT.   But what was the cause?

Was it excellent teaching?   Fierce Leadership?   I was skeptical.  My search started with the discovery of the  lowest participation rate in ten years, but that would just be the beginning.   It was even more important to figure out who did not participate.   The answer I found was unfortunately, not surprising.   In statistics circles it is well known that as participation increases, averages tend to fall, and vice versa.  

Fantastic rises in achievement were accompanied by equally fantastic decreases in SAT participation.   African American and Hispanic sub-groups led the decline in particpation.   The accountability office said it this way:

Examination of trends in SAT participation and performance provide evidence that MCPS is making progress toward the strategic plan of ensuring success for every student.

I suppose I saw it differently. What I saw as a problem, MCPS saw as evidence of success. Jerry Weast was no different

Superintendent Jerry D. Weast credits the record-breaking SAT results to talented teachers, committed staff, and motivated students as well as to the reform plan implemented 11 years ago.

“We believe in high standards, high expectations and high performance,” Weast said. “We’ve followed a clear path that works and produces exceptional results for our students at every level.  These SAT scores are extraordinary and are something that our entire community should take pride in."

 The participation data was even more disconcerting when disaggregated by service group.

Records were indeed set in 2010.   I'm just not sure they were the kind of records that should be making headlines.   And if this is how "records" were made in Montgomery County- we needed to do some real and honest soul-searching.

A Stomach Churning Letter

My wife thought it was of little consequence.  

However, as I read congratulatory emails sent out to staff and watched videos celebrating Montgomery County Public Schools' record high SAT scores, my stomach churned.   "What do you expect them to do?" my wife chimed.  I suppose I took it personally.  Yes, the mean SAT in MCPS had risen to its highest level ever- but the conclusions circulated by MCPS were nothing short of complete misrepresentations.    I couldn't help but feel insulted as I read the email from Superintendent Weast to the staff:

The College Board released SAT scores for the Class of 2010 this morning and MCPS students set an all-time record. Our 2010 graduates scored an average of 1653, which is our district’s highest score since the “new SAT” was implemented in 2006 and represents a one-year increase of 38 points. MCPS graduates outscored their Maryland peers by 151 points and the nation’s 2010 graduates by 144 points. Students in all racial subgroups improved over last year, but African American and Hispanic students made the biggest gains, further narrowing the achievement gap. The best news of all is that 51% of our students scored a 1,650 or higher, meeting the 7th Key to College Readiness—again, an all-time record.


Of course, these results did not happen in one year, or even in four years. The students whose achievements are described in this report were second graders when we began working together in 1999 and made a firm commitment that we would give all students access to an outstanding education. From elementary school, through the middle grades, and into high school, you provided our students with the opportunities and support they needed to be successful. The SAT results released today are the culmination of all the work done by you and your colleagues since these students entered MCPS.

Yes, insulted.   MCPS had long told me about the importance of data.  It had long emphasized how to use that data to guide my instruction.   Yet now I was being told that the most recent "record" SAT scores were evidence of MCPS's unparalleled success.

I finally replied to my wife, "I expect them to be reflective practioners.  I expect them to have an honest dialogue about the progress of MCPS."     

I showed my wife the data that initiallly raised an eyebrow.

Math and Verbal combined SAT scores for MCPS (writing excluded)

My wife was unconvinced.

I would need to spend more time proving my point.   The achievement gap hadn't closed.  The mean SAT was no more a record than had I served as Superintendent and asked only the top 1% of students to take the SAT.   Yet Weast would tell Bethesda Magazine that this data point was one of his top ten accomplishments as Superintendent of MCPS.

It couldn't be.  It wasn't even an accomplishment.

I dangerously decided to see if I could make my wife see it my way.

What's in a name

On May 5th 2011 I received my 3rd email invitation in three months. I was cordially invited to attend a celebration of 12 years of service for outgoing Montgomery County Public Schools Superintendent, Dr. Jerry Weast. It was now a model of communication for which I had become all too accustomed- whether through pat-on-the-back celebratory emails or via holiday robo-calls reminding me of my dedication to the school district. The invitation was titled, “A Legacy of Excellence” and directed readers to the www.weastlegacy.com where ticket purchases could be made for $38. The invitation explained that one could also make a donation to the newly established Jerry Weast Book Project. This project would “enhance the book collections in all 131 MCPS elementary school media centers.” The books purchased would also include a bookplate indicating it was purchased by the Jerry Weast Book Project. It was a fitting end, and we could now say our “collective thank you” to the tenure of Jerry Weast. The invitation was signed, “Patricia O’Neill, Member Board of Education.”




In a vacuum, the invitation was unassuming. It was a pleasant send off for the leader of one of the largest school districts in the United States, and Dr. Weast would do it in class- establishing a small foundation as a way to say thank you to the students he served. It was an appropriate way to say good-bye.


However as a teacher who knew nothing but the Weastian way of doing business, I could not help but view the invitation an entirely different light. After ten years of teaching in Montgomery County, my mind had turned what appeared to be a simple charitable act into a self-serving marketing event. With a certain amount of guilt, I imagined thousands of book plates with the Jerry Weast name affixed- now circulating  ad finem through MCPS libraries. It had become the way things were done in Montgomery County. Of course, I was not surprised by the invitation- but my vision of the “Weast Legacy” was probably much different than what Board of Education member Patricia O’Neill imagined.


The invitation- to me- was a symbol of what had become of the Montgomery County school system; one that was more concerned with creating a perception and protecting a reputation than in instituting meaningful reforms. It was a county that meticulously gathered data about student performance. It paid special attention to those metrics that were gathered by outside organizations- Newsweek, AP Board, and Education Week to name a few. It then focused its instructional programs and decision-making on ways to move this data.


Consider for a moment- the release of the Montgomery County 2010 SAT scores. The MCPS press release read the following: “MCPS class of 2010 scores at record levels on the SAT.” When the local county Gazette picked up the results they explained “Montgomery County Public Schools students… achieved the highest average SAT score in the school system's history, while black and Hispanic students posted the biggest improvement in SAT scores.” And the Washington Post could not have been more favorable in their coverage, reporting that “Montgomery County public school students posted record-high scores of 1653, and the school system took steps to narrow the persistent achievement gap.” However, these headlines and articles often concealed facts that were less than flattering. But they were facts buried in raw data that few reporters or even educational experts had time to review. The headlines were simple and celebratory. It was news that made everyone feel good- so they were easy to accept as fact.

However when we deal with large organizations- the stakes of this type of misrepresentation- the type of misrepresentation that I will show occurred with the 2010 SAT results- are substantial.   They have large ramifications because school districts accepted as successful by their peers  serve as models for others.   When we read that Montgomery County closed the achievement gap- we conclude that the district has done something worthy of repetition.   But if these types of statements are merely window dressing- then what is copied is really of no social value.   In fact- it is a waste of valuable resources.  Unfortunately, I believe that Montgomery County Public schools has done much in the last ten years to increase their brand name- to increase their national and international reputation- but little to institute reforms worthy of the brand it created.


What may sound like an indictment of MCPS is really not.   MCPS plays by the rules and the incentives valued by our culture- they just happen to do it extremely well.   Under Weast, MCPS created a thesis and advanced it through what grew to be a ten million dollar communication and outreach budget.   And it was a thesis which united unions, parents, corporate leaders and academics behind it.    The thesis- however- that MCPS has systematically narrowed the achievement gap or in some other way improved the quality of teaching within the county is simply put- not supported by the data in any meaningful way.


The stakes are thus considerable.    If our models of reform have it wrong, then our educational system will not provide the human capital ready and able to compete in  (as Friedman puts it) a flattening global environment.     We must pick our models of reform well.     


My hope is to help us do just that.

Jerry Weast on Evaluation

Superintendent of Schools Jerry Weast on evaluations in Montgomery County (with thanks for the Parent's Coalition for bringing it to my attention):

Jerry Weast- No new evaluations

Kudos to the Parents Coalition for digging up this document- a letter to the Maryland Board of Education expressing his view on the new evaluation system that will go into place throughout the state.   He sounds his more reasonable tone yet when arguing agaist the state system, in particular when arguing about the ability to effectively put in place such a wide reaching system in such a short period of time.   However, he sounds the same tired tone when he talks of how effective the current evaluation system in Montgomery County is- the one that removed (or made to resign or retire) over 400 teachers  over a TEN YEAR PERIOD.   How many teachers has Montgomery County had during that time?   Well, it's 10,000 in one year.... so over the course of 10 years maybe 13,000?   More?    Not sure.  

Michael Durso, current candidate for Montgomery County Board of Education, recently said this about Weast's use of data:

The reputation of the school system, and the reporting of statistics and test scores, have been more tied into the personality and the image of [Superintendent Jerry Weast] as our leader, than the students and teachers who are impacted even more," Durso said. "We always haven't been as candid about some of our shortcomings, as opposed to publishing our positives.

Read more at the Washington Examiner. 

It's time to talk about ways to improve our current evaluation system.

MCPS and a Former Employee

A former employee from MCPS's Office of Shared Accountability  recently wrote an opinion piece in the Washington Post critical of the MCPS/Washington Post claim that MCPS is the top school district in the state.   The author, John Larson, then offered a few data points to suggest why MCPS and the Washington Post might reconsider their claim.    Jerry Weast and MCPS Board member, Patricia O'neill, then coauthored this response, providing their own data points, concluding the following:

Ultimately, we are not afraid of comparisons — in fact, we embrace them. All we ask is that those comparisons are made using statistics that actually mean something.

The employment of statistics is indeed a useful endeavor.   Data tells a story.  And the story it tells can help hardworking organizations and people get feedback about what they do well and what they need to improve.   MCPS often uses statistics to celebrate it's own accomplishments.   However, I have recently come to wonder to what extent MCPS uses data to analyze its weaknesses.

It's all in the data.

I've been emailing Jay Mathews of the Washington Post about this quote from his May 21st blog:

I asked some Washington area school districts, nationally regarded for their high performance and sophisticated management, what percentage of their teachers were rated satisfactory in the last evaluation. I subtracted the number they gave me from 100 percent to calculate what portion of their teaching ranks could be considered ineffective. The results:Fairfax County 0.9 percent, Montgomery County 5 percent, Loudoun County 1 percent, City of Falls Church 0.45 percent, Prince William County 1.7 percent. I could not find a school district in the area that admitted to having more than 5 percent unsatisfactory teachers. Most said the figure was closer to 1 percent. Experts tell me this is common throughout the land. I don't believe the numbers.

It turns out that Jay may have made a small but understandable, logical misstep. He assumed that if 95% of Montgomery County teachers are currently rated satisfactory, then 5% must currently be rated unsatisfactory. Turns out, that's not quite right. I went to an MCPS annual report and found something different. It seems that 404 of those teachers (amongst the 5%) were novice teachers. They had not yet been evaluated. The other 67 teachers (of that same 5%) were underperforming tenured teachers. Math is not my speciality, but if it is correct then if 471 is 5%, 67 must be about .7%. This means that in 2009, over 99% of Montgomery County teachers were evaluated as satisfactory. That rates right up there with just about every other county in Maryland.

Meanwhile you've got the Baltimore Sun reporting that MCPS will not sign on to Maryland's Race to the Top application because of this same evaluation system:

Montgomery's Weast told the board Tuesday that the county will not sign the application because it does not want to give up its own teacher evaluation system. In an interview, he said that the county believes its evaluation system is better than others and that the state has not fully thought through the process for many of the reforms.

Or from Weast's own letter to the Baltimore Sun in which he claims,

MCPS has a comprehensive school reform plan that is working and is mirrored in the right-minded principles promoted by President Barack Obama and Education Secretary Arne Duncan as part of the Race to the Top competition. This isn't educational theory; this is real reform, with real results.

It's almost laughable that in a 2004 report (which incidentally, was taken off the interweb by MCPS, but graciously stockpiled here by the Parents Coalition) MCPS was asking itself why so few teachers were referred to PAR, but by 2009 was concluding:

Increases in the number of teachers provided with PAR panel support indicate greater action on the part of principals and supervisors to implement the Teachers Professional Growth System.

A conclusion apparently based on the following data on the number of under performing, tenured teachers referred to PAR:














I'm not sure I'd have come to the same conclusions. But I do have my own reasons for why so few teachers receive an unsatisfactory rating. One thing should be clear, MCPS, like the rest of the state, needs new evaluation methods and new ways to remove ineffective teachers. I still do not understand the reluctance by MCPS officials to embrace a change that could keep MCPS's innovative teacher evaluation system in place, but with added reforms that make it substantially more robust. That seems to be a change everyone should be able to get behind.

Teacher evaluations to be 50% student data?

Looks like Nancy Grasmick is acting unilaterally to make student growth 50% of teacher evaluations rather than simply a "significant" portion, as required by the new state law. Interesting, because MCPS and the MCEA seemed to "slap five" after the Maryland state law was passed, arguing that they've already got it covered. It seems this is the states's shot off the bow of that celebration, and will force MCPS to go back to the drawing board.

If this is the state's way of calling out MCPS, then I'm all for it. It's time MCPS and MCEA engage in real meaningful reform that incentivizes the teaching profession. Come up with a system that rewards teachers for doing their job well. Reward teachers who work hard, not those who just get by. There is room to create this reform, even within or with additions to the current PGS and PAR system, but only if they engage the process. MCPS should lead and initiate like places such as New Haven, Connecticut are already doing. The alternative is to continue to make arguments for the status quo. Figure out how teachers can track their performance using a value-added approach to student performance. Figure out how to do the same for administrators who could potentially have more power under new system. It's high time we improved the profession once and for all.

From the MCPS page

Their title: MCPS, MCEA Agree on Use of Student Performance Data in Evaluations


My title: Weast and Prouty agree state law doesn't require any change! Rejoice.

Weast on Race to the Top

So I read this, and from what I understand, Jerry Weast's main point is that MCPS already does evaluation of teachers so well that it would have to be wary of changing anything that is already so good. He was almost dubious that Maryland could have a better idea than his. Ok then.