Many teachers in Seattle Public Schools will learn today how they rank on a new scale based on student performance. They’ll be rated by how their students’ test scores changed from one year to the next. Teachers with high ratings may qualify for bonuses or promotions. Teachers with low ratings will get closer oversight.
But researchers say using student test data to measure teacher quality can be misleading.
In 2010 contract negotiations between Seattle’s teachers union and the district almost broke down over the issue. They finally agreed on a cautious approach to using student growth in teacher evaluations.
New Step In Teacher Evaluations
The main way teachers are ranked under Seattle’s two-year-old evaluation systemis through classroom observations. But for many teachers, student test scores will now also play a role. Teachers in about one third of the schools in the district will get ratings this year. The rest will get ratings next year. Their students’ growth on state and district tests will be ranked as low, typical, or high. So far, the scores only apply to reading and math teachers in certain grades.
Seattle Public Schools manager of research and evaluation, Eric Anderson, says the student growth scores won’t get teachers fired, but they will be part of their overall evaluation. "We believe it’s important for a school system to measure student progress as accurately as it possibly can, and we think that this data can inform the evaluation process," Anderson says.
Teachers who get high scores on both student growth and classroom observations may qualify for promotions and higher pay. Teachers with low growth scores will get extra classroom observation by their principal and more training opportunities. If they don’t improve, they could end up on probation according to the teacher contract.
Anderson acknowledges that a lot of factors can affect student test scores. So the district’s formula takes into account the number of students in a class who are living in poverty, learning English, or have disabilities.
Mathematicians Urge Caution
But no matter how elaborate the system for weighting student test scores, mathematician John Ewing says Seattle can’t use the data to draw conclusions about student growth. "One year of growth for one teacher is meaningless," he says. A teacher typically only has a student in their class for one year. The district is taking two of those sets of one-year growth data per teacher, and averaging them.
Ewing was a math professor for decades, and now heads the organization Math for America. He wasn’t involved in the decision about how to measure student growth in Seattle. But he published a paper in the journal of the American Mathematical Society about the dangers of using student test scores to measure teacher quality. "You might as well look at all the teachers and flip a coin and those that get heads, say, are good, and those that get tails are bad, and it’s not much different from using one year of growth to measure teachers," he says, even if you’re taking two of those years and averaging them. Ewing says so many variables affect a data set that small that it creates a measurement problem - what statisticians call "noise." A teacher’s student growth rating can swing wildly from one year to the next.
Teachers Feeling Nervous
In order to qualify for federal grants, many states now require student growth to comprise of half of a teacher’s evaluation.
Washington doesn’t have that rule.
Seattle Education Association President Jonathan Knapp says his union fought to make student growth a relatively minor element in teacher evaluations. But he says, Seattle teachers are still worried that public perception could lead to unfair shaming of low-scoring teachers.
"The fear is that this'll become a game of unsophisticated, facile analysis that 'oh, this student rating is this and that one is that, so that means this teacher is good and that teacher is bad, and that’s all there is to it,'" Knapp says.
In 2010 the Los Angeles Times published an online database with teachers’ names and their student growth scores. One of the lower-rated teachers was a beloved mentor and tutor in a gang-ridden neighborhood.
A few weeks after his score was published, he jumped off a bridge.
In Seattle, district officials say they intend to keep individual teachers’ student growth data private. The district plans to release a broad snapshot of student growth scores to the public next week, once teachers have gotten to see how they rate.
