After years of holding schools accountable for student test scores, the idea of using those scores to evaluate teachers and determine their pay has become the latest battleground in education across the nation.
This past school year, Maryland's 60,000 teachers were evaluated for the first time according to a formula that required half of their final rating to be based on how much their students learned.
Policymakers and proponents of the new evaluation systems hope that eventually they can be used to get rid of poorly performing teachers and reward the best with higher pay. But teachers unions contend that further emphasis on test scores narrows the focus of learning and that effectiveness in the classroom is more complex than a score.
The best models for evaluating teachers help them "enhance their professional practice rather than simply narrow their focus to high-stakes standardized testing," said Adam Mendelson, a spokesman for the Maryland State Education Association, the union that represents most of the state's teachers.
While unions are collaborating with Maryland education leaders to devise an evaluation system that works, they say it is too early to say whether the new measures — including test scores — are a valid way to judge teachers.
The results of the state's pilot evaluation program that took into account test scores varied widely, according to data obtained by The Baltimore Sun through a public records request. None of the teachers in Baltimore County, for instance, were found to be ineffective, compared with 7.2 percent in Anne Arundel County.
Some counties did not test a random sample of teachers for the pilot program during the 2012-2013 school year and instead selected participants, excluding teachers who were struggling.
But in Baltimore, the only school system to evaluate every classroom teacher in the pilot, 10 percent were rated ineffective under the new system. About 10 percent of the city's teachers were rated highly effective, and the remaining 80 percent fell into the effective category.
Two years earlier, 200 of about 4,000 city teachers, or 5 percent, were rated unsatisfactory.
In the past, teachers were evaluated based on a formal observation by a principal as well as other subjective elements, such as how well teachers constructed a lesson plan or managed a classroom. A tiny percentage of teachers across the state were rated unsatisfactory.
Despite concerns by local school leaders that another year was needed to iron out problems with the new evaluations, Maryland took its system live for the 2013-2014 school year.
Some school districts such as Baltimore County decided not to tie pay to the evaluations and put more emphasis on trying out the new system, while others such as Baltimore City did use the evaluations when deciding pay. In the city, a teacher's job performance was boiled down to a single percentage score that corresponded to raises, or none.
Districts across the state were supposed to use standardized test scores in evaluations, but after local and national backlash, Maryland and other states have backed away from the direct use of scores until the 2016-2017 school year.
Student test scores plunged last school year because of a disconnect between the material that students were taught and what they were tested on. The state has been implementing the Common Core, a set of stricter standards, but hasn't yet changed the state tests to reflect the changes. The issue became a political controversy, and the Maryland General Assembly weighed in by prohibiting the use of test scores in teacher evaluations for two years.
Instead of using state test scores, teachers and principals work together to set goals, called student learning objectives, at the beginning of the school year. Teachers then write or choose their own tests to administer to their classes, giving them at the beginning and end of the year to measure whether students have met the goals.
Teachers and principals acknowledge that they are still working out the kinks with student learning objectives. As a result, the state's boards of education, teachers unions and superintendents signed a memorandum of understanding this week, agreeing to work together to make student learning outcomes more uniform and valid.
Those learning objectives and other measures of student growth will be worth 50 percent of a teacher's evaluation until the use of state scores takes effect. The state school board may require that 20 percent of a teacher's evaluation be based on learning objectives that take into account test scores. For example, if a teacher's class did poorly on fractions in a state test, her goal could be to raise their achievement on fractions the following year.
Battles have broken out in Maryland and the nation over whether the new evaluations accurately measure a teacher's performance.
Baltimore teachers recently protested on the steps of North Avenue headquarters over last-minute changes to their evaluations that lowered some teachers in the rankings and threatened to reduce pay.
Even some of the education advocates who led the charge for reforming teacher evaluations are now seeking to slow implementation to prevent backlash.
The District of Columbia's public school district, one of the first in the nation to tie evaluations to test scores, announced two weeks ago that it was putting part of its system on hold until students and teachers adjust to the new Common Core and the tests that go with it.
And this month, the Bill and Melinda Gates Foundation, a powerful advocate for tougher teacher evaluation systems, backed away from supporting the use of test scores for a few years.
Abby Katherman, a fifth-grade teacher at Prettyboy Elementary in Baltimore County, is among those who support the new evaluation system and believes it is fair to be judged on student achievement.
Principal Susan Truesdell and all the fifth-grade teachers agreed to the student learning objectives in September: Struggling math students would have to raise their scores by at least 10 percent by the end of the year, and those who were already advanced would have to maintain that level during the school year or gain 5 percent on the year-end test. They also set goals for reading.
"I feel the year was a huge success," Katherman said. "We have made tremendous growth."
She said the school's fifth-grade teachers became more cohesive as they learned to lean on one another more and that the evaluations put an intense focus on improving their methods. She's considered a star teacher by Truesdell and received a good evaluation.
But not all teachers in the region have embraced the process, and Truesdell believes it will be several years before the evaluation system is well understood and implemented.
"The teacher evaluation is a process, and I think it is something we have initiated this year, and we will continue to get comfortable with it," she said. "We are still learning it and working through it."
Other teachers worry that schools are moving too quickly to make personnel decisions through an experimental evaluation model.
Stacie Pare, a seventh-grade math teacher at Oakland Mills Middle School in Howard County, is a national board certified teacher, a designation achieved by only a few thousand teachers in the state.
She believes that teachers should be evaluated on how much their students achieve but says the Howard system puts too much emphasis on the goals, which might not take into account all classroom circumstances.
In math, for instance, teachers needed 90 percent of students to meet student learning objectives such as improving achievement on fractions or on mathematical reasoning.
Pare said she barely met the 90 percent goal because one student in a class of 17 did not achieve as much during the year as expected, largely because of an illness in the child's family and other factors outside her control. Pare was still rated highly effective, the top ranking.
"How is that going to affect a teacher that has a class that is struggling? A lot is based on the class you get rather than your actual ability," Pare said.
The state hasn't made available any data on how many teachers were rated ineffective this year under the new system.
Jennifer Bell-Ellwanger, achievement and accountability officer for Baltimore's public schools, said the city decided that all teachers should participate in the pilot evaluation two years ago in an effort to make the new system "really fair and really rigorous."
Kate Walsh, president of the National Center on Teacher Quality, said the outcome in Baltimore from the pilot year — when the spread between ineffective, effective and highly effective was 10 percent, 80 percent and 10 percent, respectively — "comes closer to reflecting the actual distribution of teacher talent than many other pilots currently underway."
Walsh, a former member of the Maryland school board, said national research has shown that about 15 percent of teachers are weak, meaning that their students achieve on average about half what they should in a school year. At the other end, about 15 percent of teachers are considered extraordinarily strong because their students learn the equivalent of a year and a half's worth of material.
Billy Burke, Baltimore County's assistant superintendent of organizational development, said the county decided to ask teachers to volunteer for the pilot evaluation and excluded struggling teachers. Only 333 of the county's 8,000 teachers took part, and 82 percent received a highly effective rating. None was rated ineffective.
In Howard County, 126 teachers took part in the pilot. Less than 1 percent were rated ineffective, about 60 percent were effective and 40 percent were highly effective.
In Anne Arundel, 208 teachers took part, with 7 percent rated ineffective, about 49 percent effective and 44 percent highly effective.
In Harford and Carroll counties, 100 or fewer teachers took part. Of those who participated, the split between effective and highly effective was roughly 60 percent to 40 percent. In Carroll, about 2 percent were rated ineffective. No Harford teacher received the lowest ranking.
The pilot "was a practice exercise to see if formulas worked. Drawing conclusions from this data would be ill-advised," said Bill Reinhard, a spokesman for the state Department of Education.
When school districts reported the data from the pilot to the state last year, Burke said he and many of his counterparts in other counties told state leaders they believed they were being rushed to complete the evaluation system and asked for another year. While that didn't happen and the kinks continue to be worked out, he said the process has been worth it.
"It made people have deep, rich conversations about how students were learning in the classroom," he said.