Setting test standards is group effort

THERE'S HEAVY security these days at the Mount Washington Conference Center. Men and women (mostly the latter) huddle in groups of five to eight, shuffling papers and talking animatedly. Guards are stationed outside each meeting room.

They are not designing a top-secret aircraft. These are 150 teachers and other educators gathered from every corner of the state for three days. Their job is to help determine the passing scores on the new Maryland School Assessments.


The papers they're poring over are the reading and mathematics tests given in February to more than 250,000 Maryland kids in grades four, six and seven. Thus, the need for security; the items are still "live." Participants pledge in writing that they won't disclose test items, even to their spouses.

Setting standards -- that is, establishing the "cut scores" for the proficient and advanced levels -- is a scientific endeavor, but it requires the human activities of voting and compromise.


In Maryland and 31 other states, standards for passing (and thus making "adequate yearly progress" under the No Child Left Behind Act) are established in a process called bookmarking that was developed in 1996 by CTB/McGraw-Hill, the giant testing firm that is coordinating the Mount Washington conference.

The process

It works this way:

First the teachers take the tests themselves. Then they are presented with what's called an "ordered item booklet," which is a ranking of the questions from easiest to most difficult.

After a lot of discussion, each teacher marks the easiest question for which a correct response would qualify a student as proficient, and the easiest question for which a right answer would earn the advanced grade. There's more discussion of the test items, followed by a second round of bookmarking.

Before the third round, the teachers see what are called "impact data." Thanks to the computer, they are shown the pass rate derived from their bookmarking. Because the tests were taken five months ago, the state Department of Education knows how many children answered each test item correctly, so judging the effect of any cut score is a matter of a couple of key strokes.

The trick, testing experts say, is to set the bar for passing (or proficient) not too high and not too low. After all, under the No Child Left Behind Act, the states have until 2014 to move all children from the basic (or failing) to the proficient level.

But that essentially political decision is not to be made by the teachers, principals and curriculum experts gathered today in Mount Washington. Theirs is an educational recommendation based on state and local curriculum requirements and their own expertise. At what point on that continuum of questions does basic (failing) become proficient (passing), proficient become advanced? How many children pass at that point is for others to worry about.


"It's actually a pretty exciting process," says Ann Mintz, Howard County's coordinator of English and language arts who is involved in a second round of standard-setting. Last year, the teachers, who are nominated by their school systems, did the same thing for grades three, five and eight.

Those on the inside of these secured rooms say the discussions can become heated. "Everyone takes on the perspective of the population they serve," says Lisa Seymour, an instructional specialist in Montgomery County. But Mintz says the process forces participants to "represent all of the children in Maryland."

There's more to come in the next few days. A committee of testing experts will look at the recommendations. An "articulation" committee will make sure that the new fourth-grade standards, for example, aren't wildly different from the third- and fifth-grade cut scores approved last year.

Finally, sometime over the weekend or early next week, state schools Superintendent Nancy S. Grasmick will decide her recommendation, which she will make to the state Board of Education Wednesday in Baltimore. The next day, if all goes well, we'll be able to report how many Maryland kids passed this year's assessments in grades four, six and seven.

A sample question

Here is a sample fourth-grade math question presented to the teachers Monday before they took the secure Maryland tests. Would a student answering correctly be proficient?


Sam can purchase his lunch at school. Each day he wants to have juice that costs 50 cents, a sandwich that costs 90 cents and fruit that costs 35 cents. His mother has only $1 bills. What is the least number of $1 bills that his mother should give him so he will have enough money to buy lunch for five days?

Answer: nine.