The red ink continued, with a loss of $4.9 million in 2009. Near the end of the year, examiners told the bank that they would recommend a consent order. It was issued in April 2010 and spelled out steps that Harbor agreed to take.
"It's a serious matter," said Stuart Greenberg, a banking consultant in Baltimore. "A bank is required under regulatory guidelines to operate in a safe and sound manner. When it enters into an arrangement with regulators, they are not operating as safe and as sound as they should be."
Consent orders also are public and can raise a bank's borrowing costs and affect its relationship with other lenders, Greenberg said.
"Make no mistake, this is a scarlet letter," he said.
Regulators declined to comment, saying the order speaks for itself.
The Federal Deposit Insurance Corp.'s order laid out a course of action that included reducing Harbor's commercial real estate loans and narrowing a gap in its capital levels.
Haskins said he doesn't believe the consent order was warranted but viewed it as a road map. Not all his board members took the news as well.
"These guys didn't arrive at where they are by being sort of wimps or pushovers. They wanted to take the regulators to task," Haskins said.
He had some long talks with the board, explaining that, given the financial crisis, it wasn't wise to put up a fight. If the bank complied, he argued, it could do the painful work of meeting the terms of the order and restore the bank's health.
"Reluctantly, they agreed with me," he said.
State Sen. Delores Kelley, a director since the bank's founding, said the situation reminded her of an old proverb she heard growing up: "When your head is in the lion's mouth, don't wiggle."
Regulators gave bank officials a month or two to come up with a plan to comply with the order.
"You feel you have little option other than to deliver what's been asked lest there will be further repercussions," Haskins said.
The bank set about reducing its commercial real estate loans, writing off troubled loans and having some borrowers refinance loans with other lenders.
Harbor lost $4.1 million in 2010 but made profits of $298,000 in 2011 and $364,000 in 2012.
Some directors called working under the order difficult.
"Exasperating, really exasperating," said George Vaeth Jr., a retired architect and longtime director. "I personally felt that the target kept moving."
Examiners would want the bank to concentrate on one area and then return later wanting it to focus on something else, Vaeth said.
"It's not as tough for the board as it is for the people who work for you," Paterakis said. "They are under tremendous stress by the government telling you that you have to do this and you have to do that. You can't argue."
Director James DeGraffenreidt, retired chairman and CEO of WGL Holdings, said regulators set a high bar for Harbor, but in the end it made the bank stronger. "The heightened scrutiny from the financial crisis caused everybody in the bank to elevate their game," he said.