But the story is entering a sympathy-free zone this week, with the Browns coming to Baltimore to play the Ravens at Camden Yards.
Fans in Baltimore don't want to hear about those three long, painful years in Cleveland, not when they had to endure a dozen long, painful seasons without a team after the Colts left for Indianapolis in 1984.
Fans in Baltimore don't want to hear about how tough Cleveland had it, not when Cleveland got to keep its team name, colors and history, and Baltimore didn't get to keep anything.
Everyone is crying tears of joy for Cleveland this year, and that's fine. What happened to Cleveland should not happen to any city.
But it also happened to Baltimore, and in the final reckoning, Baltimore has endured greater indignities. Let us count the ways.
Cleveland didn't have to resort to begging a stuffed shirt like Cardinals owner Bill Bidwill to move his crummy franchise to town, as Baltimore did in 1987.
Cleveland didn't have to experience the humiliation of being deceived by the league in a fraudulent expansion derby, as Baltimore did in the early '90s.
Worst of all, Cleveland didn't have to suffer through the ultimate self-debasement of stealing another city's team to satisfy its pro football fix, as Baltimore did when, out of options, it lured Art Modell's Browns with the promise of a new stadium four years ago.
Cleveland knew it was getting a team back within three months, a record turnaround. Fans there knew almost immediately that football was coming back -- without the owner they hated, no less.
In Baltimore, they had to wait 12 seasons for a guy with a monster debt service to pull into town.
Starting to understand why the "plight" of the Browns' fans isn't wetting many hankies around here?
Part of the reason, no doubt, is that Baltimore still feels ashamed somewhere deep down. It was a terrible, regrettable irony that Baltimore did to Cleveland just what Indianapolis had done to Baltimore. No one wanted that. That's why Baltimore should be as happy as any other city that Cleveland is back in the NFL. So much for any lingering guilt.
Is there jealousy, too? Probably. Cleveland got everything back in a relative hurry, as well as a new owner with tons of money and a new stadium partly paid for by the NFL. Baltimore also got a team and a new stadium (without a penny's help from the NFL), but Modell needs an investor and Johnny Unitas is still an Indianapolis Colt in the Pro Football Hall of Fame.
When Jim Brown becomes a Baltimore Raven in the Hall of Fame, Cleveland can start equating its football pain and suffering to Baltimore's.
Oh, sure, it was just as devastating when the Browns left there as it was when the Colts left here. Neither city deserved the fate. The fans were just pawns in a high-stakes game being played by owners, politicians and banks.
In that sense, the cities and fans have more in common than not, not that that will keep them from raging at each other Sunday.
Still, for some reason, Cleveland seems to believe it has suffered more. That's highly debatable. Both franchises were deeply ingrained in the community. Both had proud histories. The Colts' attendance waned more than the Browns' at the end, but the Colts had become a joke under Robert Irsay's rule, a team not worth supporting. As disgusted as Browns fans became with Modell, he never fired coaches at halftime or called plays from his box, drink in hand. He just didn't win. Then he left.
In the end, the stories are the same, really, only with different names, dates and villains. Town loses team. Town gets team back.
But for some reason, saving the Browns for Cleveland became politically correct in the NFL's corridors after Modell's departure, and saving the Colts for Baltimore never became an issue after Irsay left town.
For some reason, Cleveland without an NFL team struck a chord, and Baltimore without an NFL team didn't. So one city waited three seasons and the other waited 12.
Whatever. It's all old history now. Both cities have new teams, new stadiums and plenty to cheer and boo. Both are on the inside again, looking out at all the cities that wish they had teams.
The NFL wasn't whole without the Browns, and now it is again, after three years.
But the NFL also wasn't whole without a team in Baltimore, one of the league's cornerstone cities. And that wrong took 12 seasons to right, unlike the one in Cleveland.
In fact, the wrong in Baltimore will never be righted entirely, at least not as entirely as the wrong in Cleveland was righted.
So the Browns should expect no sympathy here.