One of its earliest superstars, Cap Anson, perpetuated a culture of fierce segregation that scarred the game for decades.
All-time hit leader Pete Rose can't enter the Hall of Fame because he bet on the sport. Some of the biggest stars of the 1980s became embroiled in cocaine trials. The 1994 World Series was lost to labor strife.
Always, baseball bounced back.
So it's little surprise that, after years of widespread steroid abuse were described in former Sen. George J. Mitchell's report on Thursday, many historians, economists and baseball men expect the game to survive and thrive.
Their optimism is countered by dire assessments from Congress and by former commissioner Fay Vincent, who believes that the integrity of all sports is in peril.
But those who predict a successful future needn't look far for evidence. Baseball has already lived under a steroid cloud for almost a decade but just experienced its most lucrative year ever, earning more than $6 billion in revenue.
Fans feel a greater intimacy with baseball than with other sports, said longtime sportswriter and commentator Frank Deford, and that means they're more wounded by its letdowns but also apt to forgive.
"It's this sense of family with the game," he said. "Sometimes, your family makes mistakes, but in the end, you love them just the same."
Drug scandals are harder on individual sports, he said, but baseball fans can always return to the teams and ballparks they love after the tainted players are gone.
"By June, this will all be in the background," Deford said. "After all, if fans were really upset, they wouldn't be packing the stands."
San Francisco-based historian Jules Tygiel has written about how segregation and other aspects of baseball's past fit into the broader culture. He agrees that baseball will roll on and, in fact, called steroids a lesser scandal than gambling, segregation and labor unrest.
"I think very little will change in the game's popularity," he said. "I suppose this has done some harm to the public image, but I'm not sure how much that means. Baseball recovered quickly from the Black Sox scandal. It's pretty resilient."
Baseball has yet to find the scandal that can lead to a true economic downturn, according to Forbes associate editor Kurt Badenhausen.
"At the end of the day, fans don't care about steroids when it comes to the teams they root for," he said. "People vilify [Barry] Bonds around the country, but in San Francisco, they've drawn 3 million a year. Maybe this devastates individual reputations and the legacy of [commissioner] Bud Selig, but the game's never been healthier."
Some who love the game take a darker view of the steroid issue. The problem is not endemic to baseball, said Vincent. Instead, it's a threat to our faith in all sports.
"It's a staggering problem," he said. "We can't have sports if we can't have level playing fields."
That's why Vincent wants Congress to go beyond hounding baseball and convene a "council of wise men" to consider the issue in its fullest scope.