The former Pro Bowl pass rusher emphasized that his strong opinions are shared around the league, also mentioning former New York Jets and Ravens linebacker Bart Scott as having the same take.
"The NFL is not very big," Suggs said. "You think we don't talk to guys that have played for the New England Patriots, that have been on the New England Patriots that have been like, 'Oh, it's been like this.'
"It ain't just me. Why did Bart Scott say the same thing? You think it's just us? You think it's just got something to do with us? No. This is because we have inside information. We know."
During the interview, Suggs didn't offer specifics while saying that players who have played for New England have mentioned issues within the organization.
The Patriots first became unpopular because of Spygate and illegally taping the Jets' signals, which triggered punishment from the NFL in terms of fines and losing their first-round draft pick.
Suggs has complained in the past of Brady getting special treatment from officials as far as being able to draw unnecessary roughness penalties.
In the past, Suggs has also has brought up the famous "tuck rule" in the playoff game won by the Patriots over the Oakland Raiders in 2001.
After the Ravens defeated the Patriots in the AFC title game at Gillette Stadium, he called the Patriots "the most arrogant [expletives] in the world."
And Suggs, who's done several radio interviews this week to promote his film company's movie, "The Coalition," which he helped write, didn't back off those remarks.
"Do I apologize for what I said? No," Suggs said. "Do I mean what I said? Yeah. Could I have worded it a little better? Probably. But the fact of the matter is you can't really consider it a rivalry because you have a few more championships than we do, but this has been steaming for a while."