"I just heard about it," Pagano said during a conference call with Baltimore reporters. "I think it's more of maybe a challenge to our 12th man, maybe calling out our 12th man and our fans that they ain't loud enough. I hope our fans take it that way."
When asked if he thinks the Colts' fans will respond by cranking up the volume Sunday at Lucas Oil Stadium, Pagano replied: "Well, they've been challenged. So, I hope so. They always are."
Harbaugh first jokingly broached the allegations about the Colts using fake noise at their indoor stadium during his Monday news conference.
"Rumor has it they pipe crowd noise in there, so we'll see if that's the case or not going back there again," Harbaugh said. "But we'll be looking forward to it."
The Colts have been accused in the past of piping in crowd noise, but those allegations were never proven.
After a 2007 game against the New England Patriots when the noise was particularly amplified when quarterback Tom Brady was trying to call signals, CBS took the blame for a sound malfunction that could be heard on the telecast.
"It's a rumor, it's a rumor," Harbaugh said. "Sorry, Chuck."
When told of Pagano's comments Wednesday, Harbaugh replied: "I've challenged their 12th man? He's going to get his crowd all riled up. Chuck is a great motivator. We don't care how loud it is.
"You go everywhere, it's going to be loud. So, we have to handle it. It's our job to quiet them down. That's what we try to do."