"My comment was there was no way," Harbaugh recounted. "I might have added a word or two in there, too."
Skepticism turned into elation in the second round where the Ravens jubilantly selected Smith and gushed about the potential of their best deep threat.
Smith, 22, has the size (6 feet, 205 pounds). He has the speed (he ran the fourth-fastest time among wide receivers at the NFL combine). And he has the stats (15.7 yards per catch and 12 touchdowns last season for the Terps).
"You saw the celebration with Torrey on TV. I want you to know there was a bigger celebration in our draft room," Harbaugh said. "This is a guy from the beginning that we saw on tape ... he fits us. He's our kind of guy. He's our kind of personality. But he's also a type of player that we really want and we really need."
The Ravens needed a speedy receiver to stretch the field. In starters Anquan Boldin and Derrick Mason, they have veteran wideouts who make all the catches but won't run past cornerbacks. Last season, the Ravens wide receivers had five receptions over 40 yards.
Smith brings an element that "strikes fear" in defenses, general manager Ozzie Newsome said.
"We play against other teams and a slip here, and it's a touchdown. And that's deflating," Newsome said. "With everything we already have, we just added the 3-point shooter. At any point, the ball gets in his hands and Billy [Cundiff] is coming out to kick an extra point."
Even though Smith can beat defenses deep, he isn't a one-dimensional receiver who runs down the sideline.
"He shows a willingness to attack the middle," Hortiz said. "He's not afraid. He takes some hits and continues on."
That toughness was instilled early on in Smith. Growing up in Colonial Beach, Va., Smith helped raise his six younger siblings and served as the man of the house while his mother worked multiple jobs and went to school at night.
Harbaugh remembers reading about Smith's background in a newspaper article in February.
"I was choked up reading it," Harbaugh said. "I was so proud of this young man just reading the article. My thought was: We have to find a way to make Torrey Smith a Raven. This guy is what this organization is all about."
Smith said he's become stronger because of his past.
"I think everything I've been through, it's going to help me out," Smith said in a conference call with Baltimore reporters. "It might not have been a perfect situation and things might not have always been right, I feel like it helped me mature and grow as a man."
The addition of Smith likely means the Ravens won't re-sign veteran free agents T.J. Houshmandzadeh or Donte' Stallworth.
The Ravens selected Smith over other wide receivers like Miami's Leonard Hankerson, Kentucky's Randall Cobb and North Carolina's Greg Little.
But it was considered an easy decision to use the 58th pick on Smith, who was among the Ravens' top 32 players on their draft board.
"He's an explosive playmaker," Hortiz said. "Today, we got bigger and faster."
Like Hortiz, Smith thought that he would end up with a team that resides a short drive from College Park.
"I had a good feeling from the get-go, even from what I knew before I was even there," Smith said. "I knew the way I am as a person and the way I play kind of fit the way they do things up there. I knew it was a great possibility of me going there."
In the third round, the Ravens moved up five spots to take Central Florida offensive tackle Jah Reid. The Ravens traded their third-round pick (90th overall) and a sixth-rounder (191st) to the Philadelphia Eagles to select at No. 85.
It was a surprising move considering Reid had been projected by some draft experts to go in the fifth or sixth round.
"I think he's very underrated," ESPN draft analyst Todd McShay said. "Jah Reid has a chance to be a starter in this league."
Reid, who is 6 feet 7, 330 pounds, is expected to play right tackle, which suggests that Michael Oher could stay at left tackle.
The knock on Reid is a lack of power, average instincts and marginal foot quickness. As heavy as 375 pounds early in his career, Reid worked hard to get in better shape and started to gain attention with a strong effort last season against Georgia and at the East-West Shrine game.
Harbaugh pointed out Reid's "mean" attitude."
"I think he's going to develop fast," Harbaugh said. "This is a guy we all like. He's on the upswing. He'll be competing for the right tackle spot right away, so we'll see where it goes."
The Ravens wrap up the draft Saturday with five picks in the final four rounds.