If Aaron Schatz, founder and editor of the analytics website Football Outsiders, oversaw the blueprints for a new head coach designed at some far-off, modern football laboratory, his creation might look like a typical football coach but not think like one.
The coach would have the offense pass more than run, even more than at prevailing NFL rates. The coach would rely heavily on play-action and almost never run on second-and-long. The coach would go for it on fourth-and-short, even in their own territory. The coach would go for 2-point conversions when their team trails by double digits late and when a penalty moves the ball to the 1-yard line.
The coach would, in other words, act a lot like the John Harbaugh who became a darling of football’s analytics community Sunday, even in defeat. In their 33-28 loss to the Kansas City Chiefs, the Ravens attempted four fourth-down conversions, three of them in the first half; went for three 2-point conversions, including after the game’s opening score; and played not a “field-position game,” Harbaugh said Monday, “but a possession game.”
It was not quite Schatz’s Create-a-Coach come to life, but it was close. Afterward, Harbaugh explained he had followed the numbers, not his gut. To some vocal fans with the advantage of hindsight, the decision-making registered as questionable at best, coaching malpractice at worst. For the sport’s quants and number crunchers, this was a small victory in the fight for smarter football.
“The thought was always that a coach would not be willing to own analytics because of the fear that the owner would fire him if it backfired,” Schatz said in a telephone interview. “It just shows the whole franchise-wide commitment to the idea.”
Harbaugh has long been among the NFL’s bolder coaches. In Football Outsiders’ annual Aggressiveness Index, he’s typically ranked among the NFL’s most aggressive on fourth down. Last season, according to a Sporting News review of NFL coaches, Harbaugh went for it on fourth-and-1 80% of the time, tied for the second-most aggressive rate. (In first was the Chiefs’ Andy Reid, under whom Harbaugh previously worked on the Philadelphia Eagles’ coaching staff.)
But even Harbaugh acknowledged Monday that the Ravens’ “analytics guys will tell you that I don’t follow the analytics nearly enough. They’ll tell you that I go by my gut way more than I go by the analytics.” On that point, he is probably not alone. One 2018 study by Derrick Yam, a recently hired Ravens quantitative analyst, and Michael Lopez, the NFL’s director of data and analytics, found that the value of a more aggressive fourth-down approach for an average team could be 0.4 wins per year.
According to the study, published recently in the Journal of Sports Analytics, NFL strategies have been largely static: From 2004 to 2016, in regular-season situations that The New York Times’ 4th Down Bot would have gone for it, the rate of teams’ fourth-down aggressiveness fluctuated between 12% and 17%.
With the growing embrace of quantitative analysis by coaches like Harbaugh and the Eagles’ Doug Pederson, the NFL might be finally turning a corner, if haltingly. According to Josh Hermsmeyer, a football writer and analyst for FiveThirtyEight, coaches went for it on fourth down 14.9% of the time last season, the league’s highest such rate in the past 10 years. And still, Hermsmeyer said, teams aren’t being bold enough in their decision-making.
“We’re pretty humble when it comes to what we say analytically in terms of what the numbers say,” he said. “We’re really, actually, pretty conservative. So when we’re sitting here pounding the table on these things, it’s because it’s really, really clear.
“I don't think we're going to see these huge edges come from the rest of the data. We might. But I think these major edges we've already identified [such as on fourth down] are going to be the big ones. And if the NFL won't even embrace those, it just seems to me that it's going to be an uphill battle.”
The problems are largely systemic, analysts said. Michael McRoberts, founder and president of Championship Analytics, a company that provides college football and NFL teams with data that can inform in-game strategic decisions, said going against the sport’s long-held conventions can be difficult for coaches. A 2017 study of college coaches found that they became increasingly conservative the more likely they were to be fired.
For all their expertise, coaches are “definitely human,” McRoberts said. When coaches go for it on fourth-and-1 instead of punting and don’t convert, they know they’re “going to hear about it from the press,” he said, even if both outcomes would have led to the opponent scoring.
It’s why, when new clients make an in-game decision that’s analytically driven, McRoberts said he’s “really, really hoping” that they’re successful. Without positive feedback, he said, some coaches fall back on old habits.
“There's got to be an education that goes along with these numbers,” he said. “You just can't put a chart in front of a coach that's been coaching for 20, 30 years and say, ‘Hey, just follow this.’ ”
The NFL has long been a copycat league, as Hermsmeyer said. Successful coaches with conservative strategies have bred new coaches with equally conservative strategies. “It’s almost akin to the priesthood,” he said, “in that the only real wisdom in football for ages and ages was that which was delivered at the feet of the master.”
The rise of analytics has challenged norms, sometimes uncomfortably. Of Harbaugh’s aggressive moves Sunday, maybe none was criticized more than his ultimately unsuccessful decision to go for 2 when the Ravens, after scoring early in the fourth quarter Sunday, could’ve cut the Chiefs’ lead to 10 with an extra-point attempt. But according to a FiveThirtyEight analysis from 2017, the difference between a lead of nine points and 10 points is equivalent to a win probability difference of 2.2%. And Kansas City’s defense had just endured a 13-play drive.
“I wouldn’t say [the decision to be aggressive] is hard in that sense,” Harbaugh said Wednesday. “You just have to decide if you think you have a good chance to be successful with it.”
It is possible, maybe even likely, that Week 4 Harbaugh is unrecognizable from Week 3 Harbaugh. The Cleveland Browns will be a different test at M&T Bank Stadium than the Chiefs were at Arrowhead Stadium. According to Football Outsiders, Kansas City has the NFL’s second-most efficient offense but only the No. 21 defense; Cleveland is No. 28 and No. 8, respectively.
In the analytics community, there is hope that Harbaugh’s calls Sunday are not just an aberration, that they will embolden him and others to believe in the numbers. He was, after all, more successful than not. On two of the three drives in which the Ravens went for it on fourth down, they scored a touchdown. They couldn’t have failed on either subsequent 2-point conversion had they not kept the sticks moving in the first place.
“You want to see teams that are maximizing win probability, that are trying to win,” Hermsmeyer said. “These coaches will go out there and they’ll say things like, ‘It’s a game of inches,’ and, ‘It’s all about winning. Winning’s the only thing.’ And then they go out there and they leave points on the field. They leave wins on the field. And there’s nothing more frustrating, just from a fan’s perspective.”