It was the earliest loss at a major since 2008 for six-time Australian Open champion Djokovic, who had never dropped a set to Istomin in six previous meetings.
It was destined to be a long afternoon at Rod Laver Arena for Djokovic, though, when he needed 24 serves and 16 minutes to hold in the first game, saving six break points.
He lost the 85-minute first set in a tiebreaker, seemed to get the momentum back only for No. 117-ranked Istomin to finish stronger for a 7-6 (8), 5-7, 2-6, 7-6 (5), 6-4 win.
“All the credit to Denis for playing amazing. He deserves to win,” Djokovic said. “He was the better player in the clutch moments.
“Obviously, I was not pleased with my performance overall, but I have to congratulate my opponent. Whenever he needed — he came up with a big serve, big play.”
Serena Williams, who is also ranked No. 2 and is a six-time Australian Open champion, kept her chase for an Open-era record 23rd major title on track with a 6-3, 6-4 win over Lucie Safarova in a night session delayed by Djokovic's defeat.
No. 3-ranked Agnieszka Radwanska didn't last much longer than Djokovic, losing her second-round match 6-3, 6-2 to Mirjana Lucic-Baroni. Radwanska reached the semifinals here last year; Lucic-Baroni hadn't won an Australian Open match since 1998 until her first-round win this week.
Istomin had never beaten a top-10 player at a Grand Slam tournament.
“It is the biggest win for me. It means so much,” Istomin said. “Now I feel I can play with these guys, and to be with them on the same level.”
His only previous win in 33 matches against a top-10 player was in 2012 at Indian Wells, when he beat No. 5 David Ferrer.
Djokovic had reached the fourth round or better every year since 2007 — the year before he won his first Grand Slam title at Melbourne Park — and held the No. 1 ranking for 122 consecutive weeks until he was overhauled by Andy Murray last November.
“I'm not used to losing Australian Open second round,” Djokovic said. “I've done so well here.”
The last time he failed to advance beyond the second round at a major was at Wimbledon in 2008, when he was seeded third and lost to then No. 75-ranked Marat Safin, a two-time Grand Slam champion.
Rod Laver Arena wasn't at capacity as the match extended into the scheduled night session, and Djokovic's usually vocal Serbian fans were also not a big presence. Much of the crowd, sensing an upset, was behind Istomin in the fifth set, yelling “Come on, Denis!”
When Istomin made a backhand winner down the line to break Djokovic to go up 3-2, he let out a roar and the audience stood and roared, too.
When Djokovic's last service return floated long, the crowd leaped up and cheered.
“First of all, I feel sorry for Novak,” Istomin said, thanking his mother and coach, Klaudiya Istomina, in the crowd. “I was playing so good today. I surprised myself as well.”
He next plays No. 30 Pablo Carreno Busta.
Third-seeded Milos Raonic shrugged off a cold and a sore throat to extend his streak of reaching the third round at all seven Australian Opens he's contested, with a 6-3, 6-4, 7-6 (4) over Gilles Muller, the Sydney International champion last week. He next faces No. 25 Gilles Simon.
Ivo Karolovic followed up his tournament record 84-game first-round win with a straight sets win over wild card Andrew Whittington.
In other women's matches, U.S. Open finalist Karolina Pliskova and No. 9 Johanna Konta carried their winning form from warmup tournaments into the third round.
Brisbane champion Pliskova beat Anna Blinkova, 6-0, 6-2, and Sydney International winner Konta, a semifinalist here last year, beat Naomi Osaka to set up a third-round match against former No. 1-ranked Caroline Wozniacki.
Murray, a five-time Australian Open runner-up, returned to the practice court to allay concern over his ankle, 14 hours after tumbling to the court during his second-round win.