The 66-year-old comedian is network TV's longest running latenight host.
Leno got Carson's "Tonight" show chair, and Letterman went to CBS.
Leno retired earlier this year at NBC and was replaced by Jimmy Fallon.
Letterman's contract runs through 2015 at CBS.
He made the announcement Thursday afternoon during a taping of the show scheduled to air Thursday night. The news broke on Twitter with a tweet from Mike Mills, bassist for R.E.M., a musical guest on the show.
Here's the announcement as it was made Thursday during the taping:
“The man who owns this network, Leslie Moonves, he and I have had a relationship for years and years and years, and we have had this conversation in the past, and we agreed that we would work together on this circumstance and the timing of this circumstance. And I phoned him just before the program, and I said ‘Leslie, it’s been great, you’ve been great, and the network has been great, but I’m retiring,’” said Letterman.
“I just want to reiterate my thanks for the support from the network, all of the people who have worked here, all of the people in the theater, all the people on the staff, everybody at home, thank you very much.
"What this means now, is that Paul and I can be married," he joked referring to bandleader Paul Shaffer. "We don’t have the timetable for this precisely down – I think it will be at least a year or so, but sometime in the not too distant future, 2015 for the love of God, in fact, Paul and I will be wrapping things up.”
The audience responded with a standing ovation.
Moonves and CBS issued the following statement confirming Letterman's retirement:
“When Dave decided on a one-year extension for his most recent contract, we knew this day was getting closer, but that doesn’t make the moment any less poignant for us. For 21 years, David Letterman has graced our Network’s air in late night with wit, gravitas and brilliance unique in the history of our medium. During that time, Dave has given television audiences thousands of hours of comedic entertainment, the sharpest interviews in late night, and brilliant moments of candor and perspective around national events. He’s also managed to keep many celebrities, politicians and executives on their toes – including me.
"There is only one David Letterman. His greatness will always be remembered here, and he will certainly sit among the pantheon of this business. On a personal note, it’s been a privilege to get to know Dave and to enjoy a terrific relationship. It’s going to be tough to say goodbye. Fortunately, we won’t have to do that for another year or so. Until then, we look forward to celebrating Dave’s remarkable show and incredible talents.”Copyright © 2015, The Baltimore Sun