Diane Derby, Jeffords' former press secretary and a spokeswoman for his family, told the Los Angeles Times that he died at his residence in Washington. The cause of death was unclear but Derby said he had been in declining health.
In a statement, Sen. Patrick J. Leahy (D-Vt.) called Jeffords "a Vermonter through and through, drawn to political life to make a difference for our state and nation."
Jeffords announced in 2005 that he would not seek a fourth term, citing his and his wife's health problems. His wife, Liz, died in 2007.
Jeffords served more than 30 years in Washington. He won election to the House in 1974 as a Republican.
He was the only Republican in the House to vote against President Reagan's tax cuts in 1981. Later, upset with President George W. Bush's opposition to the Individuals With Disabilities Education Act, Jeffords declared in May 2001 that he would leave the Republicans and caucus with the Democrats.
Jeffords is survived by his daughter, Laura, and son, Leonard, who reside in the Washington, D.C. area.
The Associated Press contributed to this report.