Dr. Carter involved his family at New Shiloh. The congregation's "first lady" was his wife, Weptanomah Bermuda Washington, who ran the women's ministry and wrote nine books, including "The Black Minister's Wife."
His son, Dr. Harold A. Carter Jr., now leads the congregation, which is housed in a church at 2100 N. Monroe St.
Rep. Elijah E. Cummings said in a statement that his "heart is heavy" after learning of the passing of Dr. Carter, whom he called "a true friend and mentor who blessed my life in countless ways."
"Dr. Carter was a man of God. As a pastor, an author and a community leader, he built a strong ministry rooted in Christian faith and embodying the principles of civil rights and justice that were instilled in him in his youth," Cummings said, adding that he often turned to him for counsel when facing difficult decisions.
U.S. Sen. Barbara A. Mikulski said in a statement that she has worshiped at New Shiloh for many years.
"I remember his joy when we talked about his work to empower young people," she said. "I remember his lovely wife of 48 years, Dr. Weptanomah W. Carter, and his talented brother, Dr. Nathan Carter, who treated me like family.
"Reflecting on his life and works as a spiritual giant, I can call him a legend in his own time."
Dr. Carter was honored at a "Living Legend" reception by the Reginald F. Lewis Museum of Maryland African American History and Culture in 2006.
His wife died in 2006. In addition to his son, survivors include a daughter, Weptanomah Carter Davis of Bowie; three sisters, Dorothy Carter Jackson of Selma, Ala., Marian Carter McKinnie of Indianapolis and Blanche Carter Thrash of Atlanta; and four grandchildren.
Funeral arrangements are being planned at the Vaughn Greene Funeral Home.