Born in Baltimore and raised on Adelle Terrace, he was the son of James Phillip Cragg Sr. and Bertha Lenore Cragg. He got the name Tim after the Charles Dickens character, Tiny Tim. As a boy he befriended a schoolmate, the future mayor and governor, William Donald Schaefer. The pair attended Gwynns Falls Junior High School together and Baltimore City College, where they were 1939 graduates. They remained friends.