Pitt was born in Decatur, Alabama. He attended the University of Alabama and graduated with a Bachelor of Arts degree in 1967. He was active in campaign for Robert F. Kennedy in the 1968 presidential election.
After graduating from college Pitt entered the officer corps of the United States Army, serving from 1969 to 1972 in Vietnam, where he attained the rank of captain and earned a Bronze Star. Later he served in the Office of the Chief of Staff for Intelligence in Washington, D.C.
Following his Army service, Pitt attended Boston College Law School and received his Juris Doctor in 1977, after which he returned to his home state and began his public career. Pitt served as Alabama's Assistant Attorney General from 1981 to 1991, and Chief Deputy Attorney General from 1991 to 1994. He has served as a legal advisor to several Alabama state government officials. From 1994 to 1998 he served as United States Attorney for the Middle District of Alabama.
Pitt currently practices law in Birmingham, Alabama. He was one of the attorneys of record for former Alabama Governor Don Siegelman during Siegelman's federal trial in the U.S. District Court for the Middle District of Alabama on charges of bribery, mail fraud, and obstruction of justice. He continues to serve campaigns for Democratic candidates, including his Boston College classmate John Kerry in 2004. He serves on the Democratic National Committee and was elected in 2003 to chair the Alabama Democratic Party.
Pitt is married to Jane LaCroy Brannon of Montgomery, Alabama. He has one son, William Rivers Pitt, by his previous wife, Jane H. Pitt. He is related to the 18th-century British Prime Minister William Pitt.