Gordon was born in Murfreesboro, Tennessee, where he has lived all of his life. He served in the United States Army Reserve in 1971 and 1972. He graduated from Middle Tennessee State University in 1971, earning a law degree from the University of Tennessee in 1973. He then entered private practice in Murfreesboro.
Active in Democratic politics early on, he was briefly executive director of the Tennessee Democratic Party in 1979 and state party chairman from 1981 to 1983. When 6th District Congressman Al Gore announced in 1983 that he would run for the United States Senate in 1984, Gordon stepped down as state party chairman to run for the seat. He initially faced a hard-fought race against the brother of the publisher of Nashville's former conservative newspaper, the Nashville Banner. However, he won handily in November 1984, riding Gore's coattails in the midst of Ronald Reagan's landslide victory in that year's presidential election. Gordon is regarded as a moderate. He has favored the repeal of the inheritance tax and the "marriage tax penalty".
Gordon was reelected by huge margins until 1994, when his Republican opponent was attorney Steve Gill, a former basketball player at the University of Tennessee who is now a radio talk show host. Gordon only won by one percentage point, but managed to defeat Gill more handily in 1996. Gordon was re-elected in 1998 and 2000 by margins similar to those he scored in the 1980s and early 1990s. He faced no significant opposition in 2002, 2004 and 2006 and is unopposed in 2008. This is largely because the 2002 reapportionment by the Democratic-controlled Tennessee General Assembly removed Williamson County, a wealthy and heavily Republican suburban area south of Nashville, from the Sixth District and added it to the already heavily Republican Seventh District. Ironically, that district is now represented by Marsha Blackburn, who in 1992 became Gordon's first well-financed Republican opponent since his initial 1984 race.
Gordon has posted one diary at Daily Kos.
In March 2007 it was reported that Rep. Gordon, chairman of the U.S.House science committee, said that NASA is headed for "a train wreck" if the space agency isn't better funded to finish building the international space station and develop the next-generation spacecraft.