Wesley was born in Columbus, Georgia, the son of a high school teacher and big band leader. During the 1960s and 1970s he was a pivotal member of James Brown's bands, playing on many hit recordings including "Say it Loud - I'm Black and I'm Proud", "Mother Popcorn" and co-writing tunes such as "Hot Pants". His slippery riffs and pungent, precise solos, complementing those of saxophonist Maceo Parker, gave Brown's R&B, soul, and funk tunes their instrumental punch. In the 1970s he also served as band leader and musical director of Brown's band the J.B.'s and did much of the composing and arranging for the group. He left Brown's band in 1975 and spent several years playing with George Clinton's various Parliament/Funkadelic projects, even recording a couple of albums as the leader of a spin-off group, The Horny Horns.
Wesley became a force in jazz in 1978 when he joined the Count Basie Orchestra. He released his first jazz album as a leader, To Someone in 1988. It was followed by New Friends in 1990, Comme Ci Comme Ca in 1991, the live album Swing and Be Funky, and Amalgamation in 1994.
In the early nineties Wesley toured with his colleagues from the James Brown band, Pee Wee Ellis and Maceo Parker, as the JB Horns. With the departure of Ellis the band became The Maceo Parker Band. Wesley was featured trombonist with Parker until 1996 when he formed his own band, The Fred Wesley Group.
Wesley's 35-year career includes playing with and arranging for a wide variety of other artist such as Ray Charles, Lionel Hampton, Randy Crawford, Vanessa Williams, The SOS Band, Cameo, Van Morrison, Socalled and rappers De La Soul, to name a few. Many other artists have sampled his work.
In 2002 Wesley wrote Hit Me, Fred: Recollections of a Sideman (ISBN 0-8223-2909-3), an autobiography about his life as a sideman. Also in 2002 he recorded an album entitled Cuda Wuda Shuda with a group of jazz musicians calling themselves the Fred Wesley Band.
Wesley served as an adjunct professor in the Jazz Studies department of the School of Music at the University of North Carolina at Greensboro from 2004 to 2006.