Mimms grew up in Philadelphia, where he sang in gospel music groups such as the Evening Stars, the Harmonizing Four, and the group with which he would record his first record in 1953, the Norfolk Four. He returned to Philadelphia after serving in the military and formed doo-wop group The Gainors in 1958.
In 1961 Mimms and Sam Bell from The Gainors left to form a new group, Garnet Mimms and the Enchanters with Zola Pearnell and Charles Boyer. The group moved to New York and began to work with the songwriter and record producer, Bert Berns. Berns signed them to the United Artists label and wrote the hit "Cry Baby" for them with songwriting partner Jerry Ragovoy. The song topped the R&B chart and went to #4 on the Billboard Hot 100 chart in 1963, and paved the way for soul hits by Aretha Franklin and Otis Redding later in the decade.
Mimms went solo and performed another Berns and Ragovoy hit "Take Good Care Of You" in 1966. Mimms worked with Jimi Hendrix in the UK the following year. He did some recording on the MGM and Verve Records labels.
In the 1970s he released a few funk songs as Garnet Mimms and the Truckin' Co. He had his only hit in the United Kingdom at this time, when "What It Is" reached number 44 for one week on the UK Singles Chart in June 1977.
Mimms was given a Pioneer Award in 1999 by the Rhythm and Blues Foundation.
In the 1980s, Garnet found his calling ministering to lost souls in prison, but in 2007 returned to recording, and in 2008 released a new gospel album Is Anybody Out There? on the Evidence label, produced and (primarily) written by Jon Tiven.