Johnson played drums from age 13 and picked up saxophone shortly thereafter; he continued to play drums well enough to play as a session drummer professionally. He attended Wilberforce University, where he played in the bands of Ernie Fields and Tiny Bradshaw. He joined up with Andy Kirk's Twelve Clouds of Joy from 1942 to 1947, then worked briefly with Count Basie before founding his own band, the Peppermint Sticks, in Detroit. Late in the 1950s he did work with Bill Doggett.
Johnson credited himself as Candy Johnson on many of his early recordings, but in the early 1960s, a dancer named Candy Johnson released a few hit records, and he returned to Floyd Johnson, in addition to reducing his activity in music. In the 1970s he returned to full-time playing, touring with Milt Buckner in France in 1971, then played in Paul Gonsalves's place in the Duke Ellington Orchestra when Gonsalves fell ill. He also worked in the New McKinney's Cotton Pickers in the 1970s.