Music kept drawing McGriff's attention away from the police force. His childhood friend, organist Jimmy Smith, had begun earning a substantial reputation in jazz for his Blue Note records (the two played together once in 1967) and McGriff became entranced by the organ sound while Richard "Groove" Holmes played at his sister's wedding. Holmes went on to became McGriff's teacher, friend and they recorded together on two occasions in 1973 for two Groove Merchant records.
McGriff bought his first Hammond B-3 organ in 1956, spent six months learning the instrument, then studied at New York's Juilliard School of Music. He also studied privately with Milt Buckner, Jimmy Smith, and Sonny Gatewood. He was influenced by the energy and dynamics of organist Milt Buckner and the diplomatic aplomb of Count Basie, and by local pianists such as Sonny Gatewood, Howard Whaley and Austin Mitchell.
In 1961, McGriff's trio was offered the chance to record an instrumental version of Ray Charles' hit "I've Got a Woman" by Joe Lederman's Jell Records, a small independent label. When the record received substantial local airplay, Juggy Murray's Sue label picked it up and recorded a full album of McGriff's trio, released in 1962. The album also turned out another huge hit in McGriff's "All About My Girl", establishing McGriff's credentials as a fiery blues-based organist, well-versed in gospel soul and fatback groove.
McGriff recorded a series of popular albums for the Sue label between 1962 and 1965, ending with what still stands as one of his finest examples of blues-based jazz, Blues for Mister Jimmy. When producer Sonny Lester started his Solid State record label in 1966, he recruited McGriff to be his star attraction. Lester framed McGriff in many different groups, performing a wide variety of styles and giving the organist nearly unlimited opportunities to record. McGriff was heard everywhere from an all-star tribute to Count Basie (The Big Band), a series of "Organ and Blues Band" records (such as A Thing to Come By (1969), pop hits ("Cherry", "The Way You Look Tonight") and funk classics (Electric Funk and singles such as "The Worm" and "Step 1").
During this time, McGriff performed at clubs and concert halls worldwide. He settled in Newark, New Jersey, and eventually opened his own supper club, the Golden Slipper - where he recorded Black Pearl and a live album with Junior Parker in 1971. Beginning in 1969, he also performed regularly with Buddy Rich's band, though the two were only recorded once together in 1974 on The Last Blues Album Volume 1).
By 1980, McGriff broke away from Sonny Lester and began working actively with producer (and former funk lord) Bob Porter (and recording engineer Rudy Van Gelder). McGriff began a long relationship with Fantasy's Milestone label, collaborating with Rusty Bryant, Al Grey, Red Holloway, David "Fathead" Newman, Frank Wess and Eric Alexander.
In 1986, McGriff started a popular partnership with alto sax player Hank Crawford. Their partnership has yielded a few interesting moments (1987's Soul Survivors and 1997's Road Tested). But it was only during their brief period at Telarc in the mid 1990s that McGriff's name headlined the popular club and cruise ship attraction.
However, it should be noted that Jimmy was one of the first B3 players to add MIDI to the upper keyboard his personal B3 to add and extend "his sound" beyond just the drawbar sound of the B3. He incorporated synthesizers in his live performances as he liked Vibes, Piano, String, Brass and other sounds that could only be created by a synthesizer as the classic B3 can not. Jimmy purchased the XB-3 as he had more control over the MIDI functions and the XB-3 weighs about half of the classic B3 which made it easier to move.
Jimmy, as well as Groove Holmes, spent a great deal of time experimenting and modifying their B3's and Leslie Speakers over the years. Some of these modifications made their way into products manufactured by both Hammond and Leslie, which they didn't always get credit for.
Along with the soul-jazz sound, McGriff experienced renewed popularity in the mid-1990s, forming The Dream Team group, which featured David "Fathead" Newman (a longtime sax player with Ray Charles) and drummer Bernard Purdie, and recording the Straight Up (1998), McGriff's House Party (2000), Feelin' It (2001), and McGriff Avenue (2002) albums.
Jimmy McGriff died on Saturday May 24, 2008 of multiple sclerosis.