The song is a medley of fragments of blues songs and lyrics, including "Shake 'Em on Down" by Bukka White and "Lone Wolf Blues" by Oscar Woods. Therefore, the song is both a tribute to contemporary folk singer Roy Harper and the influential American blues singers who recorded in the 1960s.
Roy Harper is a folk singer from England whom Jimmy Page met at the Bath Festival in 1970. He became close friends of the band, who invited him to perform as the opening act on some later Led Zeppelin concert tours. In 1971, Page played on Harper's album Stormcock, appearing in the credits under the pseudonym "S. Flavius Mercurius." Harper would go on to perform the lead vocals on Pink Floyd's "Have a Cigar", from 1975's Wish You Were Here. In 1985, Page recorded an album with Harper called Whatever Happened to Jugula?. According to Page, during recording sessions for Led Zeppelin III, the band "did a whole set of country blues and traditional blues numbers that Robert [Plant] suggested. But ["Hats Off to (Roy) Harper"] was the only one we put on the record."
"Hats Off to (Roy) Harper" begins with a strange audio snippet from the sessions which is quickly introduced and then faded out again, featuring Plant's voice and Page's slide guitar in tandem.
An alternate studio outtake of the track in the same style and similar instrumentation is available on some Led Zeppelin bootleg recordings. Likely from the same recording session as the official release, it features lyrics from the songs "Feel So Bad," (recorded by Otis Rush and Elvis Presley), Robert Johnson's "Traveling Riverside Blues" & "32-20 Blues", Sleepy John Estes' "Diving Duck Blues", Bukka White's "Fixin' To Die", and Elvis' "That Alright Mama." These songs were frequently performed in medley by the band at Led Zeppelin concerts during "How Many More Times" and, later, "Whole Lotta Love." Thus, this outtake perhaps gives insight into the inspiration for the track, a desire to lay down an acoustic, studio take of a staple of their live performances. However, Led Zeppelin never performed "Hats Off to (Roy) Harper" live in concert.