Garbage's cover of the song was included on the Sweet Relief tribute album, created to help fund Vic Chesnutt's medical bills. Their version of "Kick My Ass" was performed and recorded at Smart Studios during the January 1996 sessions in which Garbage completed several B-sides and remixes. It is a mellow number, where a quiet and regretful verse gives way to a stunning shimmering chorus. Bass on "Kick My Ass" was played by Daniel Schulman. Shirley Manson said:
"I first came across the songs of Vic Chesnutt during a tour of the United States where we both shared a bill as opening acts for the band Live. During that tour I was to fall madly in love with Vic's music and in particular, the song 'Kick My Ass'. I dreamed of recording a version myself. Having totally converted the rest of my band into rabid fans, we jumped at the chance to record a track for Sweet Relief. A noble cause indeed, to lend a hand is considered a pleasure but to pay tribute to the brilliance of one of the finest songwriters I've had the honor to meet is a thrill beyond measure. To you, sir - with love"
"Kick My Ass" was included on the tribute record Sweet Relief II: Gravity of the Situation - The Songs of Vic Chesnutt, which was released on Columbia Records on August 6 1996. "Kick My Ass" is the only Garbage tribute song to not be subsequently released on a proper Garbage record. It was first performed live by Garbage on 24 February 1996 in Dallas, Texas when Garbage opened on their Spring U.S. tour. The song was a regular encore during the 'Garbage' tour and was also performed frequently during the 'Version 2.0' autumn 1998 tour of North America.
Garbage performed at "An Evening of Sweet Relief" at the Bottom Line, 15 West 4th Street, New York on 14 July 1996. The show consisted of solo acoustic performances by the album's artists. Garbage performed "Supernatural" (sung by Live on the album) and "Kick My Ass", which she explained had been taught to her by Chesnutt personally when they toured together a few years earlier.
At the Sydney show on their "Bleed Like Me" tour, September 23 2005, a miscommunication led the band to take audience requests as "payback". After taking one request for "Supervixen"', one sharp-minded member of the audience requested "Kick My Ass", of which, to the amusement of all present, the band could not remember more than the first few words. A subsequent request for "Trip My Wire" from the same man lead Shirley to admonish the man, as they similarly could not remember the words for that B-side. They ended up performing "Nobody Can Win".