It was the second of three UK singles from the band's last studio album Strangeways, Here We Come, and was released after the band had announced their split. The record company had originally intended to release "Stop Me If You Think You've Heard This One Before" as a single in the UK but felt this would be inappropriate following the Hungerford massacre (the lyrics contain a reference to "mass murder").
"Stop Me..." was still released as a single in other countries, but its promotional video - which featured Morrissey plus a large number of Morrissey lookalikes - was used in the UK to promote "I Started Something I Couldn't Finish".
The cassette version featured a cover of the James song "What's The World?"
The Wool Hall was the recording studio in Bath where the Smiths had recorded latest album "Strangeways Here We Come", but it also was where Morrissey, at the time of this single's release, was recording his debut solo album "Viva Hate". Sheridan Whiteside was one of Morrissey pseudonyms. The b-side etching is a reversal of "You are sleeping, you do not want to believe", a sample heard at the end of the Smiths song "Rubber Ring".