Poets also frequently appear as characters in his novels, always on the side of good: Sappho in Wolfwinter (1972); Robert Herrick in Will-o-the-Wisp (1977, serialized 1974); a fictionalized Charles Sorley in The Goat Without Horns (1971); and Thomas Chatterton in The Not-World (1975).
Swann began writing fiction in 1958 with "Winged Victory", a science fiction story based on the famous headless statue known as the Winged Victory of Samothrace. In Swann's story the statue's head is discovered and found to have been modeled upon an alien visitor whom the sculptor took for a goddess.
Extraterrestrials also feature in "The Painter", in which the painter Hieronymous Bosch is abducted by hideous aliens and forced to paint them, thereby providing the inspiration for the grotesque images in his painting The Garden of Earthly Delights. This and many other early stories appeared in the British magazine Science Fantasy. Some stories also appeared in The Magazine of Fantasy & Science Fiction (F&SF).
Most of Swann's fiction was outright fantasy. The early story "The Dryad-tree" is set in contemporary Florida and features a woman's reaction to the knowledge that her new husband's garden contains a tree possessed by a jealous dryad.
The bulk of Swann's fantasy fits into a rough chronology that begins in ancient Egypt around 2500 BC and chronicles the steady decline of magic and mythological races such as dryads, centaurs, satyrs, selkies and minotaurs. The coming of more "advanced" civilisations constantly threatens to destroy their pre-industrial world, and they must continually seek refuge wherever they can. They see the advent of Christianity as a major tragedy; the Christians regard magic and mythological beings as evil and seek to destroy the surviving creatures, although some manage to survive and preserve some of their old ways through medieval times down to the late 19th Century and perhaps the 20th.
An undercurrent of sexuality runs through all of these stories. Many of Swann's characters are sexually adventurous and regard sexual repression as spiritually damaging. Casual and sometimes permanent nudity is common. Homosexual relationships between both male and female characters are often hinted at, although seldom made explicit.
The most openly homosexual relationship in Swann's stories is also the most controversial. His novel How Are the Mighty Fallen (1974) depicts the Biblical Jonathan and David as lovers, and furthermore suggests that Jonathan was himself a member of an ancient winged half-human race. The book appeared from Swann's regular publisher DAW Books, but only after DAW's founder and chief executive Donald A. Wollheim fought to prevent distributor New American Library from banning it. However, Swann was reportedly unhappy with George Barr's cover artwork, which showed two of the characters being chased by a cyclops, because he felt it misrepresented the style of the novel.