The term will-o'-the-wisp comes from wisp, a bundle of sticks or paper sometimes used as a torch, and will-o' ("Will of").
The folklore phenomenon will-o'-the-wisp (Will of the wisp) is sometimes referred to as Jack o' lantern (Jack of the lantern), and indeed the two terms were originally synonymous. In fact the names "Jacky Lantern" and "Jack the Lantern" are still present in the oral tradition of Newfoundland. These lights are also sometimes referred to as "corpse candles" or "hobby lanterns", two monikers found in the Denham Tracts. They are often called spooklights or ghost lights by folklorists and paranormal enthusiasts in the United States. Sometimes the phenomenon is classified by the observer as a ghost, fairy, or elemental, and a different name is used. Briggs' "A Dictionary of Fairies" provides an extensive list of other names for the same phenomenon though the place they are observed (graveyard, bogs etc.) influences the naming considerably.
One version, from Shropshire, recounted by K. M. Briggs in her book A Dictionary of Fairies, refers to Will the Smith. Will is a wicked blacksmith who is given a second chance by Saint Peter at the gates to Heaven, but leads such a bad life that he ends up being doomed to wander the Earth. The Devil provides him with a single burning coal with which to warm himself, which he then used to lure foolish travellers into the marshes (compare Wayland Smith).
An Irish version of the tale has a ne'er-do-well named Drunk Jack or Stingy Jack who makes a deal with the Devil, offering up his soul in exchange for payment of his pub tab. When the Devil comes to collect his due, Jack tricks him by making him climb a tree and then carving a cross underneath, preventing him from climbing down. In exchange for removing the cross, the Devil forgives Jack's debt. However, because no one as bad as Jack would ever be allowed into Heaven, Jack is forced upon his death to travel to Hell and ask for a place there. The Devil denies him entrance in revenge, but, as a boon, grants Jack an ember from the fires of Hell to light his way through the twilight world to which lost souls are forever condemned. Jack places it in a carved turnip to serve as a lantern. Another version of the tale, "Willy the Whisp", is related in Irish Folktales by the late Henry Glassie.
Sometimes the lights are believed to be the spirits of unbaptized or stillborn children, flitting between heaven and hell (compare Wilis). Modern occultist elaborations bracket them with the salamander, a type of spirit wholly independent from humans (unlike ghosts, which are presumed to have been humans at some point in the past).
Danes, Finns, Swedes, Estonians and Latvians amongst some other groups believed that a will-o'-the-wisp marked the location of a treasure deep in ground or water, which could be taken only when the fire was there. Sometimes magical tricks were required as well, to uncover the treasure. In Finland and other northern countries it was believed that midsummer was the best time to search for will-o'-the-wisps and treasures below them. It was believed that when someone hid treasure in the ground, (s)he made the treasure available only at the midsummer, and set will-o'-the-wisp to mark the exact place and time so that (s)he could come to take the treasure back. Finns also believed that the creature guarding the treasure used fire to clean precious metals bright again.
The will-o'-the-wisp can be found in numerous folk tales around the United Kingdom, and is often a malicious character in the stories. Wirt Sikes in his book British Goblins mentions a Welsh tale about a will-o'-the-wisp (Pwca). A peasant travelling home at dusk spots a bright light travelling along ahead of him. Looking closer, he sees that the light is a lantern held by a "dusky little figure", which he follows for several miles. All of a sudden he finds himself standing on the edge of a vast chasm with a roaring torrent of water rushing below him. At that precise moment the lantern-carrier leaps across the gap, lifts the light high over its head, lets out a malicious laugh and blows out the light, leaving the poor peasant a long way from home, standing in pitch darkness at the edge of a precipice. This is a fairly common cautionary tale concerning the phenomenon; however, the Ignis Fatuus was not always considered dangerous. There are some tales told about the will-o'-the-wisp being guardians of treasure, much like the Irish leprechaun leading those brave enough to follow them to sure riches. Other stories tell of travellers getting lost in the woodland and coming upon a will-o'-the-wisp, and depending on how they treated the will-o'-the-wisp, the spirit would either get them lost further in the woods or guide them out.
In Guernsey, the light is known as the faeu boulanger (rolling fire), and is believed to be a lost soul. On being confronted with the spectre, tradition prescribes two remedies. The first is to turn one's cap or coat inside out. This has the effect of stopping the faeu boulanger in its tracks. The other solution is to stick a knife into the ground, blade up. The faeu, in an attempt to kill itself, will attack the blade.
One Asian theologist ponders the relation of will-o'-the-wisp to that of the foxfire produced by kitsune, an interesting way of combining mythology of the West with that of the East.
In addition to Kitsunebi (aka Foxfire) described above, additional similar phenomena are described in Japanese folklore, including Hitodama (literally "Human ball" as in ball of energy), Hi no Tama (Ball of Flame), Aburagae, Koemonbi, Ushionibi, etc. All these phenomena are described as balls of flame or light, at times associated with graveyards, but occurring across Japan as a whole in a wide variety of situations and locations. These phenomena are described in Shigeru Mizuki's 1985 book Graphic World of Japanese Phantoms (妖怪伝 in Japanese)
More recently Professors Derr and Persinger put forward a theory that earth lights may be generated piezoelectrically under a tectonic strain. This theory suggests that the strains which move faults also causes heat in the rocks, vaporising the water in them. Rocks and soils containing piezoelectric elements such as quartz (or silicon) may also produce electricity, which is channeled up through soils via a column of vaporised water until it reaches the surface — somehow displaying itself in the form of earth lights. If correct, this explains why such lights can behave in an electrical and erratic – or even apparently intelligent – manner.
Others explanations associate will-o'-the-wisps with bioluminescence (e.g. honey fungus). Barn owls are also known to have luminescent plumage with a very high albedo that can reflect enough light from other sources (for example, the moon) to generate the appearance of a will-o'-the-wisp. Hence the possibility of them floating around, reacting to other lights, etc.
In literature, Will o' the wisp sometimes has a metaphorical meaning, describing a hope or goal that leads one on but is impossible to reach, or something one finds sinister and confounding. Samuel Taylor Coleridge's poem The Rime of the Ancient Mariner describes the Will o' the wisp, and two Will-o-the-wisps appear in Johann Wolfgang von Goethe's fairy tale of The Green Snake and the Beautiful Lily (1795). John Wilmot, 2nd Earl of Rochester satirically refers to human reason as "an ignis fatuus in the mind" in his 1670 poem "A Satyre Against Reason and Mankind." The Will o' the wisp also makes an appearance in the first chapter of Bram Stoker's Dracula, wherein the Count, masquerading as his own coach driver, takes Jonathan Harker to his castle in the night. "Ignis Fatuus" can also be seen in Charlotte Brontë's Jane Eyre in chapter 28 when she is lost in the moor.
Hinkypunk, the name for a Will o' the wisp in South West England has achieved fame as a magical beast in JK Rowling's Harry Potter series.. Here, the Hinkypunk is a translucent creature that bears a lantern in one hand and hops on a single foot. Humans who follow this light may find themselves immersed in marsh. In J.R.R Tolkien's work The Lord of the Rings, Will o' the wisps are present as "candles lit by the dead" in the Dead Marshes outside of Mordor.
In Michael Ende's fantasy novel, The Neverending Story, a Will o' the Wisp is one of the many species of creatures living in the land of Fantastica. The Will o' the Wisp described in the novel is a messenger sent to visit the Childlike Empress. He is described as being a small man inside an orb of light, who traveled by bouncing irregularly through the woods.
Will-o'-the-Wisps are mentioned as magical creatures in Cornelia Funke's novel Dragon Rider. No comprehensive description is given, though they are implied to be tiny, luminous creatures who are equally likely to be found in mountains and in deserts.
In Blindness (novel), by José Saramago, on pages 313-314, the doctor's wife notices will-o'-the-wisps around a supermarket basement door: "A few minutes later she said, They are dead, Did you see anything, did you open the door, asked her husband, No, I only saw will-o'-the-wisps around the doors, they clung there and danced around and did not let go, I think it must have been the phosphorised hydgrogen as a result of the decomposition of the bodies, What could have happened, They must have found the basement, rushed down the stairs looking for food".