Prior to recording for friends under the NMH moniker, Mangum recorded at least one cassette under the name Milk; the most well-known of these tapes was called Pygmie Barn in E Minor. These are the earliest known recordings that Mangum initiated and executed himself. There were only an estimated dozen or so copies ever made of this tape, none of which has been made public. It was revealed circa 2006 on the Elephant 6 online forums that another tape under said name, entitled Beauty, was released, along with several recordings under the name Rubby Bulbs (also a song dating from that era, perhaps the only Milk song to be played by Neutral Milk Hotel).
Neutral Milk Hotel began simply as a recording project for Mangum, in the early 1990s. He produced several demo cassettes, among them 1991's Invent Yourself a Shortcake, 1992's Beauty (not to be confused with the earlier cassette), and 1993's Hype City Soundtrack, along with two more from this period, seemingly unnamed. Although easily found on the Internet, these demos capture the project at a very embryonic stage: songs are played in between various sound collages and tape experiments, one of which consists only of a six minute conversation between Mangum and Hart. Another track, "Digestion Machine", features a varied collage of voices answering the question asked by Mangum, "What does the digestion machine mean to you?" The more accessible "Synthetic Flying Machine", also titled "Up and Over", later became "The King of Carrot Flowers Pt. 3" on In the Aeroplane Over the Sea.
During this period, Mangum was wandering the country, staying in the closets of friends, and in a state of perpetual unemployment. It was in these circumstances that the band's first formal releases took shape. Strictly speaking, however, the 'band' usually consisted of Mangum and whoever else was present at the time. This is obvious on Neutral Milk Hotel's first release, a 7" record entitled Everything Is, recorded when Mangum was spending time in Seattle, Washington, released on Cher Doll Records in 1994.
After the release of Avery Island, Neutral Milk Hotel became a fully-fledged band, as Julian Koster, Scott Spillane and Jeremy Barnes joined Mangum, the band now being based in New York. Soon after this, they moved to Athens, GA, where many of Mangum's friends had begun to settle, and the Elephant 6 recording company began to fully take shape. After this, the band went back to Denver, to record a proper follow up to On Avery Island.
Before Neutral Milk Hotel began their indefinite hiatus, Mangum played live at a house show on December 5, 1998 in Athens on Chris Bilheimer's birthday. The bill was shared with Elf Power, and the audience was made up almost completely of friends and bandmates. Playing solo and acoustic, Mangum opened the set with what was to be the only post-Aeroplane composition to be performed in public, "Little Birds". He explained to the audience that the song is about a boy whose body becomes filled up with miniature birds that come out of the bathtub faucet and protect him from his murderous father (who has apparently already killed the main character's younger brother).
Did you know the burning hell it took your baby brother?
Did you see how far he fell and how he made us suffer?
Another boy in town at night he took him for his lover
And deep in sin they held each other
So I took a hammer, nearly beat his little brains in
Knowing God in Heaven would have never could forgive him
Mangum finished the rest of the evening with most of the Aeroplane songs, encouraging the audience to sing along in substitute for the horn parts. For the last several songs, Scott Spillane took his trumpet in to an adjacent room and played his parts through the wall. A recording of the show is known to exist, but it has never been officially released, most likely due to the poor quality of the recording.
Mangum played another show on December 31, 1998, at which he performed the songs "Oh Sister", "Engine", and "In the Aeroplane Over the Sea", joined by Koster and Spillane.
On February 4, 2001, Mangum played his very last show in Auckland, New Zealand at The King's Arms at the request of Chris Knox. The show was referred to as The World of Wild Beards Incorporated and consisted of acts from Mangum, Laura Carter, and Chris Knox. Mangum played 13 songs, about half of them from Aeroplane as well as covering Phil Spector's "I Love How You Love Me" and John Lennon's "Mother" with Chris Knox joining him on the latter of the two. A recording of the show does exist, but has never been officially released.
There has, however, been some activity since Aeroplane. First, Orange Twin re-released Everything Is, complete with extra bonus tracks, as well as Live at Jittery Joe's, a live recording by filmmaker Lance Bangs of a show Mangum played in a coffee house in Athens, Georgia in 1997. The Major Organ and the Adding Machine album, a rather secretive project released by Orange Twin in 2001 and consisting mainly of experimental musical collage, features Mangum, along with Julian Koster, of Montreal's Kevin Barnes, Elf Power's Andrew Reiger and the Olivia Tremor Control and Circulatory System's Will Cullen Hart and Eric Harris. There was also the release of Orange Twin Field Works: Volume I, Mangum's field recording of a Bulgarian folk festival, also on Orange Twin in 2001. Additionally, he plays drums on the first Circulatory System album.
Since the breakup of Neutral Milk Hotel, Jeff Mangum has only played live proper again once, at the Kings Arms pub in Auckland, New Zealand, at the request of the Tall Dwarfs's Chris Knox. The band was billed as the World of Wild Beards Incorporated, although when he explains this bizarre name during the course of the show, Mangum gives a small speech about an obscure, seemingly fictional company existing in the early 1900s named "Walking Wall of Beards Incorporated". He has also made appearances with Elf Power and The Instruments, and came onstage at the first of the Olivia Tremor Control's New York shows on their 2005 reunion tours, to sing briefly. Further releases from Neutral Milk Hotel, however, are indicated to be unlikely.
In 2005, In the Aeroplane Over the Sea saw its re-release by Domino Records in the UK, in a sleeve featuring praise from, among others, Franz Ferdinand and Arcade Fire, both very much influenced by the band. Mangum has also done a few small art projects - following a charity auction on eBay of an acoustic guitar decorated by Mangum himself, more guitars hand-decorated by Mangum were sold via Orange Twin Records for $900 apiece. Also, a number of Mangum's original drawings, this time costing only $10 each, were sold via Neutral Milk Hotel's website.
In 2006, a message was posted on the Elephant 6 message board, purportedly from Jeff Mangum. The message suggested that Mangum would again record music and tour, though it was unclear if it would be under the Neutral Milk Hotel moniker or another one. It soon caused an enormous stir within the indie community, and news of the message appeared in Rolling Stone, Pitchfork, and Billboard. However, it was eventually discredited by Robert Schneider, frontman for The Apples in Stereo and close friend of Mangum's. The perpetrator of the hoax (which convinced many sources close to Mangum) is still unknown.