Sag Harbor is about three fifths in Southampton and two fifths in East Hampton. The dividing line is Division Street which becomes Town Line Road just north of the village. Most of the defining landmarks of the village -- including its Main Street, the Whalers Church, Jermain Library, Whaling Museum, the Old Burying Ground, Oakland Cemetery, Mashashimuet Park, and Otter Pond are all in Southampton. However, almost all the Bay Street marina complex at the foot of Main Street is in East Hampton as are the village's high school, the Sag Harbor State Golf Course, and the freed slave community of Eastville.
Sag Harbor was settled sometime between 1707 and 1730 (the first Bill of lading using the name Sag Harbor was recorded in 1730). Most accounts say it was named for neighboring Sagaponack, New York which at the time was called "Sagg." Sag Harbor's first spellings used the double "G.
The port supplanted the East Hampton community of Northwest which is about five miles east of Sag Harbor. International ships and the whaling industry had started in Northwest but its port was too shallow. The most valuable whale product was whale oil which was used in lamps; thus it could be said that Sag Harbor was a major oil port.
By 1789 Sag Harbor had "had more tons of square-rigged vessels engaged in commerce than even New York City. It had become an international port.
During the American Revolutionary War, American raiders under Return Jonathan Meigs attacked a British garrison on May 23, 1777, on a hill at what today is the Old Burying Ground next to the Whaler's Church, killing six and capturing 90 British soldiers in what was called Meigs Raid.
During the War of 1812, the British attacked the town on July 11, 1813 but were driven back.
The whaling industry in Sag Harbor peaked in the 1840s. Sag Harbor is mentioned in Chapters 12, 13, 57 and 83 of Moby-Dick including this passage:
- Arrived at last in old Sag Harbor; and seeing what the sailors did there; and then going on to Nantucket, and seeing how they spent their wages in that place also, poor Queequeg gave it up for lost. Thought he, it's a wicked world in all meridians; I'll die a pagan.
Among the sea captains who died whaling from Sag Harbor was Charles Watson Payne, the great-great-great grandfather of Howard Dean
Relics of this period include the Old Whaler's Church which is a Presbyterian Church that sported a 168-foot high steeple which was claimed to be the tallest structure on Long Island when it opened in 1843. The steeple collapsed during the Great Hurricane of 1938 The Sag Harbor Whaling and Historical Museum now shares the handsome, 1840 Greek Revival building designed by Minard Lafever for Wamponamon Lodge #437 Free & Accepted Masons. The Masonic Lodge is celebrating its 150th anniversary in 2008. The Architecture of both the Whaler's Church & the Masonic Temple are attributed to prominent 19th century American architect Minard LaFever. The broken mast monument in Oakland Cemetery is the most visible of several memorials to those who died at sea.
The whaling business collapsed after 1847 initially with the discovery of other methods to create kerosene with the first being coal oil. The discovery of petroleum in Titusville, Pennsylvania in 1859 sealed the end. Many of the ships based in Sag Harbor were sailed to San Francisco, California where they were simply abandoned during the California Gold Rush. The last whaler -- the Myra -- sailed from Sag Harbor in 1871.
One sailor who continued on other endeavors was Mercator Cooper who sailed out of Sag Harbor on November 9, 1843 on the Manhattan (1843) on a voyage that would make him the first American to visit Tokyo Bay. Aboard the ship was Pyrrhus Concer a former slave who was the first black man the Japanese had seen.
During World War I the E.W. Bliss Company tested torpedoes in the harbor a half mile north of Sag Harbor. As part of the process, Long Wharf in Sag Harbor was reinforced with concrete and rail spurs built along the wharf as the torpedoes were loaded onto ships for testing. The torpedoes were shipped via the Long Island Rail Road along the Sag Harbor to the wharf which was owned by the railroad at the time. Among those observing the tests was Thomas Alva Edison. Most of the today's buildings on the wharf including the Bay Street Theatre were built during this time. The torpedoes which did not have live warheads are occasionally found by divers on the bay floor.
The Sag Harbor-North Haven Bridge is notable as the site of Pop artist Ray Johnson's presumed suicide in 1995 as well as two abortive suicide attempts by monologist Spalding Gray, in September 2002 and October 2003.
World-renound rock star, Billy Joel is a constant figure in Sag Harbor. He owns a boathouse in the harbor area and is there daily, often seen on one of his three boats, or numerous motercycles.
According to the United States Census Bureau, the village has a total area of 2.5 square miles (6.4 km²), of which, 1.7 square miles (4.5 km²) of it is land and 0.8 square miles (1.9 km²) of it (30.36%) is water.
There were 1,120 households out of which 18.8% had children under the age of 18 living with them, 39.6% were married couples living together, 9.4% had a female householder with no husband present, and 47.9% were non-families. 40.7% of all households were made up of individuals and 18.4% had someone living alone who was 65 years of age or older. The average household size was 2.06 and the average family size was 2.81.
In the village the population was spread out with 16.5% under the age of 18, 5.4% from 18 to 24, 25.4% from 25 to 44, 28.6% from 45 to 64, and 24.2% who were 65 years of age or older. The median age was 46 years. For every 100 females there were 91.6 males. For every 100 females age 18 and over, there were 89.8 males.
The median income for a household in the village was $52,275, and the median income for a family was $70,536. Males had a median income of $41,181 versus $34,750 for females. The per capita income for the village was $40,566. About 1.8% of families and 4.2% of the population were below the poverty line, including 1.9% of those under age 18 and 3.8% of those age 65 or over.