According to the United States Census Bureau, the town has a total area of 3.3 square miles (8.6 km²), of which, 2.4 square miles (6.3 km²) of it is land and 0.9 square miles (2.3 km²) of it (27.11%) is water.
There were 797 households out of which 29.1% had children under the age of 18 living with them, 50.9% were married couples living together, 7.0% had a female householder with no husband present, and 39.3% were non-families. 29.9% of all households were made up of individuals and 7.0% had someone living alone who was 65 years of age or older. The average household size was 2.40 and the average family size was 3.01.
In the town the population was spread out with 24.0% under the age of 18, 5.0% from 18 to 24, 29.0% from 25 to 44, 31.0% from 45 to 64, and 10.9% who were 65 years of age or older. The median age was 41 years. For every 100 females there were 100.7 males. For every 100 females age 18 and over, there were 97.7 males.
The median income for a household in the town was $72,955, and the median income for a family was $96,455. Males had a median income of $58,571 versus $41,029 for females. The per capita income for the town was $49,427. About 1.4% of families and 4.2% of the population were below the poverty line, including 2.2% of those under age 18 and 0.9% of those age 65 or over.
Sullivan's Island was the primary disembarkation port and entrance to the British North American colonies of over 40% (4-8 Million Persons) of the Slave traded Blacks to the British Colonies using the Middle Passage. It was the largest slave port in North America. Sullivans Island served as the primary quarantine quarters and slave market for the American Colonies that would later become the United States. It is estimated that nearly half of all African Americans had ancestors that passed through Sullivans Island. “There is no suitable memorial, or plaque, or wreath or wall, or park or skyscraper lobby,” Toni Morrison said in a 1989 magazine interview “There’s no 300-foot tower, there’s no small bench by the road.” The first small bench by the road was dedicated by the Toni Morrison Society on Sullivan's Island, July 26, 2008.
On 28 June, 1776, a makeshift log fort was held by colonial forces against a sustained siege and bombardment by British forces under Lord Cornwallis attempting to enter the harbor to besiege and conquer the City of Charleston. The palmetto logs used in the construction proved to be remarkably spongy and absorbed the cannon balls. The Battle of Fort Moultrie was commemorated by the addition of a white palmetto tree to the blue and white crescent moon flag of South Carolina. The victory is still celebrated every June 28, known as Carolina day.
The history of the island has been dominated by the extensive coastal defenseworks known as Fort Moultrie, which served as the base of command for the defenses of the City of Charleston until it was closed in the late 1940s.
Edgar Allan Poe was stationed at Fort Moultrie from 1827 to 1828. The island was the setting of his short story "The Gold-Bug." The town library, situated in a refurbished military battery, is named after the poet. Several streets on the island bear the names of his works as well, including "Raven" and "Gold Bug" Drives.
Other literary connections to the island include the novel Sullivan's Island by Dorothea Benton Frank and inclusions in the novel Beach Music and the semi-autobiographical memoir The Boo, both by novelist Pat Conroy.
E. Lee Spence, a pioneer underwater archaeologist and prolific author of books and articles about shipwrecks and sunken treasure, was a long time resident of Sullivan's Island and discovered many shipwrecks along its shores in the 1960s and 1970s. Those discoveries included the Civil War blockade runners Flora, Beatrice, Stono, Flamingo, Prince Albert, Celt (aka Colt) and the Hunley which was first submarine in history to sink an enemy ship. The Hunley was just over 3.5 miles from shore, while the other wrecks were in the shallow waters along the rocks by Fort Moultrie.
For most of its history, the town on the island was known as Moultrieville, on the south-west half of the island. Another community on the north-east portion, Atlanticville, was established and later merged to form the Town of Sullivan's Island.
South Carolina's current Governor, Mark Sanford is a resident of Sullivan's Island