During the American Civil War, the Union Navy and the Union Army occupied the islands early in the war. The whites had fled to the mainland while the blacks stayed, largely running their own lives (as they already had much of the time, since plantation families often stayed on the mainland to avoid malaria and isolation). This changed after President Abraham Lincoln signed the Emancipation Proclamation. When the proclamation went into effect, military officers read it to them and told them they were free. (However, it was technically not until the Thirteenth Amendment to the U.S Constitution was ratified that the blacks would actually be free, due to the fact that Southerners ignored the Emancipation Proclamation.)
They were also struck by the Sea Islands Hurricane in 1893.