The 54th Massachusetts, an infantry regiment composed of African-American soldiers, led the Union attack at dusk on July 18. They were backed up by several other regiments in Gillmore's brigade, consisting of white troops from Connecticut, Pennsylvania, New Hampshire, Ohio, and New York. Members of the brigade scaled the parapet but after brutal hand-to-hand combat were driven out with heavy casualties. The 54th's colonel, Robert Gould Shaw, was killed. The 54th was hailed for its valor. Their conduct improved the reputation of African Americans as soldiers, leading to greater Union recruitment, which strengthened the Northern states' numerical advantage.
The approach to the fort was constricted to a strip of beach 60 yards (55 m) wide. After a bombardment from both land and sea, the Union infantry moved in. The assault force was headed by the 54th Massachusetts and included five other regiments, around 5,000 men in total. Unfortunately for the assault force, the prior bombardment failed to seriously damage the fighting power of the fort. Consequently, the Union infantry suffered considerable casualties in the rush towards the fort.
As the Union troops reached the parapets, the fighting proved intense. Three brigades managed to occupy a portion of the walls, but they were forced to withdraw after an hour of fierce hand-to-hand combat where almost every officer was killed.
In all, 1,515 Union soldiers were killed, captured, or wounded. Only 315 men were left from the 54th after the battle. One hundred members of the 54th were reported missing after the battle and were never seen again. Union Brig. Gen. George Crockett Strong was wounded and died twelve days later in a hospital. Confederate casualties numbered 174.
Following the Union repulse, engineers besieged the fort. Confederates abandoned the fort on September 7, 1863, after resisting 60 days of shelling.
A depiction of the battle takes place in the film Glory.