White clothes are traditionally associated with summer because they keep the wearer cool. Labor Day is celebrated on the first Monday in September, making it a natural endpoint for some of the social conventions surrounding the summer months, including summer fashions such as white clothing.
According to Time magazine, people probably first began wearing white in summer for purely practical reasons. Without air conditioning and other artificial methods of keeping cool, they had to find some way to beat the oppressive heat. It was also quite uncommon to wear fewer or lighter clothes in summer. Women still wore heavy dresses, and men wore shirts and suit coats regardless of the temperature. White clothing was seen as being "of a lighter weight" than darker colors.
It's difficult to determine when, exactly, Labor Day became the cut-off point for white clothing. As it was not declared a holiday until 1894, it's likely that it was not until the 20th century. Many fashion magazines from the mid-1900s have references to putting away white clothes after Labor Day, meaning that it's likely it was part of the oral tradition long before it made it into print. The rule may have grown in popularity throughout the 1950s, as the U.S. middle class expanded.