A material's color affects its ability to absorb light. An object of a certain color reflects that same color but absorbs light from all the other colors. Since light is energy, the more light that an object absorbs, the more heat it absorbs.
Black absorbs the most heat because it does not reflect any light. This explains why black uniforms are often avoided for outdoor recreation. White absorbs the least heat because it reflects all light wavelengths. For this reason, white is known as a cooling color, and it is sometimes used on building roofs and siding to repel sunlight.