They are held up either by a garter (old style) or attached to undershorts with Velcro tabs. In the past, players kept their socks inside the upper ankle area of their skates, but newer style (mid-90s to the present) calls for the sock to be pulled down over the outside heel of the skate.
Athletic tape is generally used below the knee to hold the sock in place, and sometimes at the ankle as well.
Hockey socks, along with a jersey, are considered part of a team uniform under USA Hockey's rule 304(a) (2005-06 season) and are required as coverage for protective equipment (shinguards). Organized teams wear hockey socks in designated team colors, complimentary to the team jersey.