For popular music, MTV provides an extensive alphabetical list of bands and musicians. About.com has a shorter alphabetical list of musicians that is mainly focused on classical music.
MTV's website provides an alphabetical list of musicians that features thousands of names and goes from the Britpop-influenced guitar rock band simply named A all the way to ZZ Top and beyond. For many of these artists there are also biographies.
About.com's alphabetic list for classical music is sorted by last name and ranges from Spanish composer Isaac Albéniz to the renowned German-born film composer Hans Zimmer. This alphabetical list provides biographies for all its entries as well.