Asteroids and comets differ in composition due to where they were formed. Most asteroids are the rocky remnants of failed planets created in the inner solar system, while comets formed further out where frozen gases and liquids could not be thawed by the heat of the sun.
When the solar system formed, it came together out of a great disc of stellar matter, rocks and gas. As the star formed at the center, heavier elements were pulled towards it while less dense materials were not affected. These heavier fragments began to collide, first forming spinning discs and then resolving into proto-planets. Mercury, Venus, Earth and Mars formed from these rocky planetesimals, but any that began to form past Mars were torn apart by the growing gravity well of Jupiter, creating a belt of small asteroids. These were close enough to the sun that any ice remaining melted, leaving only minerals behind.
Beyond the gas giants in the solar system, leftover fragments of ice and frozen gas still orbited the sun, but far enough out that they remained solid. These eventually began to collide and form bodies of their own, mostly made up of ice. The Oort Cloud at the periphery of the solar system is filled with these frozen bodies, and occasionally gravity sends one close enough to the sun to begin to melt, creating the distinctive frozen tail of a comet.