The National Center for Home Food Preservation recommends using glass jars with tight-fitting lids for storing pickled eggs. Although any container with that seals and can withstand high heat and acidity should be safe to use, glass jars are easier to sterilize.
Most sources on pickling eggs give instructions using glass jars for storage. Because botulism can grow in acidic environments, even in the refrigerator, and glass jars are non-reactive, they are probably the safest choice. Pay careful attention directions for sterilizing and sealing jars for long-term storage. Should a jar’s contents suddenly change color, the lid poke up or the contents smell off when you open the jar, safely dispose of the jar immediately.
As long as plastic or metal containers can withstand the heat of sterilization, you may be able to use them to pickle eggs. However, unless you have a way to test plastic or metal containers or buy them from a trusted source, you may have difficulty knowing that the container can maintain an airtight seal as the eggs cure. Further, because pickling solutions are highly acidic and can break down heavy plastics and even metal which can break a container’s seal (and even create holes) can let in air and bacteria.