There are insect parts in peanut butter because The Food and Drug Administration allows certain levels of "natural or unavoidable defects in foods [...] that present no health hazard." Up to 145 bug parts may be in an 18-ounce jar.
According to a 2009 New York Times article, the FDA determines levels of acceptable contamination based on criteria such as "lengths of hairs, sizes of insect fragments and distribution of filth." If the substances pose no fundamental threat to health, they are allowed. The FDA justifies this policy level determination because it believes that it is economically infeasible to expect producers to "grow, harvest or process raw products" that do not contain any naturally occurring defects.