It is normal for a newborn's belly button to bleed a little bit after the umbilical cord falls off, as noted by Mayo Clinic. It is also normal when the cord is still in place as it heals.
While a small amount of bleeding is normal, parents should be aware of signs of infection. This includes redness, warmth or pus in or around the belly button. Swelling is another sign of infection. If any of these symptoms are in place, the parents should seek medical treatment for their child as soon as possible in order to have the infection treated properly.