Bubbles in plain water come from dissolved gases in the water, such as nitrogen and oxygen, that escape when the temperature changes. Pressurized water pipes also contribute to bubbles in plain water because water holds more oxygen when under pressure.
Cold water and high pressure cause more oxygen to dissolve in water. When water comes out of the tap, it is cold, but it begins to warm to room temperature. This heating causes the dissolved gases in the water to start coming out of the solution as bubbles.
Water inside pipes has more pressure and contains more dissolved gases, so when water exits the faucet the pressure decreases. This decrease in pressure causes bubbles to form as dissolved gases escape the water.