The village gets its name from the Gobion family, who resided in this area, and dates from after the Norman invasion of 1066. The village built up around Higham Gobion Castle, an 11th century motte and bailey fortress. The ruins of the castle are still visible today. It also has a catholic church, St. Margaret's, that contains a monument of Dr. Edmund Castell, who was a rector there. The church is part of the Diocese of St Albans.