The eastern end of the building dates from the fifteenth century; the western end is a nineteenth-century addition. The manor, which is mentioned in the Domesday Book, was built by Cardinal Beaufort as a hunting lodge. At one point it was occupied by insurrectionist Jack Cade. Thereafter it was a family home until the mid 1960s, when it became the folk music centre. It has been designated by English Heritage as a grade II* listed building.
In 1859, John Henry Parker wrote in his Account of Domestic Architecture: