From 1812 until 1820, he was Member of Parliament for Boston in Lincolnshire. On 29 June 1820, he succeeded his father as 2nd Baron Gwydyr, 3rd Baronet Burell of Knipp and Deputy Lord Great Chamberlain. And on 29 December 1828, he succeeded his mother as 22nd Baron Willoughby de Eresby and Joint (1/2) Hereditary Lord Great Chamberlain.

On 19 October 1807, he married Sarah Clementina née Drummond, daughter of James Drummond, 11th Earl of Perth and Clementina Elphinstone. They had three children:

• Clementina Drummond-Willoughby, 24th Baroness Willoughby de Eresby (1809–1882) — had issue;
• Charlotte Augusta Annabella Drummond-Willoughby (1815–1879), who married the 2nd Baron Carrington and had issue;
• Albyric Drummond-Willoughby, 23rd Baron Willoughby de Eresby (1821–1870).

His wife died on 26 January 1865. He died only nearly a month later.