He was the son of Hon. George Villiers (1759-1827) and Theresa Parker, daughter of John Parker, 1st Baron Boringdon. He was grandson of Thomas Villiers, 1st Earl of Clarendon and brother of the George Villiers, 4th Earl of Clarendon. A statue of Villiers stands in West Park in Wolverhampton.
He was educated at St John's College, Cambridge, becoming a barrister at Lincoln's Inn in 1827. He held Benthamite political views, and enjoyed a long career in public service and Parliament. In 1832, he was a Poor Law Commissioner, and from 1833 to 1852 was examiner of witnesses in the Court of Chancery.
He served as a Liberal Member of Parliament (MP) for Wolverhampton from 1835 to 1885 and for Wolverhampton South from 1885 until 1898 (switching to the Liberal Unionist party in 1886). In 1847 he was also returned for Lancashire South but elected to sit for his former constituency. During his time in Parliament he worked towards free trade and opposed the Corn Laws and home rule for Ireland. He is noted as being the voice in parliament of the free trade movement before the election of Richard Cobden and John Bright. He also served as Judge Advocate General from 1852 to 1859, and President of the Poor Law Board from 1859 to 1866. He was the Father of the House of Commons from 1890 to 1898; however, the last time he attended Parliament was in 1895. He was the last remaining MP to have served under King William IV.
He was raised to the rank of an Earl's son in 1839 and appointed a Privy Counsellor in 1853. He was offered a peerage in June 1885, but declined.