Poltimore was the son of George Bampfylde, 1st Baron Poltimore, and his second wife Caroline, daughter of General Frederick William Buller, and succeeded his father in the barony in 1858. He sat on the Liberal benches in the House of Lords and served in the first Liberal administration of William Gladstone as Treasurer of the Household from 1872 to 1874. In 1872 he was sworn of the Privy Council.
Lord Poltimore married Florence Sara Wilhelmine, daughter of Richard Brinsley Sheridan of Frampton Court, in 1858. He died in May 1908, aged 71, and was succeeded in the barony by his son Coplestone. Lady Florence Poltimore had a diamond tiara made for her by Garrards of London, the "Poltimore Tiara" was worn by HRH Princess Margaret, on her wedding day in 1960. Lady Poltimore died in 1909.