Lumley was the son of Brigadier General Osbert Lumley. He himself saw service in World War I with the 11th Hussars in France 1916–1918. He continued to be attached to the Yorkshire Dragoons 1921–1937.
Lumley sat in the House of Commons as Member of Parliament for Hull East 1922–29, then York 1931–37. In 1923 he was Parliamentary Private Secretary to William Ormsby-Gore, from 1924–26 to Sir Austen Chamberlain and subsequently to Anthony Eden. In 1937, he was appointed Governor of Bombay, serving until 1943.
He succeeded to the Earldom of Scarbrough in 1945 following the death of his uncle. He served as Lord Chamberlain from 1952 to 1963 and chancellor of the University of Durham from 1958 to 1969. He was made a Knight of the Garter in 1948.