Sumner was educated at Eton and at Trinity College, Cambridge and graduated BA in 1814, MA in 1817. Later on he was ordained deacon and priest. In the two winters of 1814-1816 he ministered to the English congregation at Geneva, and from 1816 to 1821 was curate of Highclere, Hampshire. In 1820 George IV wished to appoint him canon of Windsor, but the prime minister, Lord Liverpool, objected; Sumner received instead a royal chaplaincy and librarianship, and other preferments quickly followed, till in 1826 he was consecrated bishop of Llandaff and in 1827 bishop of Winchester.
In his long administration of his latter diocese he was most energetic, tactful and munificent. Though evangelical in his views he by no means confined his patronage to that school. In 1869 he resigned his see, but continued to live at the official residence at Farnham until his death on the 15th of August 1874.
He published a number of charges and sermons, and The Ministerial Character of Christ Practically Considered (London, 1824). He also edited and translated John Milton's De doctrina christiana, which was found in the State Paper office in 1823, and formed the text of Macaulay's famous essay on Milton.