Ruggles-Brise was born in Finchingfield in Essex, the second son of Sir Samuel Brise Ruggles-Brise (1825–1899) and his wife, Marianne (née Smith). He had three brothers and seven sisters. His family have deep roots in Essex, having been based at Spains Hall in Finchingfield since the house was bought by Samuel Ruggles, a clothier, in 1760. His father was Conservative MP for East Essex from 1868 to 1884. Another relation, Sir Edward Ruggles-Brise, 1st Baronet, was MP for Maldon from 1922 to his death in 1942 (with a short intermission in 1923-4), and became a baronet in George V's Silver Jubilee honours list in 1935.
Ruggles-Brise was educated at home and at a private school near Hitchin, before attending Eton from 1869 to 1876 on a scholarship. His older brother, Archie, was already an Oppidan, and President of Pop. He read Mods and Greats at Balliol College, Oxford, graduating with a first in 1880. He also played in the college cricket team.
Ruggles-Brise came sixth in the civil service exam, and became a clerk in the Home Office in 1880. He was Principal Private Secretary to four Home Secretaries, William Harcourt, Richard Cross, Hugh Childers, and Henry Matthews. The latter appointed him as a Commissioner of Prisons for England and Wales in 1892. The long-serving incumbent chairman of the Prison Commission, Sir Edmund du Cane, was criticised by Gladstone Committee in 1895, and resigned. Herbert Asquith appointed Ruggles-Brise in his place, and he served as chairman until 1921. His main task was implementing the report of the Gladstone Committee, to combine reform with deterrence, and to separate youths from older men in adult prisons. Reform was undertaken under the Prison Act 1898, and physical punishments such as the treadwheel and the crank were abolished.
He travelled to the US in 1897 to study study the American reformatory system, visiting Zebulon Brockway's Elmira Reformatory. On his return, he formed a facility for young offenders at Bedford prison, but the regime took its name from the prison at Borstal near Rochester in Kent. The experiment became widespread under the Prevention of Crime Act 1908.
He became a CB in 1899 and was advanced to KCB in 1902. He wrote The English Prison System, published in 1921, and Prison Reform at Home and Abroad, published in 1924.
A confirmed batchelor for many years, he married Jessica Philippa Stonor (née Carew), widow of Francis Stonor, 4th Baron Camoys, on 3 September 1914. She died on 29 November 1928. He remarried on 6 June 1933, to Sheelah Maud Emily Reade, daughter of Captain the Hon. Francis Algernon James Chichester, and widow of Essex Edgeworth Reade. He died of throat cancer in Peaslake in Surrey, survived by his second wife. He was buried at Finchingfield.