Born in Canterbury, England, he served in the Royal Navy for over twenty years, attaining the rank of Commander. During his time in the Navy, he became a champion boxer and rugby player, and when he was discharged he was one of the youngest men ever to obtain the rank of Command in the Royal Navy. Gwillim began his acting career in earnest in the '50s, working on both stage and screen. Onstage, he was a member of both the Royal Shakespeare Company and the Old Vic. He performed in an extensive amount of theatre, both classics and modern plays, on the both the West End and Broadway. Some of his most notable roles include: playing in Sir Ralph Richardson's production of The Merchant of Venice; The Right Honourable Gentleman with lifelong friend Anthony Quayle; a revival of My Fair Lady with Rex Harrison, playing Colonel Pickering; John Gielgud's The Constant Wife, with Ingrid Bergman; and The Iceman Cometh'', with James Earl Jones.
Gwillim also featured in over sixty films and television series, usually war films or historical epics. His military background, commanding presence and deep, booming voice typecast him as soldiers and authority figures. Some of his most notable roles include playing a warship captain in Sink the Bismarck! (1960), the obnoxious club secretary in Lawrence of Arabia (1962), King Aeetes in Jason and the Argonauts (1963), an RAF officer in the James Bond film Thunderball (1965), the Lord High Judge in A Man for All Seasons (1966), and General Harold Alexander in Patton (1970). He also had a recurring role on the TV series Danger Man, The Saint, and The Troubleshooters.
Other film roles include The Battle of the River Plate (1956), The One That Got Away (1957), Solomon and Sheba (1959), The Curse of the Mummy's Tomb (1964), Battle of Britain (1969), Cromwell (1970),Circus of Horrors (1960), Sword of Sherwood Forest (1960), Northwest Frontier (1959), In Search of the Castaways (1962) and Clash of the Titans (1981).
Gwillim was twice married: to Peggy Bollard, until 1958, and Olivia Selby, from 1969 until his death. He had two children from his first marriage, Sarah-Jane Gwillim and David Gwillim, and a third from his second, Jaxon Duff Gwillim. Gwillim's children also became actors, and he acted onstage with them in 1995 in a production of On Borrowed Time, which was his last onstage performance.