Mills took an early interest in acting, making his professional debut at the London Hippodrome in The Five O'Clock Girl in 1929. He made his film debut in The Midshipmaid (1932), and appeared as Colley in the 1939 film version of Goodbye, Mr Chips, opposite Robert Donat.
Mills joined-up in September 1939 at the start of World War II, and was posted into the Royal Artillery. He was later commissioned as a Second Lieutenant, and was discharged in 1941 due to medical reasons. He starred in his friend Noel Coward's In Which We Serve.
He took the lead in Great Expectations in 1946, and subsequently made his career playing traditionally British heroes such as Captain Robert Falcon Scott in Scott of the Antarctic (1948). Over the next decade he became particularly associated with war dramas, such as The Colditz Story (1954), Above Us the Waves (1955) and Ice Cold in Alex (1958). He often acted in the roles of people who are not at all exceptional, but become heroes due to their common sense, generosity and right judgement. Altogether he appeared in over 120 films.
For his role as the village idiot in Ryan's Daughter (1970) — a complete departure from his usual style — Mills won an Academy Award for Best Supporting Actor. His most famous television role was probably as the title character in Quatermass for ITV in 1979. Also on the small screen, in 1974 he starred as Captain Tommy "The Elephant" Devon in the six-part television drama series The Zoo Gang, about a group of former underground freedom fighters from World War II, with Brian Keith, Lilli Palmer, and Barry Morse.
In 2002 Mills released his extensive home movie footage in a documentary/film entitled John Mills' Moving Memories, with interviews with Mills, his children Hayley, Juliet and Jonathon and Richard Attenborough. The film was directed and edited by Marcus Dillistone, and features behind the scenes footage and stories from films such as Ice Cold in Alex and Dunkirk. In addition the film also includes home footage of many of John Mills' friends and fellow cast members including Sir Laurence Olivier, Harry Andrews, Walt Disney, David Niven, Dirk Bogarde, Rex Harrison, Tyrone Power.
His first wife was the actress Aileen Raymond. They were married in 1927 and divorced in 1941.
His second wife was the dramatist Mary Hayley Bell. Their marriage on 16 January 1941 lasted 64 years, until his death in 2005. They were married in a rushed civil ceremony, due to the war, and it was not until 60 years later that they had their union blessed by a church . They had two daughters, Juliet, star of television's Nanny and the Professor and Hayley, a Disney child star noted for starring in The Parent Trap, and one son Jonathan Mills. In 1947 he appeared with his daughters in the film So Well Remembered.