Baldwin was educated at Eton College, and grew up in the shadow of his father's political career. He joined the Irish Guards in 1916 and served in France through the remainder of World War I. After the war he travelled extensively and worked as a journalist and travel writer. He was in Armenia with the job of an infantry instructor. There the Bolsheviks imprisoned him for two months and later he was imprisoned by the Turks for a further grim five months. Despite his Conservative family, he gradually grew to adopt left-wing views and eventually announced that he was a Marxist and joined the Labour Party. He frequently addressed crowds from a socialist platform at Hyde Park Corner.
At the 1924 elections Baldwin contested the seat of Dudley for Labour. By this time his father was leader of the Conservative Party and Prime Minister, and his candidacy naturally attracted press comment. At the 1929 election he won Dudley, and served as a backbench member of Ramsay MacDonald's Labour government, facing his defeated father across the House.
He remained on personal good terms with his father despite their different politics, as each regarded their differences as being of principle and not personality. Baldwin refrained from personally attacking his father, and when he visited him, there was a tacit agreement that politics was not a suitable subject for discussion. Lucy Baldwin, who was also a strong Conservative, came from a background where questioning received opinion was regarded as a good thing, supported her son - although she did not like to attend the House of Commons to see her son and husband on opposite sides.
Like other young left-wing Labour MPs, Baldwin was critical of MacDonald's insistence on strict financial management and refusal to launch large Keynesian public works programmes. Early in 1931 Baldwin resigned from the Labour Party and briefly associated with Oswald Mosley's New Party, but repudiated Mosley after one day and rejoined Labour. When MacDonald formed the National Government Baldwin remained with the opposition Labour Party and inevitably lost his seat in the 1931 general election. He returned to journalism.
Baldwin fought Paisley at the 1935 election. In 1937 Stanley Baldwin retired from politics and was created Earl Baldwin of Bewdley. As a result Oliver Baldwin acquired the courtesy title Viscount Corvedale, although he remained a commoner. In 1939 he rejoined the army, becoming a major in the Intelligence Corps and serving in Egypt, Palestine, Syria, Eritrea and Algeria.
Baldwin was homosexual, a fact well known within the family but not to the public (his mother was again supportive and both parents acknowledged his long term relationship with John Boyle). Among the consequences of Baldwin's homosexuality was a rift with the novelist Rudyard Kipling, (who was Stanley Baldwin's first cousin, and was sometimes referred to as Oliver's uncle). Baldwin had idolised Kipling in his youth and had been a favourite of the Kipling family. But when Kipling learned of Oliver's "beastliness" (and his radical politics), he cut him off. When Kipling died in 1936, Baldwin made a speech attacking his famous relative which was widely reported, although the real reason for the hostility could not be mentioned.
At the 1945 general election, when Labour returned to power under Clement Attlee, Baldwin was elected for Paisley. In 1946 he was appointed Parliamentary Secretary to the Secretary for War, a post he held until 1947. But there was little chance that he would hold high office. His homosexuality was well-known, and Attlee held puritanical views on this issue: he kept Tom Driberg out of the government for the same reason.
When Stanley Baldwin died in 1947, Oliver succeeded him as Earl Baldwin of Bewdley. It was not possible at this time to renounce a peerage, and Baldwin had no choice but to leave the Commons and take his seat in the House of Lords. Later that year, presumably to give him a dignified exit from politics, he was appointed Governor of the Leeward Islands, a British colonial territory in the Caribbean. He created a minor scandal by taking John Boyle with him.
Partly for this reason, and partly because he made no secret of his continuing socialist views among the British planter elite in Antigua, Baldwin was recalled in 1950. He died in 1958 and was succeeded in the earldom by his brother.