In 1947 he became secretary of the World Union of Mapam. Between 1948 and 1951 he worked as the Israeli consul in Poland. After returning to Israel, he served as political secretary of Mapam between 1953 and 1955.
He was first elected to the Knesset on Mapam's list in 1955, and was appointed Minister of Health in David Ben-Gurion's government. In November 1958 he also became Minister of Postal Services after the National Religious Party withdrew from the government. After the 1959 elections he continued as Minister of Health.
Although he was re-elected in 1961, Mapam were excluded from the government and Barzilai lost his cabinet position. However, he did serve as Deputy Speaker of the Knesset. In the 1965 elections he lost his seat, but was re-appointed Minister of Health by Levi Eshkol. Following the 1969 elections (in which he also failed to win a seat) he was appointed Minister without Portfolio, serving until his death in June 1970. Following his death, the hospital in Ashkelon (whose cornerstone he had laid in 1961) was renamed Barzilai Medical Center in his honour.