Born in Richibucto, New Brunswick to Henry O'Leary and Mary O'Leary, he received his higher education from St. Joseph's College in Memramcook then studied Theology at the Grand Seminary in Montreal. On September 21 1901, O'Leary was ordained a priest in Richibucto. The same year he went to Rome, where he later earned doctorates in Theology, Philosophy, and Canon Law.
In 1907 O'Leary was appointed priest for the Sacred Heart Parish in Bathurst, New Brunswick. The following year, he was appointed Vicar General for the Diocese of Chatham. In 1913 O'Leary became Bishop for the Diocese of Charlottetown on Prince Edward Island but as St. Dunstan's Basilica in Charlottetown had recently burned down, O'Leary was consecrated at the Sacred Heart Church in Bathurst.
One of his first acts was to have a new cathedral built on the site of the ruined Basilica. It was completed during O'Leary's tenure and opened in 1919.
In World War I many Diocesan parishioners joined the armed services and several priests became chaplains. During this time, the new Saint Vincent Orphanage was completed to replace the old one in Charlottetown.
O'Leary founded a convent in 1916 when the Sisters of St. Martha of Prince Edward Island was established.
In 1919, during his tenure, St. Dunstan's College, which had been started years ago by Bishop Bernard McDonald on the closure of St. Andrew's College, became a university with the power to grant its own degrees. The first fundraising for the university raised over $40,000.
The Charlottetown Hospital, which had been backed by Bishop Peter McIntyre in the 19th century continued to grow as a new maternity department opened in 1918 and a school of nursing was established in 1920. The hospital was under the supervision of the Grey Nuns of Quebec, whose presence had been felt in the Diocese of Charlottetown for many years.
After seven years as Bishop of Charlottetown, O'Leary was appointed Archbishop of the Archdiocese of Edmonton, in the Western Canadian region. Archbishop Henry Joseph O'Leary died in 1938. His remains were interred in St. Joachim's Cemetery in Edmonton.
The Archbishop O'Leary Catholic High School in Edmonton, founded in the 1960s, was named after him.