The son of Sir John Erskine, Laird of Dun, he was educated at King's College, University of Aberdeen. At the age of twenty-one Erskine was the cause — probably by accident — of a priest's death, and was forced to go abroad, where he came under the influence of the new learning. It was through him that Greek was first taught in Scotland by Pierre de Marsilliers, whom he brought to live at Montrose. This was a factor in the progress of the Reformation. Erskine was also drawn towards the new faith, being a close friend of George Wishart, the reformer, from whose fate he was saved by his wealth and influence, and of John Knox, who advised him to discountenance the mass openly.
In the stormy controversies of the time of Mary I of Scotland and her son, James VI, Erskine was a conspicuous figure and a moderating influence. He was able to soothe the queen when her feelings had been outraged by Knox's denunciations — being a man "most gentill of nature" — and frequently acted as mediator both between the Roman Catholic and reforming parties, and among the reformers themselves. In 1560 he was appointed — though a layman — superintendent of the reformed church of Scotland for Angus and Mearns, and in 1572 he gave his assent to the modified episcopacy proposed by James Douglas, 4th Earl of Morton at the Leith convention.
Though never himself ordained, he was held in such high esteem by the leaders of the church that he was elected moderator of the general assembly several times (first in 1564), and he was amongst those who in 1588 drew up the Second Book of Discipline. From 1579 he was a member of the king's council. Erskine owed his peculiar influence among the Scottish reformers to his personality; Queen Mary described him as "a mild and sweet-natured man, with true honesty and uprightness."