He was born in Turriff, Aberdeenshire, Scotland and emigrated to Lower Canada with his parents in 1836 where he was active in suppressing the Lower Canada Rebellion of 1837. He was admitted to the bar in 1842 and established a commercial practice in Montreal. From 1857 until 1867 he was a member of the Legislative Assembly of the Province of Canada and was a member of the Executive Council from 1858 until 1861.
Rose was a delegate to the Colonial Conference in London in 1867 and again in 1868. In 1869, Rose moved to England to practice law and acted as an unofficial representative of the Canadian government. His was one of the very first missions of a partly diplomatic nature which Canada ever sent abroad. He was sent partly because his was perceived as being "gentlemanly" enough for the world of London politics. He worked for Canada's interests in trade and immigration, and lobbied for Canada's viewpoint on matter of Anglo-American relations. In this function he answered directly to the Prime Minister of Canada, Sir John A. Macdonald, since it was an informal position and was therefore not under the purview of the Governor General, meaning that communications could be kept secret from the Imperial Government. As well, Canada did not have a foreign affairs department at the time. This unusual situation did lead to a tradition of direct executive involvement in foreign affairs in Canada which still exists.