He was born in the hamlet of Dragley Beck in the parish of Ulverston in Lancashire. He started in life as superintending clerk of an iron foundry at Liverpool and afterwards, in his twenties, taught mathematics at a private school in Greenwich.
Through the interest of Sir George Leonard Staunton, to whose son he taught mathematics, he was attached on the first British embassy to China from 1792-94 as comptroller of the household to Lord Macartney. He soon acquired a good knowledge of the Chinese language, on which he subsequently contributed interesting articles to the Quarterly Review; and the account of the embassy published by Sir George Staunton records many of Barrow's valuable contributions to literature and science connected with China.
Although Barrow ceased to be officially connected with Chinese affairs after the return of the embassy in 1794, he always took much interest in them, and on critical occasions was frequently consulted by the British government.
In 1797 he accompanied Lord Macartney, as private secretary, in his important and delicate mission to settle the government of the newly acquired colony of the Cape of Good Hope. Barrow was entrusted with the task of reconciling the Boers and "Kaffirs" and of reporting on the country in the interior. On his return from his journey, in the course of which he visited all parts of the colony, he was appointed auditor-general of public accounts. He now decided to settle in South Africa, married Anne Maria Trüter, and in 1800 bought a house in Cape Town. But the surrender of the colony at the peace of Amiens (1802) upset this plan. He returned to England in 1804, was appointed Second Secretary of the Admiralty by Viscount Melville, a post which he held for forty years (apart from a short period in 1806-07 when there was a Whig government in power).
In particular, when Lord Grey took office as Prime Minister in 1830 Barrow was especially requested to remain in his post, starting the principle that senior civil servants stay in office on change of government and serve in a non-partisan manner. Indeed, it is during his occupancy of the post that it was renamed Permanent Secretary.
He enjoyed the esteem and confidence of all the eleven chief lords who successively presided at the Admiralty board during that period, and more especially of King William IV while lord high admiral, who honoured him with tokens of his personal regard.
In his position at the Admiralty, Barrow was a great promoter of Arctic voyages of discovery, including those of John Ross, William Edward Parry, James Clark Ross, and John Franklin. Point Barrow in Alaska is named for him. He is reputed to have been the initial proposer of St Helena as the new place of exile for Napoleon Bonaparte following the Battle of Waterloo in 1815.
He retired from public life in 1845 and devoted himself to writing a history of the modern Arctic voyages of discovery (1846), as well as his autobiography, published in 1847. He died suddenly on November 23, 1848.
The Sir John Barrow monument on Hoad Hill overlooking his home town of Ulverston was built in his honour (though it is more commonly called The Hoad).
Besides the numerous articles in the Quarterly Review already mentioned, Barrow published among other works: