James Murray was the son of Colonel Robert Murray of Clermont (after 1769, Sir Robert Murray, Bt.) and his first wife Janet Murray, a younger sister of Patrick Murray, 5th Lord Elibank. He inherited his father's title in 1771, while still a minor.
He had had his first commissions purchased in his mid-teens: Lieutenant, 19th Regiment, 1770, then Captain, 57th Regiment, 1771. Having spent 1772-73 and 1775 travelling in Europe, he returned to his regiment in Ireland in November 1775 and embarked for the Colonies in 1776 to serve in the American War of Independence. He was wounded at the battle of Brandywine, Pennsylvania, and shared his convalescence with his cousin Patrick Ferguson.
On 24 July 1794, he married Laura Pulteney, daughter of his cousin William Pulteney in Bath House, London. He changed his name to Murray-Pulteney. Henrietta inherited the Pulteney estates, worth about £50,000 per year.
Serving in the 18th Foot, he reached the rank of Colonel and from 1790 to 1811, he was Member of Parliament (MP) for Weymouth and Melcombe Regis. In 1807, he was invested to the Privy Council and was Secretary at War between 1807 and 1809. He died in Buckenham, Norfolk, from complications after losing an eye when a powder-flask accidentally exploded in his face.
There are two portraits of him in the Scottish National Portrait Gallery, one of which shows him with his younger half-brother John (by their father's second wife).