Born at Stamford Hill, London, on December 24, 1777, he was taught by Mr. Burnside at Walworth in 1787, and afterwards by a Mr. Freeman. He received adult baptism at Maze Pond Chapel, Southwark on August 1, 1796. Adopting the profession of his father and his grandfather, he commenced practice as a shorthand writer in 1803, and between that date and 1844 he took down in shorthand many of the most important appeals, trials, courts-martial, addresses, speeches, and libel cases, a number of which were printed as volumes from his notes. In pursuit of his calling he frequently visited Ireland and Scotland and many parts of England. He reported the impeachment of Lord Melville in 1806, the proceedings against the Duke of York in 1809, the trials of Lord Cochrane in 1814 and of Arthur Thistlewood in 1820, and the proceedings against Queen Caroline. In 1802, in conjunction with his father, he was appointed to take notes of evidence before the committees of the Houses of Lords and Commons, and in May 1813 he was formally appointed shorthand writer to the houses of parliament, his emolument being two guineas a day for attendance, and one shilling a folio for the transcript of his notes (MATTHIAS LEVY, Shorthand, 1862, pp. 86–94). He is mentioned as a famous shorthand writer in Byron's Don Juan, canto i. st. clxxxix.
In 1812, on the establishment of the Westminster auxiliary of the British and Foreign Bible Society, he was elected a member of the first committee, and soon after became secretary. In connection with the baptist denomination he was treasurer of Stepney College from 1828, and of their foreign missions from 1835. Like his father he was warmly interested in the anti-slavery movement. Towards rebuilding chapels in Jamaica and sending additional ministers there he was a liberal contributor, besides frequently receiving baptist missionaries into his own house. He purchased a residence at Muswell Hill, Middlesex, in 1826, when the Rev. Eustace Carey, who had recently returned from India, came to reside with him. The house was then licensed as a place of worship, and during four years Carey and other ministers held Sunday evening services in the drawing-room. Gurney died at Denmark Hill, Camberwell, on 25 March 1855. He married in March 1803 Miss Benham, who died at Muswell Hill in 1830.