He was born at Comrie, Perthshire, where his father, the Rev. Samuel Gilfillan, the author of some theological works, was for many years minister of a Secession congregation. After an education at the University of Glasgow, in March 1836 he was ordained pastor of a Secession congregation in Dundee. He published a volume of his discourses in 1839, and shortly afterwards another sermon on Hades, which brought him under the scrutiny of his co-presbyters, and was ultimately withdrawn from circulation.
Gilfillan next contributed a series of sketches of celebrated contemporary authors to the Dumfries Herald, then edited by Thomas Aird; these, with several new ones, formed his first Gallery of Literary Portraits, which appeared in 1846 and had a wide circulation. It was quickly followed by a Second and a Third Gallery.
In 1851 his most successful work, the Bards of the Bible, appeared. His aim was that it should be a poem on the Bible and it was far more rhapsodical than critical. His Martyrs and Heroes of the Scottish Covenant appeared in 1832, and in 1856 he produced a partly autobiographical, partly fabulous, History of a Man. From 1853 to 1860 he was occupied with editing the 48-volume Library Edition of the British Poets.
For thirty years he was engaged upon a long poem, on Night, which was published in 1867, but its theme was too vast, vague and unmanageable, and the result was a failure. As a lecturer and as a preacher he drew large crowds, but his literary reputation has not proved permanent. He died having just finished a new life of Burns designed to accompany a new edition of the works of that poet.