He graduated at the University of Virginia in 1849, studied law at the College of William and Mary, and in Baltimore (where he was admitted to the bar), and was engaged in newspaper work in California until 1855. During 1857 to 1861 he was clerk of the Judiciary Committee of the United States House of Representatives. By 1859 he had become an outspoken secessionist, and during the Civil War he was one of the principal editors of the Richmond Examiner (along with Robert William Hughes, which supported the Confederacy but was hostile to President Jefferson Davis.
In 1864 Pollard sailed for England, but the vessel on which he sailed was captured as a blockade runner, and he was confined in Fort Warren in Boston Harbor from 29 May until 12 August, when he was paroled. In December he was placed in close confinement at Fort Monroe by order of the Secretary of War Edwin M. Stanton, but was soon again paroled by General B. F. Butler, and in January proceeded to Richmond, Virginia to be exchanged there for Albert D. Richardson (1833–1869), a well-known correspondent of the New York Tribune, who, however, had escaped before Pollard arrived. During 1867 to 1869 Pollard edited a weekly paper at Richmond, and he conducted the Political Pamphlet there during the presidential campaign of 1868.
His publications include Black Diamonds Gathered in the Darkey Homes of the South (1859), in which he advocated a reopening of the slave trade; The Southern History of the War (3 vols.: First Year of the War, with B. M. DeWitt, 1862; Second Year of the War, 1864; Third Year of the War, 1864); Observations in the North: Eight Months in Prison and on Parole (1865); The Lost Cause (1866); Lee and His Lieutenants (1867); The Lost Cause Regained (1868), a southern view of Reconstruction urging the necessity of white supremacy; The Life of Jefferson Davis (1869), an arraignment of the Confederate president; and The Virginia Tourist (1870).