Born in Phoenixville, Pennsylvania, he attended Yale University and graduated 10th in his class in 1853. He was admitted to the bar in 1856 and was District Attorney of Chester County, Pennsylvania, from 1859 through 1864.
He led militia forces organized to battle back threatened Confederate invaders in 1862 and 1863 and served in the Union army during the American Civil War as an infantry captain and as a major in the cavalry.
He became a leader in the Republican party, and was a prominent opponent of his father-in-law, Simon Cameron, in the fight within the party in 1871.
MacVeagh was U.S. Ambassador to Turkey in 1870 through 1871, and was a member of the state constitutional convention of 1872 and 1873. He also served as chairman of the MacVeagh Commission, sent in 1877 by President Rutherford B. Hayes to Louisiana, which secured the settlement of the contest between the two existing state governments and thus made possible the withdrawal of U.S. troops from the state.
MacVeagh was the 37th Attorney General of the United States in 1881 under President James A. Garfield, but resigned immediately after Garfield's death. In 1892 he supported Grover Cleveland, the Democratic nominee for the presidency, and from 1893 to 1897 was U.S. Ambassador to Italy. He returned to the Republican party in 1896. In 1903 he was chief counsel of the United States before the Hague tribunal in the case regarding the claims of Germany, Britain and Italy against the republic of Venezuela.