McMillan was born in Hamilton, Ontario, Canada to William and Grace McMillan, both Scottish natives. He was educated in the public schools of Hamilton before moving to Detroit, Michigan in 1855 to embark on a career in business.
McMillan's first position was as a clerk for Buhl, Ducharme & Co., a wholesale hardware firm. At the age of 20, he left to become the purchasing agent for the Detroit & Milwaukee Railway. In 1863, he helped, along with John Stoughton Newberry, to organize the Michigan Car Company. He later built and became president of the Duluth, South Shore and Atlantic Railway and was largely interested in shipbuilding and lake transportation companies. For three years he was president of the Detroit Board of Park Commissioners and for four years a member of the Detroit Board of Estimates.
McMillan was the only person to be elected Chairman of the Michigan Republican Party three non-consecutive times (1879, 1886 and 1890). He was a presidential elector on the Republican ticket in 1884. He was elected as a Republican to the United States Senate in 1889 and was reelected in 1895 and 1901, serving from March 3, 1889, until his death. He was chairman of the Committee on Manufactures in the Fifty-first and Fifty-second Congresses, and of the Committee on the District of Columbia in the Fifty-fourth through Fifty-seventh Congresses. He is also remembered for his chairmanship of the Senate Park Improvement Commission of the District of Columbia (better known as the McMillan Commission), which was responsible for the creation of the National Mall.
McMillan died in Manchester, Massachusetts and is interred in Elmwood Cemetery, in Detroit. He was survived by his wife Mary and their six children: William Charles, Grace Fisher, James Howard, Amy, Philip Hamilton and Francis Wetmore.