Smith was born in Belfast, Ireland and immigrated to the United States in 1855 with his parents, who settled near Plymouth, Ohio, where he attended the public schools. He moved to Charlotte, Michigan, in 1867 where he engaged in agricultural pursuits and also worked as a mason. He graduated from the academic department of the University of Michigan at Ann Arbor in 1879 and from the law department in 1880. He was admitted to the bar in 1882 and commenced practice in Detroit. He was married to Lena Parkhurst in 1887.
Smith served as prosecuting attorney of Eaton County 1885-1888. He was president of the First National Bank of Charlotte in 1898 and also engaged in manufacturing and agricultural pursuits. Smith was a member of the board of aldermen in 1903 and a member of the State constitutional convention in 1908.
Smith was elected as a Republican to the Sixty-second and to the four succeeding Congresses, serving from March 4, 1911 until March 3, 1921. He was chairman of the United States House Committee on Labor in the Sixty-sixth Congress. Smith was not a candidate for renomination in 1920, but returned to Congress when he was elected to the Sixty-seventh Congress to fill the vacancy caused by the death of William H. Frankhauser, who died in office on May 9, 1921. Smith was reelected to the Sixty-eighth Congress, serving from June 28, 1921, until his death in Charlotte. He is interred in Maple Hill Cemetery in Charlotte.