Born near Newberry, Indiana, Bynum attended the country schools, and was graduated from Indiana University in 1869. He studied law. He was admitted to the bar in 1872 and commenced practice in Washington, Indiana. He served as the first city clerk. City attorney of Washington 1871-1875. He served as mayor of Washington 1875-1879. He moved from Daviess County to Indianapolis in 1880. He served as member of the State house of representatives 1881-1885, and served as speaker in 1885.
Bynum was elected as a Democrat to the Forty-ninth and to the four succeeding Congresses (March 4, 1885-March 3, 1895). He served for some time as whip of the Democratic minority. Censured by the House of Representatives on May 17, 1890, for the use of unparliamentary language. He was an unsuccessful candidate for reelection in 1894 to the Fifty-fourth Congress. He was active in the organization of the National (Gold-Standard) Democratic Party in 1896, and was chairman of its national committee 1896-1898. He settled in Washington, D.C.. He was appointed by President McKinley in 1900 a member of the commission to codify the United States criminal laws and served until 1906. He retired from the practice of law. He died in Indianapolis, Indiana, October 21, 1927. He was interred in Oak Grove Cemetery, Washington, Indiana.