Born on a farm near the hamlet of Lewis Center, Ohio, Willis was the son of a Civil War veteran, Vermont-born J.B. Willis and his wife Lavinia A. (Buell). Willis graduated from Ohio Northern University in 1894. After teaching at Ohio Northern for twelve years, Willis was admitted to the bar and began practicing law. He served in the Ohio House of Representatives from 1900 to 1904 while teaching at Ohio Northern, and was subsequently elected to the U.S. House of Representatives in 1910, serving from 1911 to 1915. Elected to the governorship in 1914, he served one two-year term from 1915 to 1917, but was not re-elected, being defeated by James M. Cox, whom he had defeated in 1914. Cox also defeated Willis in 1918.
After placing Warren Harding's name in nomination at the 1920 Republican National Convention, Willis was elected to the U.S. Senate in 1920, replacing Harding, who then resigned his seat to take the presidency, allowing Willis to take his seat early. During his Senate tenure, Willis served as Chairman of the Senate Committee on Territories and Insular Possessions, which had jurisdiction over territories including Alaska, Hawaii, the Philippines, and Puerto Rico, from 1923 to 1928.
Willis died in office in 1928 at Gray Chapel, Ohio Wesleyan University, in Delaware, Ohio, during a Republican Party event involving a "favorite son" presidential bid for Willis organized by the Delaware County Willis-for-President Club. He was buried at Oak Grove Cemetery. Willis's official papers were donated to and are open for research at the Ohio Historical Society.