Born near Syracuse, New York, Upson attended the common schools and Williams College, Williamstown, Massachusetts. He studied law. He was admitted to the bar in 1851 and commenced practice in Syracuse, New York, in 1851. He moved to San Antonio, Texas, in 1854 and engaged in the practice of law. During the Civil War he served in the Confederate States Army as a volunteer aide, with the rank of colonel, on the staff of Gen. W.H.C. Whiting. He was appointed by the Confederacy associate justice of Arizona in 1862.
Upson was elected as a Democrat to the Forty-sixth Congress to fill the vacancy caused by the death of Gustave Schleicher. He was reelected to the Forty-seventh Congress and served from April 15, 1879, to March 3, 1883. He was an unsuccessful candidate for renomination in 1882. He resumed the practice of law in San Antonio, Texas, and died there February 8, 1902. He was interred in Confederate Cemetery.