Despite his long career in politics, he was not very conspicuous, and few outside his home district knew who he was and he had never introduced any legislation.
When Congress voted a pay raise in 1873 and made it retroactive for five years, Wheeler not only voted against the raise, but returned his salary adjustment to the Treasury department.
Wheeler's reputation for honesty was celebrated by Allan Nevins in his introduction to John F. Kennedy's Profiles in Courage. Roscoe Conkling, a Senator and a political boss offered "Wheeler, if you will act with us, there is nothing in the gift of the State of New York to which you may not reasonably aspire." Wheeler declined with "Mr. Conkling, there is nothing in the gift of the State of New York which will compensate me for the forfeiture of my self-respect." (John F. Kennedy, Profiles in Courage (New York, 1956), p. xiv.)
The convention was recessed for dinner, and as a sop to Roscoe Conkling, the party bosses announced that they would let the New York delegation pick the candidate for Vice President. So some of the delegation were discussing the matter and they were stymied. They couldn't think of anyone who they would want to stick with the position. Then one of them began to giggle. "What about Wheeler?" he chuckled. Soon everyone was having a hearty laugh, including Wheeler, and the next morning he was, much to everyone's surprise, nominated by acclamation. He won the nomination with 366 votes to the 89 for his nearest rival Frederick T. Frelinghuysen, who later served on the Electoral Commission.
Governor Hayes, when he heard of what had happened, remarked: "I am ashamed to say: Who is Wheeler?"
Not having done much campaigning, Wheeler didn't participate in the firestorm that took place after the election results were in November 1876.
Since Wheeler was a recent widower, his wife having died three months before he took the oath of office, President and Mrs. Hayes took pity on him, and the Vice President was a frequent guest at the White House's alcohol-free luncheons. Other than that, Wheeler merely presided over the Senate, which he found extremely tedious, and was little heard from otherwise. According to Hayes, Wheeler "was one of the few Vice Presidents who were on cordial terms, intimate and friendly, with the President. Our family were heartily fond of him."
Hayes had long announced he wouldn't run for a second term, and Wheeler wasn't even considered, even jokingly, for the 1880 nomination.