After a stint the Navy, Impellitteri attended Fordham Law School, where he received his law degree in 1924. He served as a New York Assistant District Attorney from 1929 to 1938 before becoming a clerk to secretary to Tammany Supreme Court Justice Joseph A. Gavagan.
In 1945, Mayor William O'Dwyer picked Impellitteri to run for President of the City Council on the Tammany Hall slate. In 1945 he ran on the Democratic and American Labor Party lines, but when he was up for reelection in 1949 he ran on the Democratic Party line alone.
On August 31, 1950, O'Dwyer, pursued by both federal and state investigators, was suddenly appointed by President Harry S. Truman as ambassador to Mexico, where he would be beyond the reach of officials who wanted his public testimony in several matters on which he preferred not to speak. (He did not return to NYC for eleven years, long after his ambassadorship had ended, preferring to remain in Mexico).
Under the City Charter of the day, when O'Dwyer resigned, City Council President Impellitteri became acting mayor. The Tammany bosses didn't think he was Mayor material, and they refused to nominate him as the Democratic candidate for the special election in November 1950, which instead went to highly regarded New York State Supreme Court Judge Ferdinand Pecora, who was also given the Liberal line. "Impy" (as he was fondly known) was forced to run as an independent under the banner of the new “Experience Party”. He also popularized the phrase "unbought and unbossed" during his 1950 campaign.
Impellitteri was the first mayor since the consolidation of greater New York in 1898 who was elected without a major party’s ballot line, and his election was a populist uprising against the political system.
Impellitteri’s inauguration, held on November 14, 1950, absent either a band or a platform, was both swift and simple. Outside City Hall, he pledged to “do my level best to justify the confidence you have reposed in me.”
Shortly after Impellitteri's succession, the Brooklyn District Attorney arrested bookie Harry Gross and launched a corruption scandal that ultimately caused nearly 500 police officers of all ranks to resign, retire, or be fired. This famous scandal caused Impellitteri to vigorously support the Brooklyn District Attorney, Miles McDonald, and fire everyone who had been associated with former Mayor William O'Dwyer.
Impellitteri is credited with trying to rein in the budget, raising the bus and subway fare to fifteen cents, establishing parking meters on city streets for enhanced revenue and increasing the sales tax. He aspired to be a new light in city politics, but his administration met with some resistance from the established order. At the time Robert Moses wielded significant influence, who, according to Robert Caro (in his Robert Moses biography The Power Broker), Impellitteri deferred to behind the scenes.
Impellitteri ran for reelection in 1953, but was defeated by then Manhattan Borough President Robert F. Wagner, Jr., who appointed him a criminal court judge in 1954. He served out his career in public office on the bench. Following his wife's death he spent much of his time at the New York Athletic Club
"Impy" retired from the bench in 1965 and succumbed to Parkinson’s disease on January 29, 1987, in Bridgeport, Connecticut.