Sullivan was one of the final "bosses" of the infamous political machine, Tammany Hall. Try as he might, he could not restore the suffering machine, and witnessed its slow loss of power during his reign.
Sullivan was elected as a Democrat to the Sixty-fifth Congress, and re-elected eleven times, holding federal office from March 4, 1917 to January 3, 1941. While in the House he was chairman of the Committee on Expenditures in the Department of Labor (Sixty-fifth Congress). Sullivan did not seek re-election in 1940, and resided in New York City until his death in 1942. He was interred at Calvary Cemetery in Woodside, New York.