O'Sullivan attended American University in Washington, D.C. He majored in business administration, and graduated with a bachelor's degree in 1974. While still in college, he earned money working as a laborer for a company building the city's subway system, Metro. It was during this time that he first joined the Laborers' Union (Local 456).
After matriculating, O'Sullivan was a high school teacher and baseball coach in West Virginia for three years. In 1978, O'Sullivan started computer services company. He married, and had a son and a daughter.
In 1993, LIUNA president Arthur Coia appointed O'Sullivan assistant director of the international union's construction, maintenance and service trades department. O'Sullivan moved his family to Clifton, Virginia. During his tenure as assistant director of the department, O'Sullivan convinced the union to build a number of new training centers around the country.
In February 1999, O'Sullivan was elected an international vice president. O'Sullivan was appointed mid-Atlantic regional manager of the union and assistant to the president, and the union's chief of staff shortly thereafter.
The United States Department of Labor (DOL) and United States Department of Justice (DOJ) had prosecuted the Laborers' Union in 1995 for racketeering, corruption and ties to organized crime. A consent decree permitted Coia (elected in 1992) to remain president so long as he made significant progress toward internal reforms. DOJ retained the authority to take over the union (appointing its own officers, setting its own budget, and making its own reforms) if Coia did not make what DOJ considered to be adequate progress toward reform. The agreement was modified and extended for one year in January 1998, and again in January 1999. By 1999, 226 individuals were expelled from the union, and 40 trusteeships of local unions established. A major reform was the first secret-ballot election for president and secretary-treasurer at the union's 1996 convention, which was also the first contested election for a LIUNA presidency.
Coia, had been cleared by federal and union officials of a number of serious crimes and violation of union rules in 1998, but new evidence pointed to more fraud. According to the government, Coia obtained three Ferraris from a luxury car dealer at a cost far below fair-market value. In return, he used his influence to steer union business to the dealer. Coia later pled guilty to federal tax evasion and fraud. He avoided a jail term, and was permitted to retire while still receiving a portion of his salary.
After being vetted by federal officials, the union's executive council elected O'Sullivan president to replace Coia.
However, O'Sullivan's presidency has not been without controversy in other ways.
In 2001, O'Sullivan broke with the AFL-CIO and supported the drilling for oil in Arctic National Wildlife Refuge. He even co-authored an op-ed piece with United States Secretary of the Interior Gale Norton advocating increased oil production.
The scandal was reported by the press in March 2002. AFL-CIO president John Sweeney pushed the board to appoint an internal investigator. When the investigator's report was complete in November 2002, Sweeney pushed the board to make the report public. A majority of the board voted to keep the report secret. Sweeney resigned from the board, then threatened to bring the matter up before a public meeting of the executive council of the AFL-CIO. The board established an eight-member special advisory committee, headed by O'Sullivan, to decide whether to release the report. The committee unanimously agreed to release the report, but O'Sullivan was out-voted 6 to 2 to accept the report's recommendation that directors return their profits to ULLICO.
In a hastily-organized board meeting late on April 23, 2003, Sweeney, O'Sullivan and Edwin D. Hill (president of the International Brotherhood of Electrical Workers) nominated a reform slate of 13 new board members. Georgine withdrew his name as a candidate for the board. O'Sullivan was only one of two incumbents to return to the board. O'Sullivan was elected ULLICO's new chairman, replacing Georgine.
O'Sullivan threatened to disaffiliate from the AFL-CIO at its quadrennial convention in late July 2005, but did not do so. Nevertheless, he said his union would leave the federation in time.
In February 2006, O'Sullivan took the first step toward disaffiliation by withdrawing his union from the Building and Construction Trades Department (BCTD) of the AFL-CIO. The Laborers and the International Union of Operating Engineers (an AFL-CIO affiliate) quit the BCTD and formed a rival group, the National Construction Alliance. Joining the Alliance were the United Brotherhood of Carpenters and Joiners of America (not an AFL-CIO affiliate), the Teamsters (not an AFL-CIO affiliate), the Iron Workers (an AFL-CIO affiliate which remained part of BCTD) and the Bricklayers (an AFL-CIO affiliate which remained part of BCTD). O'Sullivan said BCTD had been ineffective in organizing new members and stopping the proliferation of nonunion contractors. O'Sullivan made four demands on BCTD: Its leaders must resign and new elections must be held; its budget must be trimmed to permit more money to be spent on organizing; it must alter its proportional representation rules, which give more delegates to smaller unions; and it must revise its rules for determining which unions have jurisdiction over various kinds of work. AFL-CIO officials say they agreed to all of O'Sullivan's demands except for the first, arguing that the constitutionally scheduled BCTD elections should occur on schedule.
Finally, on June 1, 2006, the Laborers' Union disaffiliated from the AFL-CIO and joined the Change to Win Federation.