A native of Winthrop, Massachusetts, O'Connell attended Boston College. He worked in private business and taught at Sanborn Academy, East Kingston, New Hampshire, until the outbreak of World War II. While serving in the U.S. Navy, he befriended a fellow officer, Jim Britt, the radio play-by-play announcer for the Red Sox and Boston's National League team, the Braves. After the war, Britt put O'Connell in touch with the Red Sox front office, and he entered baseball in as business manager of the Lynn Red Sox, Boston's Class B New England League farm club.
As the GM seat lay vacant — but perhaps being kept warm for Williams, who was named an "executive assistant" to Yawkey — O'Connell was promoted to executive vice president; meanwhile, field manager Pinky Higgins, who had become a friend of Yawkey's, staked out a position as the top "baseball man" in the Red Sox organization. The New York Times reported on October 1, 1960, that Higgins would assume responsibility for all player personnel decisions in the Boston organization. Indeed, the club spent 1961-62 without a true general manager. Although O'Connell is listed by the Red Sox media guide as de facto GM, he likely focused only on the business aspect of the job; it it is unclear (and doubtful) that he had any baseball operations role. Higgins shed his on-field responsibilities and formally became executive vice president/GM at the close of the season.
In the early 1960s, Boston overhauled its farm system and scouting operation and was beginning to produce outstanding talent, but the big league Red Sox continued to struggle and attendance dwindled. Finally, during the closing days of a dispiriting 100-loss season, Yawkey fired Higgins and offered the general manager title to O'Connell on September 16.
The Red Sox posted winning seasons from 1968-74 and continued to be among the AL leaders in home attendance, but could not match the talent of the league's dominant teams of the era, the Detroit Tigers, Baltimore Orioles and Oakland Athletics. Nonetheless, Boston continued to harvest great talent from its farm system, including Carlton Fisk, Dwight Evans, Cecil Cooper, Bill Lee, John Curtis, Lynn McGlothen, Ben Oglivie, Juan Beniquez, Rogelio Moret, Rick Burleson, Jim Rice and Fred Lynn. The team also somewhat overcame its reputation for racial prejudice by increasing the number of African-Americans and Latin Americans on its playing roster. In , led by rookies Lynn and Rice, the Red Sox won the AL East title, swept defending world champion Oakland in the ALCS, and battled another NL powerhouse, the Cincinnati Reds, to the limit in a thrilling World Series. Once again, O'Connell was hailed as Executive of the Year in Major League Baseball.
The firing ended O'Connell's baseball career, although almost six years later, on June 6, , a bizarre postscript was added. A power struggle broke out among the Red Sox ownership group, and one of the general partners, Edward "Buddy" LeRoux, staged a coup d'état. LeRoux announced a takeover of the Red Sox, and fired Sullivan, his fellow owner, from the GM role. Surprisingly, he unveiled O'Connell, then 68, as his choice to lead the team — the first time O'Connell set foot inside Fenway Park since his 1977 dismissal. But LeRoux' "coup" was halted by court order, and Sullivan remained in power.
Over time, however, O'Connell and the Red Sox mended fences and he was admitted to the Boston Red Sox Hall of Fame in 1997. Pundits hailed him as the architect who most helped to create Red Sox Nation by bringing the team back from near-irrelevance in 1967.
By the time of his death, at age 87 on August 18 2002, in Lexington, Massachusetts, O'Connell was recognized as one of the most important men in Red Sox annals.