He was influenced in his career by two uncles, John Hubbard Sturgis (architect) and Richard Ogden (a decorator), and admired Italian architecture of the sixteenth, seventeenth, and eighteenth centuries, French architecture, and the colonial architecture of Boston.
After brief apprenticeships with Boston architectural firms, Codman started his own practice in Boston, where he kept offices from 1891-1893, after which time he relocated his main practice from Boston to New York City. Codman also opened offices in Newport, Rhode Island as early as 1891, and it was in Newport that he first met novelist Edith Wharton. She became one of his first Newport clients for her home there, Land's End. In her autobiography, A Backward Glance, Wharton wrote:
On October 8 1904, Codman married Leila Griswold Webb, widow of railroad magnate H. Walter Webb. She died in 1910. In 1907, Codman built what was later to be known at the Codman-Davis House in Washington, D.C. for his cousin Martha Codman. It is currently the official residence of the Ambassador of Thailand, and one of the few intact homes that he designed.
Codman's New York clients included John D. Rockefeller, Jr., for whom he designed the famous Rockefeller family mansion of (Kykuit, at the family estate in Westchester County, in 1913; and Frederick William Vanderbilt's (Vanderbilt Mansion National Historic Site in Hyde Park, New York and 459 Fifth Avenue in New York City), as well as another collaboration with Wharton on her townhouse at 882-884 Park Avenue. All told, Codman designed 22 houses to completion, as well as the East Wing of the Metropolitan Club at 1 East 60th Street. He also began the trend of lowering the townhouse entrance door from elevated stairways to the basement level. He designed a series of three houses in Louis XIV style at 7 (his own residence), 12, and 15 East 96th Street from 1912-1916. The Landmarks Preservation Commission later described the facade of number 7 as being "full of gaiety and frivolous vitality" and further, "on approaching the house, Paris and the Champs-Élysées immediately come to mind."
In 1920 Codman left New York to return to France, where he spent the last thirty-one years of his life at the Château de Grégy. Codman died at age 87 in 1951. His architectural drawings and papers are collected at the Avery Architectural and Fine Arts Library at Columbia University; the Codman Family papers are also held by Historic New England and the Boston Athenaeum.