Elliott was Theodore's best-man on October 27, 1880, on Theodore's first marriage to Alice Roosevelt. When Elliott was 23, he met the beautiful Anna Hall, and they wed quickly, producing three children, Anna Eleanor (1884-1962), Elliott Jr (1889-1893), and Gracie Hall (1891-1941). After this point, Elliott Sr. developed a "casual drinking" problem, which soon became alcoholism. Despite his drunkenness, Eleanor adored her father and he doted on her. Eleanor would always remember her father's tender affection for her and write a book about her father's hunting adventures. While not excusing his conduct, she sensitively tried to highlight his positive virtues.
Due to his drinking problem, Elliott was exiled to Abingdon, Virginia, where he would constantly write letters, mostly to Eleanor. Eleanor later recalled that on his many horseback riding expeditions with the young children in Virginia, he became attached to "one girl in particular whom I was jealous." On occasion, he would, to the jubilation of Eleanor, return back home for a few days. Finally, his drinking ruined him. At the age of 34, Elliott attempted suicide by jumping out a window; he survived the fall, but suffered a seizure and died.
On Franklin and Eleanor's wedding day in 1905, her "Uncle Rob" gave the bride away.
Elliott fathered a child with Katy Mann, a young servant girl employed by Anna. His brother sent a detective who specialized in likenesses to look at the child and subsequently the Roosevelts settled out of court for $10,000. The sum was placed in a trust, but according to the Manns the child never received a dime. It was apparently looted by Katy's lawyers. There was some correspondence between Eleanor Roosevelt and her half-brother Elliott Roosevelt Mann (died 1941) after FDR was elected president in 1932.
Elliot's grandson and namesake, Elliott Roosevelt, the third son of Eleanor and Franklin, became a war hero and author.