While her family resided in New York City, where her father was a key man in the Standard Oil Trust, young Millicent Rogers and her sisters and brother had been happy vacation and weekend habitués of the little town from birth, and with their parents, both of whom were themselves Fairhaven natives -- had spent happy, care-free hours there among relatives and friends, in particular communion with both maternal and paternal grandparents.
Millie dearly loved to sketch and read. She is to have once said on a visit to Fairhaven "I wish we had a good library! When she died at such a young age, her grieving family sought an appropriate means of memorializing her short life. Because she had been an avid reader, especially of poetry, the Rogers decided that they would build and donate to the town of Fairhaven a library named for her and given in the names of her sisters and brother.
In the same year she had died, land was acquired and plan were begun to erect in Fairhaven a unique and lavish tribute to the arts, the splendid yet functional Millicent Library. The cornerstone was laid in September, 1891 at quiet morning ceremony with only the family and their clergyman, Rev. J.M. Leighton, in attendance. After prayer, Millie's little brother, Harry, set the cornerstone. Within its confines were a sketch of Millicent, a tracing of the Rogers' ancestry, and a copy of the Fairhaven Star carrying a picture of the proposed building.
The Millicent Library describes the memorial window to the little girl who wished for a good library in Fairhaven as follows: