The collection was started by Charles Feleky as an effort to collect everything published in the English language about Hungary. Feleky was born in Budapest in 1863, but came to the United States in 1885 as a musician. Later, he conducted the music for plays, especially an itinerant production of Ben Hur. Feleky's fascination with Hungary and Hungarian history began after he purchased a book about Louis Kossuth, leader of the Hungarian revolt against the Habsburgs from 1848-49. Over the course of next several years Charles Feleky would develop a network of book dealers in the United States, Canada, England, Ireland, British Africa, India and Australia who would send him copies of books written in English about Hungary and Hungarians. His apartment in New York became his library.
After his death, his wife sold the floor-to-ceiling collection to the Hungarian National Museum, which established the Hungarian Reference Library in New York. By then, the Feleky library covered many academic disciplines. When Hungary declared war on the United States in 1941, the collection was seized by the U.S. Office of Alien Property and put into storage, where it remained until purchased by the Library of Congress in 1953 for $2,000.00.