The Holy Sepulchre Cemetery originated in 1871 by order of Bishop Bernard J. McQuaid, who sought to bury Rochester's Roman Catholic population in a single location regardless of ethnic origin. Bishop McQuaid undertook this project as part of his wider effort to bring together the various Catholic groups in the newly formed Diocese of Rochester and to preserve Church traditions in the community.
Before the diocese founded the Holy Sepulchre Cemetery, four German Catholic parishes opened cemeteries of their own, and an Irish parish opened two cemeteries before them. These separate graveyards were not large enough for Rochester's entire Catholic population, so Bishop McQuaid purchased a 110-acre tract of land near downtown Rochester to serve as a common cemetery for all Catholics in the diocese. He consecrated it as the Holy Sepulchre Cemetery. From its founding to the 1950s, about 5,000 Catholics buried in other graveyards in the city were reinterred in the Holy Sepulchre Cemetery.
Bishop McQuaid consulted with a number of landscapers and horticulturists to give the burial ground the aesthetic appeal of a garden. The cemetery's numerous tree and plant species resulted from this original idea, with later specialists and administrators working to maintain and expand the greenery.