The building was constructed shortly after the unification of Italy in 1870, when the Kingdom of Italy captured Rome from the Napoleonic regime, which backed the Papal States. Victor Emmanuel II dismantled the Roman Ghetto and granted the Jews citizenship. The building which had previously housed the ghetto synagogue (a complicated structure housing five scolas in a single building) was demolished, and the Jewish community began making plans for a new and impressive building.
Designed by Vincenzo Costa and Osvaldo Armanni, the synagogue was built from 1901 to 1904 on the banks of the Tiber, overlooking the former ghetto. The eclectic style of the building makes it stand out even in a city known for notable buildings and structures. This attention-grabbing design was a deliberate choice made by the community at the time who wanted the building to be a visible celebration of their freedom and to be seen from many vantage points in the city. The aluminium dome is the only squared dome in the city and makes the building easily identifiable even from a distance. Plates honor the local Jewish victims of Nazi Germany and of a Palestine Liberation Organization attack in 1982.
On April 13, 1986, Pope John Paul II made an unexpected visit to the Great Synagogue. This event marked the first known visit by a pope to a synagogue since the early history of the Roman Catholic Church. He prayed with Rabbi Elio Toaff, the former Chief Rabbi of Rome. This was seen by many as an attempt to improve relations between Catholicism and Judaism and a part of Pope John Paul II's program to improve relations with Jews.
The synagogue celebrated its centenary in 2004. In addition to serving as a house of worship, it is also serves a cultural and organizational center for la Comunità Ebraica di Roma (the Jewish community of Rome). It houses not only the offices of the Chief Rabbi of Rome as well as the Jewish Museum of Rome.
On January 17, 2005, 13 cantors, in conjunction with the Jewish Ministers Cantors Association of America (the Chazzanim Farband), performed in a cantorial concert for the first time in the synagogue's history.