The name Gimzo is thought to derive from the fruit of the sycamore tree known as "Gomez" which was abundant in this area, based on the biblical verse Chronicles II 1:15; "The king made silver and gold as plentiful in Jerusalem as stones, and he made cedars as plentiful as sycamores in the lowland." In the period of the second Temple, the great sage Rabbi Shimon ben Gamliel says: "A sign of mountains is "milin", a sign of valleys is palm trees, a sign of rivers is cane, and a sign of the plains is Sycamore trees, and whereas there is no proof of this, we remember the words: And he made cedars as plentiful as sycamores in the lowland" (Tosephta Shviit:87:6). Moshav Gimzo was the home of the sage (the Ta'ana) Nachum (Rabbi Akiva's teacher), who was wont to say "It is all for the best" which translates to "Gam-zo le-tova", a word play on the name of his home at "Gam-zo".
In 1917, the British, under the command of General Edmund Allenby, took control of Palestine. Jimzu was cited as the rendezvous point for the British 52-nd division, for its advance to Jerusalem through the Beit Horon Pass.
During the 1948 Arab-Israeli War, "Operation Dani" was planned to free Lod, Ramla, Latrun, and Ramallah, and to release the pressure around Jerusalem. Plans for this operation mention Jimzu, and, on 10 July 1948, the Yiftah Brigade captured the settlements of Anabe, Jimzu, Daniel and Dahariya. The soldiers of the Arab Legion tried unsuccessfully to retake Jamzo.
The moshav was founded by a group of immigrants from Hungary on 28 February 1950. The founders were a group of Satmar Chassidim called Etz Chaim ("tree of life"), affiliated with the Agudat Israel Workers party.
The moshav has grown substantially, having and absorbed new families and has built a new neighborhood with housing for Gimzo's newer generation. Today the moshav consists of 140 families with over 700 residents, including ultra-orthodox and orthodox Jewish residents of both Sephardic and Ashkenazi backgrounds who lead a "religious-Zionist lifestyle". The spiritual leader of the community as well as the Hevel Modi'in Regional Council is Rabbi Eliav Meir.