The abbey was originally built in the 7th century by Saint Germer, its founder and first abbot, as a Benedictine abbey. It was destroyed twice during the invasions of the Normans in the 9th and early 10th centuries, and re-founded as a Cistercian abbey in the 12th century by the Bishop of Beauvais. The relics of Saint Germer were moved from Beauvais Cathedral to the newly reconstructed abbey church, which still stands.
In the 13th century, Pierre de Wessencourt, the 25th abbot, built the Sainte-Chapelle at the chevet of the abbey church, which closely resembled the Sainte-Chapelle in Paris, built twelve years earlier. It is connected to the main church by a narrow passage, and is composed of three vaults with stained glass window-walls. It was used as a funerary chapel.
The abbey suffered greatly during the Hundred Years' War, losing its western front and six vaults. In 1414 the troops of the Duke of Burgundy destroyed the towers and one bay of the nave. The present west front was rebuilt in the 16th century.
In 1644 the abbey was transferred to the reformed Benedictine Congregation of St. Maur, who continued the rebuilding.
Like all other monasteries of France, Saint-Germer-de-Fly was confiscated as State property during the French Revolution and the abbey buildings were destroyed. After the Revolution, the surviving church became the town's parish church. The northern transept, weakened by the removal of tributary structures, was rebuilt in 1808.
The abbey forms a Latin cross, terminating at the head in a semi-circle. The façade is quite plain, in contrast with the rest of the church. The Sainte-Chapelle is a delicate work of Rayonnant Gothic architecture.