Rauch was born at Arolsen in the Principality of Waldeck. His parents were poor and unable to place him under efficient masters. His first instructor taught him little else than the art of sculpting gravestones, and Professor Ruhl of Kassel could not give him much more. A wider field of improvement opened up before him when he removed to Berlin in 1797; but he was obliged to earn a livelihood by becoming a royal lackey, and to practise his art in spare hours. Queen Louisa of Prussia, surprising him one day in the act of modeling her features in wax, sent him to study at the Academy of Art.
Not long afterwards, in 1804, Count Sandrecky gave Rauch the means to complete his education at Rome, where Wilhelm von Humboldt, Antonio Canova and Bertel Thorvaldsen befriended him. Among other works, he executed bas-reliefs of "Hippolytus and Phaedra," "Mars and Venus wounded by Diomede," and a "Child praying."
In 1811 Rauch was commissioned to execute a monument for Queen Louisa of Prussia. The statue, representing the queen in a sleeping posture, was placed in a mausoleum in the grounds of Charlottenburg, and procured great fame for the artist. The erection of nearly all public statues came to be entrusted to him. There were, among others, Bülow, Yorck, and Scharnhorst at Berlin, Blücher at Breslau, Maximilian at Munich, Francke at Halle, Dürer at Nuremberg, Luther at Wittenberg, and Grand Duke Paul Friedrich at Schwerin.
At length, in 1830, Rauch began, along with the architect Karl Friedrich Schinkel, the models for a colossal equestrian monument at Berlin to honor King Frederick II of Prussia (Frederick the Great). This work was inaugurated with great pomp in May 1851, and is regarded as one of the masterpieces of modern sculpture. Princes decorated Rauch with honors and the academies of Europe enrolled him among their members. A statue of Immanuel Kant for Königsberg and a statue of Albrecht Thaer for Berlin occupied his attention during some of his last years; and he had just finished a model of Moses praying between Aaron and Hur when he was attacked by his last illness.