Caro was born in Gnesen (Gniezno), Grand Duchy of Posen, the son of Joseph Chayyim Caro. After several years of diligent study at the universities of Berlin and Leipzig, he attracted considerable attention by his work Das Interregnum Polens im Jahr 1586, oder die Häuser Zborowski und Zamojski (Gotha, 1861) and was immediately entrusted with the continuation of Röppel's history of Poland in the series of Geschichten der Europäischen Staaten, edited by Arnold Hermann Ludwig Heeren and Friedrich August Ukert, and published at Gotha. Caro contributed volumes ii through v (1863-88) of this monumental work.
Before publishing the results of his research he undertook several extensive journeys through Galicia and the south of Russia, and upon his return to Germany in 1863 was appointed privat-docent at the University of Jena. Shortly afterward, at the invitation of Grand Duchess Helena of Russia, he accompanied her on her travels, and was for some time attached to her suite at St. Petersburg. Later he was promoted to the position of assistant professor at the University of Jena; and in 1869 was called by the University of Breslau to fill a special chair of history. From 1882 he occupied the position of professor at that institution. He died in Breslau (Wrocław).
His reputation is based chiefly on his researches in the history of Poland. Among his works, besides those already mentioned, are: