Gräbe studied in Frankfurt, Karlsruhe and Heidelberg. Later he worked for the chemical company Meister Lucius und Brüning (today Hoechst AG). He supervised the production of Fuchsine and researched violet colorants made of Iodine. The work with Iodine gave Gräbe an eye infection, so he returned to academia.
Under the supervision of Robert Wilhelm Bunsen, Carl Gräbe received his PhD from the University of Heidelberg in 1862. In 1868 he wrote his habilitation, and became a professor in Leipzig. Gräbe was Professor of Chemistry at the University of Königsberg from 1870 until 1877, and at the University of Geneva from 1878 until 1906.
He also introduced the "ortho", "meta" and "para" nomenclature on benzene rings.