In 1864 he received his medical degree from the University of Heidelberg, and for several years was an assistant to pathologist Nikolaus Friedreich (1825-1882) at Heidelberg. As a young man, he also worked for a period of time under Ludwig von Buhl (1816-1880) in Munich. In 1880 Erb attained the chair of special pathology at the University of Leipzig, where he also became head of its policlinic. Afterwards, he returned to Heidelberg, where he succeeded his former teacher Nikolaus Friedreich. He worked in Heidelberg for the remainder of his career.
Erb began his medical career in the fields of toxicology and histology, but later his interest switched to neurology, where he became one of the 19th century's leading neurologists. He extensively used electrodiagnostic testing and demonstrated motor nerve irritability in tetanus. Erb is also credited with popularizing the reflex hammer for use in neurological examinations.
He made early observations relating to syphilis and tabes dorsalis (nerve fiber and nerve cell degeneration). In his research of tabes dorsalis he tried to find the link between this condition and syphilis. He also contributed research material concerning poliomyelitis, claudication intermittens, and progressive muscular atrophy.
He wrote more than 250 medical essays including Handbuch über die Elektrotherapie, and an important study on spinal paralysis. In 1878, Erb described myasthenia gravis, which is sometimes referred to as the Erb-Goldflam disease (named along with neurologist Samuel Goldflam). Myasthenia gravis is a neuromuscular disorder that leads to fatigue and muscle weakness. Neurologists Ernst Julius Remak (1849-1911) and Max Nonne (1861-1959) were two of his students.
Several eponyms are named after him including: