Abdel-Rahim was born in Cairo to a musical father, and began playing the piano at an early age. His early musical studies were supported by the Music Society of the Faculty of Arts of Cairo University (then called Fuad I University), graduating with a degree in history. In 1950 he began university studies in musicology at the Musikhochschule of Heidelberg in West Germany, deciding on a career as a composer. From 1952 to 1957 he studied composition with Harald Genzmer (a pupil of Paul Hindemith) at the Musikhochschule of Freiburg in Breisgau.
In 1959, Abdel-Rahim was appointed to teach theory and harmony at the newly opened Cairo Conservatory of Music. He was later appointed head of the composition department there (the first of its kind in the Arab world), which he founded in 1971. Abdel-Rahim was quite influential among Egyptian composers of the next generation, as the majority of them studied with him during his time at the Cairo Conservatory.
Abdel-Rahim's style fuses traditional Egyptian musical elements with contemporary European ones, focusing on Egyptian materials in his later works more than in his earlier ones. He composed many works for orchestra, chorus, and chamber ensembles, as well as songs and music for film, theater, and ballet. He was awarded the State Prize for Composition, as well as the Order of Arts, from the Egyptian government.