At this time the Holy See was engaged in a conflict with Manfred, the illegitimate son and designated heir of Frederick II of Hohenstaufen, but whom papal loyalists, the Guelfs, called "the usurper of Naples". Clement IV, who was in France at the time of his election, was compelled to enter Italy in disguise. He immediately took steps to ally himself with Charles of Anjou, his erstwhile patron's brother, the impecunious French claimant to the Neapolitan throne. Charles allowed the Pope to be his feudal overlord (a bone of contention with the Hohenstaufen) and was crowned by cardinals in Rome, where Clement IV, permanently established at Viterbo, dared not venture, the Ghibelline party was so firmly in control. Then, fortified with papal money and supplies, Charles marched into Naples. "Papal legates and mendicant friars appeared upon the scene, preaching a formal crusade, with the amplest indulgences and most lavish promises" (Catholic Encyclopedia). Among the Italians who failed to see any nobler crusading purpose in the conflict was Dante (Inferno, Canto xxvii). Having defeated and slain Manfred in the great Battle of Benevento, Charles established himself firmly in the kingdom by the conclusive Battle of Tagliacozzo, in which Conradin, the last of the house of Hohenstaufen, was taken prisoner. Clement IV is said to have disapproved of the cruelties committed by his protegé, and there seems no foundation for the statement by Gregorovius that Clement IV became an accomplice by refusing to intercede for the unfortunate Conradin whom Charles had beheaded in the marketplace of Naples.
Within months Clement IV was dead too, and buried at Viterbo. Owing to unbridgeable divisions among the cardinals, the papal throne remained vacant for nearly three years.
Clement IV's private character was praised by contemporaries for his asceticism, and he is especially commended for his indisposition to promote and enrich his own relatives. He also ordered the Franciscan scholar Roger Bacon to write his Opus maius, which is addressed to him. He was buried at Viterbo, where he had resided throughout his pontificate.
In 1264, Clement IV, assigns Talmud censorship committee. He ordered that the Jews of Aragon to submit their books to Dominican censors for expurgation.
In his 1267 letter written from Viterbo, the Pope wrote:
Jayme Alaric would return to Europe in 1269 with a Mongol embassy, again proposing an alliance. Pope Clement welcomed Abaqa's proposal in a non-committal manner, but did inform him of an upcoming Crusade.