Some information about his childhood and upbringing is preserved in the collection of his father's Hebrew poetry, which Joseph writes that he began copying at the age of eight and a half. For example, he tells how once (aged nine and a half, in the spring of 1045) he accompanied his father to battlefield, only to suffer from severe homesickness, about which he wrote a short poem.
His primary teacher was his father. On the basis of a letter to Rabbi Nissim Gaon attributed to him, in which Joseph refers to himself as R' Nissim's disciple, some claim that he also studied under R' Nissim at Kairwan. Joseph later married R' Nissim's daughter.
On R' Shmuel's death, Joseph succeeded him as vizier and rabbi, directing at the same time an important yeshiva. Among his students were Rabbi Isaac ben Baruch ibn Albalia and Rabbi Isaac ibn Ghayyat.
Rabbi Abraham ibn Daud describes Joseph in highly laudatory terms, saying of him that he lacked none of his father's good qualities, except that he was not quite as humble, having been brought up in luxury.
The 1906 edition of the Jewish Encyclopedia states that "Arabic chroniclers strangely relate that he believed neither in the faith of his fathers nor in any other faith. It may also be doubted that he openly declared the principles of Islam to be absurd. Arabic poets also praised his liberality.
Further, the Jewish Encyclopedia reports that Joseph "controlled" the King and "surrounded him with spies." He was also accused of several acts of violence, which drew upon him the hatred of the Berbers, who were the ruling majority at Granada. The most bitter among his many enemies was Abu Ishak of Elvira, a fanatical Arabic poet who hoped to obtain an office at court and wrote a malicious poem against Joseph and his coreligionists. This poem made little impression upon the king, who trusted Joseph implicitly; but it created a great sensation among the Berbers. They spread a rumor to the effect that Joseph intended to kill Badis, deliver the realm into the hands of Al-Mutasim of Almería with whom the king was at war, then to kill Al-Mutasim and seize the throne himself.
Other sources report that Joseph attempted to ease the tension between the Berbers and the Arab population and prevent excesses against the local Arabs, which led to a civil war.
On December 30, 1066 (9 Tevet 4827), Muslim mobs stormed the royal palace where Joseph had sought refuge, then crucified him. In the ensuing massacre of the Jewish population, most of the Jews of Granada were murdered. "More than 1,500 Jewish families, numbering 4,000 persons, fell in one day.
Joseph's wife fled to Lucena with her son Azariah, where she was supported by the community. Azariah died in early youth.