In 939, Muhammad became ill and Abu 'Ali was assigned to succeed him in his posts. As governor of Khurasan he undertook several campaigns in northern Iran, conquering Gurgan and territory as far west as Ray from the Ziyarids, forcing the Ziyarids to acknowledge Samanid authority, and killing the Dailamite adventurer Makan b. Kaki. After he departed for Khurasan the Buyid Rukn al-Daula seized Ray. In 945 Abu 'Ali returned and expelled the Buyids from Ray, but they returned a year later.
That same year, 945, the Samanid amir Nuh b. Nasr dismissed Abu 'Ali from the governorship of Khurasan after hearing complaints of the latter's harsh rule, and sought to replace him with a Turk, the Simjurid Ibrahim b. Simjur. Abu 'Ali refused to accept his dismissal and rebelled. He convinced a Samanid, Nuh's uncle Ibrahim b. Ahamd, to come from Iraq and installed him as amir in Bukhara when he took the city in 947. Ibrahim, however, was unpopular with the people of Bukhara, and Nuh soon retaliated by retaking the city and blinding Ibrahim and two brothers. He was unable, however, to defeat 'Abu Ali, who had recruited the support of other Samanid vassals, such as the rulers of Khuttal, and the Kumiji mountain people, and in the end made peace with him, allowing him to keep Chaghaniyan.
In 952 following the death of the governor of Khurasan, Mansur b. Qara-Tegin (Ibrahim b. Simjur had already died in 948), 'Abu Ali was reinstated by Nuh as governor of the province. At the insistence of the Ziyarids he began a war with the Buyid Rukn al-Daula, but was unable to take Ray and made peace. This action upset Nuh, who again removed him from the governorship. 'Abu Ali fled to the Buyids and received an investiture diploma for Khurasan from the Caliph, but was unable to enforce his claim or even return to Chaghaniyan. In 955 he and one of his sons died of the plague in Ray.