Panah Ali's paternal great grandfather and namesake Panah Ali bey served at the court of Ganja beylerbeys (governors) in the early 17th century, at the time when the region's was directly controlled by the Safavid Empire. He soon retired, married a woman from the Javanshir clan of Karabakh and had a son by the name of Ali (nicknamed Sarija Ali). They lived in their estate located in Arasbar (present-day Khojavend and Agdam rayons of Azerbaijan) but also owned land in Tartar and the northern shores of the Aras River. The Arasbar estate was rebuilt into a castle in Sarija Ali's son Ibrahim Khalil's lifetime and has been known as Ibrahim Khalil Galasi since.
After the dethronement of the Safavids in 1736 by Nader Shah, the landed classes of Ganja and Karabakh gathered in Mugan (the Javanshirs were also among them) deciding to oppose the usurper's reign and agreeing on trying to get the Safavids back on the throne. When this news reached Nadeh Shah, he ordered all Muslim landowners of the region and their families deported to Khorasan (northeastern Iran) as a punishment. Thus the future khan Panah Ali happened to be among the deportees.
In 1747, Panah Ali, by then already a successful naib and royal gérant de maison, found himself displeased with Nader Shah's attitude towards him and having gathered many of those deported from Karabakh in 1736, returned to his homeland. Due to his reputation as a skillful warrior and his wealthy ancestor's legacy in Karabakh, Panah Ali proclaimed himself and was soon recognized throughout most of the region as a ruler (khan). The shah sent troops to bring back the runaway however the order was never fulfilled: Nader Shah himself was killed in Khorasan in June of the same year. The new ruler of Persia, Adil Shah issued a firman (decree) recognizing Panah Ali as the Khan of Karabakh.
Strengthening of Panah Ali khan's power faced resistance from other khans. The struggle between the Karabakh khan and Haji Chelebi Khan of Shaki, one of the most powerful feudal rulers of the South Caucasus, started in 1748. Haji Chalabi Khan wishing Panah Ali khan's power not increased further, allied with the Khan of Shirvan and surrounded the castle of Bayat. The allies for the whole month unsuccessfully tried to capture the capital of the Karabakh khanate. The Shaki and Shirvan khans withdrew, incurring huge casualties and failing to accomplish the mission. Haji Chelebi Khan was forced to admit: "Until now Panah Khan was raw silver that was not minted. We came, minted it, and returned." Same quote from another Karabakh historian of 19th century, Mirza Yusif, reads: "Until now Panah Khan was merely gold, we came and minted a coin from that gold.