The Dukedom was a perfect square of 25 leagues on a side, extending towards the west from the mouth of the Río Belén in the Caribbean, in what is today Panamanian territory. As Panama is less than 25 leagues in width at this point, the Duchy extended to the Pacific. By this circumstance, the area of the previous territorial division, Castilla de Oro, was split into two separated parts. The western part, from the Gulf of Nicoya to the border of the Duchy, was united with Veragua Real (Royal Veragua) in 1540 to create the province of Nuevo Cartago y Costa Rica.
The first Duke of Veragua sent out various expeditions to try to enforce his authority throughout the territory, but they all resulted in disasters due to the resistance of the Indigenous and the difficulties of the topography and climate. In one of these expeditions the brother of the Duke, Francisco Colón, died at the hands of the natives.
In 1556 the Duke decided to return the domain to the Crown, in exchange for an increased annual rent of 17,000 ducats (which was paid to his heirs up to 1898) and the retention of the title (which is still retained in Spain). The current Duke of Veragua bears the same name as his illustrious ancestor, Cristóbal Colón.
In 1560 King Philip II created the Province of Veragua from the territory of the Dukedom, under the jurisdiction of the Audiencia of Panama. This corresponds approximately to the present-day Panamanian province of Veraguas.