In 1664 Gyldenløve was appointed Statholder (viceroy) to Norway, succeeding the first Statholder, Hannibal Sehested. He studied conditions in Norway very carefully, and became a strong advocate for many important reforms, such as a simplified tax system and the establishment of a Court of appeals in Norway separate from the one in Denmark. He is also remembered for his role in the construction and improvement of nine fortresses along the Swedish border.
In 1671, Gyldenløve founded the city of Laurvig (modern-day Larvik), and was named Count of Laurvig. In Larvik, he is remembered for the construction of its main church as well as Herregården (The Manor), which is still one of the largest wooden structures in Norway.
Count Ulrik Frederik married thrice. His two first marriages ended in divorces. According to contemporary stories, he also had an amorous relationship with count Peder Griffenfeld, who went on to become a leading statesman in Denmark-Norway.
Through his daughter countess Charlotte Amalie af Laurvigen, and her daughter Countess Frederikke Louise Danneskiold-Samsoe, count Ulrik Frederik became ancestor of the Dukes of Augustenborg and thus for example of Frederick VIII, Duke of Schleswig-Holstein.