Four years later, Muir’s “Hunter’s Hill” farm of roughly , was granted to a man named Robert Ryan for his services in the Marines and in the New South Wales Corps. By 1801, the property had passed into the hands of Robert Campbell, an esteemed Sydney merchant. Campbell built Australia’s first shipbuilding yards in 1807, at the site that is now the Royal Sydney Yacht Squadron, Kirribilli.
Campbell’s property in Kirribilli was used for grazing under lease to Campbell’s friend James Milson, hence the name “Milson's Point”. Milson's Point is the next point along from Kirribilli point, where the Sydney Harbour Bridge crosses the harbour. In 1842, the five-acre site where Admiralty House now sits was leased to the Collector of Customs for the Colony, Lieutenant-Colonel (later full Colonel) John George Nathaniel Gibbes, MLC. Colonel Gibbes (1787-1873) intended to build a private home on the site. (Since his arrival in the colony in 1834, Gibbes and his family had been living in Henrietta Villa, also known as the Naval Villa, on Sydney's Point Piper, under a leasehold arrangement.) On the superb Kirribilli Point location, Gibbes erected, between 1842 and 1843, a graceful single-storey house with wide verandahs and elegant French doors. Gibbes designed the house, which he called "Wotonga" (or "Woottonga"), himself. The stone for the house's walls was quarried locally and the hardwood and cedar joinery came from George Coleson's timber-yard in George Street, Sydney. Gibbes engaged James Hume, a well-known builder who dabbled in ecclesiastical architecture, to supervise the construction of the building and its stables. Gibbes, however, hired his own masons, bricklayers, carpenters, plumbers and ironmongers to work on the project, paying each of them separately as work progressed. Gibbes used the Custom Department's cutter to commute to and from the building site. Once completed, Gibbes' L-shaped residence featured a plain, yet stylish, double facade to maximise the building's magnificent, sweeping views across Sydney Harbour. These views enabled Gibbes to monitor shipping traffic in and out of Darling Harbour and, more importantly, Circular Quay, where the Sydney Customs House was situated. Today, Wotonga forms the core of Admiralty House and the building's 180-degree, east-west panoramic sight-lines are even more spectacular than they were in Gibbes' day, owing to the subsequent high-rise growth of Sydney's CBD.
Colonel Gibbes, incidentally, was said to be the illegitimate child of His Royal Highness Frederick, Duke of York, (King George III's second son). This reputed connection to the British monarchy adds spice to the house on Kirribilli Point's subsequent role as a vice-regal establishment. For more information about Wotonga's construction, see the North Shore Historical Journal article "John Gibbes: Builder of Admiralty House", in Volume 37, Number 1, April 1997, pages 8-12.
A small portion of the Kirribilli Point land, a little over an acre was sold in 1854 to a merchant, Adolph Frederic Feez. On this land, Kirribilli House was built. Kirribilli House, next door to Admiralty House, now serves as the official Sydney residence of the Australian Prime Minister.
In April 1874, Wotonga House was auctioned and bought for 10,100 pounds by Mr Thomas Cadell, a Sydney merchant and member of the New South Wales Legislative Council from 1881 to 1896. At that time, the house was described as containing a wide verandah, a spacious entrance hall, drawing and dining rooms, 10 bedrooms and the usual rooms in the main part of the house, as well as having a large courtyard, servant's rooms, kitchen, stables, etc, with an abundant water supply, which never failed in the driest weather.
In 1913, this part of the history of Admiralty House came to a close as the last British Admiral left the house as the Royal Australian Navy took over responsibility of the Naval Defence of Australia.
With the departure of the last British Admiral from Admiralty House the following year, the Admiralty handed the house back to the New South Wales Government. This provided Lord Denman’s successor, Sir Ronald Munro Ferguson, with a Sydney residence. Admiralty House was the residence of the Governors-General for the following fifteen years.
In 1930, during the Great Depression, the Scullin Government had Admiralty House closed, and its contents were sold at Auction in 1931.
Governor-General Sir Isaac Isaacs, appointed in 1931, was the first Governor-General to live permanently at Yarralumla, in Canberra. During his term, Admiralty House remained empty and neglected. Sir Isaac described it in 1934 as being “stripped of its glamour, with no furnishings but a few fine mirrors, its garden wild and overgrown”. In 1936, the State of New South Wales reopened Admiralty House as the Sydney residence for the new Governor-General, Lord Gowrie. The house has been used ever since as a vice-regal establishment.
Formal title to Admiralty House finally passed from the State Government to the Commonwealth by Crown grant in 1948, on the condition that the house was to be used only as a residence for the Governor-General.
Admiralty House is at present the official residence of the Governor-General of Australia, and of important overseas visitors. The Royal Family and other important dignitaries, such as the American President and the Pope, visit Admiralty House when they are in Sydney.
The house is furnished with colonial furniture, porcelain and numerous historical artworks such as portraits of Captain James Cook and former Governors General including Hallam Tennyson, 2nd Baron Tennyson. Many were acquired for the nation by The Australiana Fund.