An ancient town, it is thought to have been a Pict stronghold. The castle, reputedly built under Malcolm III (late 11th cent.), was involved in many wars and was blown up by the Jacobites in 1746. A new castle was built in 1835. Frequent invasions have destroyed most of the town's old buildings. Cromwell's Fort was demolished at the Restoration by Charles II. Inverness, a thriving tourist center, has a museum of Highland relics and hosts an annual Highland Gathering.
One aspect of the town is a concentration of recreational (and some commercial) boating. There is a small public marina, a few private piers, and the Inverness Yacht Club.
The region became the property of James Shafter, who began to develop the property in the 1890s. It became a summer resort where people from San Francisco and Oakland came to camp, hike and swim in Tomales Bay. Many built small summer cabins that still exist today. Small steamboats took day trippers down the bay to secluded beaches. They left from Brock Schreiber's boathouse, which has been preserved and is a prominent local landmark with its prominent sign "Launch for Hire."
In 1995, Inverness Ridge was the site of the Mt. Vision Fire, which burned a large area of Point Reyes National Seashore and a number of homes built on the ridge. The town itself was threatened but was saved by helicopters dipping water from Tomales Bay to drop on the Bishop pine forest between the town and the burning ridgetop.
Inverness is located at (38.092984, -122.848432). According to the United States Census Bureau, the CDP has a total area of 6.4 square miles (16.6 km²), of which, 6.0 square miles (15.5 km²) of it is land and 0.4 square miles (1.1 km²) of it (6.86%) is water.
The town is adjacent to the San Andreas Fault.
Inverness is spread out along approximately of the western shore and valleys of Tomales Bay on the Point Reyes Peninsula, it provides services to visitors to the Point Reyes National Seashore and Tomales Bay State Park.
Other nearby towns include Point Reyes Station, Inverness Park, Olema, and Marshall.
There were 707 households out of which 16.0% had children under the age of 18 living with them, 41.9% were married couples living together, 5.5% had a female householder with no husband present, and 48.4% were non-families. 37.2% of all households were made up of individuals and 10.0% had someone living alone who was 65 years of age or older. The average household size was 1.98 and the average family size was 2.56.
In the CDP the population was spread out with 14.5% under the age of 18, 2.7% from 18 to 24, 20.3% from 25 to 44, 41.8% from 45 to 64, and 20.8% who were 65 years of age or older. The median age was 51 years. For every 100 females there were 90.5 males. For every 100 females age 18 and over, there were 86.9 males.
The median income for a household in the CDP was $47,500, and the median income for a family was $78,398. Males had a median income of $42,448 versus $37,321 for females. The per capita income for the CDP was $37,340. About 5.5% of families and 10.6% of the population were below the poverty line, including 20.7% of those under age 18 and 3.6% of those age 65 or over.