The town was the location of the first recorded Christian church in Scotland, Candida Casa the 'White [or 'Shining'] House', built by Saint Ninian (original form Nynia) about 397. A monastery and diocese of the Anglo-Saxon kingdom of Northumbria was founded on the site in the 8th century. It was the centre of the revived See of Galloway (or Candida Casa) under the patronage of Fergus, Lord of Galloway and Bishop Gille Aldan from the 12th century. The late medieval cathedral priory is ruinous, much of it having disappeared completely apart from the much-altered aisleless nave and vaults at the former eastern end which once held the shrine of St. Ninian, one of medieval Scotland's major pilgrimage destinations. A museum in the town contains finds from the site, which has been extensively excavated in recent years. A late medieval gateway with the arms of the King of Scots leads into the site of the priory, which contains the 19th century parish church and a museum of carved stones (Historic Scotland). The collection of early medieval stones is one of the largest in Scotland, and includes the country's earliest surviving Christian memorial, the 5th century inscribed 'Latinus Stone'. The museum layout and display was revised and greatly improved in 2005.
Whithorn was first known (in Latin) as Candida Casa. 'Whithorn' is a modern form of the Anglo-Saxon version (actually a literal translation) of this name, Hwit Ærne, 'white house'. In Gallovidian Gaelic, it was called Rosnat, or Futarna, the latter a version of the Anglo-Saxon name (Gaelic has no sound corresponding to English 'wh').
Whithorn's link to the sea was the port known as the Isle of Whithorn (a separate community from Whithorn itself and actually a peninsula). Much used in the Middle Ages by pilgrims arriving by boat. The thirteenth century St Ninian's Chapel marked the point where pilgrims came ashore (the roofless remains are looked after by Historic Scotland).