Born Patrick Stuart, he was the second son of John Stuart, Lord Mount Stuart, eldest son of John Stuart, 1st Marquess of Bute, son of Prime Minister John Stuart, 3rd Earl of Bute. His mother was Lady Elizabeth Penelope, daughter and heiress of Patrick McDouall-Crichton, 6th Earl of Dumfries, while John Crichton-Stuart, 2nd Marquess of Bute, was his elder brother. His father was killed in a riding accident six months before he was born and his mother died when he was two. In 1817 he was granted the rank of a younger son of a marquess and assumed by Royal license the surname of Crichton. The following year he was returned to Parliament for Cardiff, succeeding his uncle Lord Evelyn Stuart, a seat he held until 1820 and again from 1826 to 1832. Between 1847 and 1859 he also served as Lord-Lieutenant of Buteshire.
Crichton-Stuart married Hannah, daughter of William Tighe, MP, in 1818. Their son James also represented Cardiff in the House of Commons. Crichton-Stuart died in September 1859, aged 65. His wife survived him by thirteen years and died in June 1872.