The "castle" in the village is a single pele tower that was built by Robert de Shirburn in the 14th century. It is within Castle Farm but can be easily accessed. The castle may have been built by the Shirburn family during the time of Owain Glyndŵr. A French army landed at Angle in 1405 to assist Glyndŵr . Some sources see this as a tower but others see evidence of a moat and another tower and see this ruin as the remains of a castle.
In the nineteenth century it was reported that 388 people lived in the village with the women involved in plaiting straw for bonnets and mats, whilst the men would trawl for oysters when they were in season.
In the same century a large number of forts were constructed around Pembroke Dock and Milford Haven. Three of these are on the coast around Angle, the East Blockhouse Battery, Thorn Island Fort and the Chapel Bay Battery. Their construction was funded as part of advice to Lord Palmerston following a Royal Commission.
The church itself is thought to have been built in the thirteenth century with the tower added in the fifteenth century. The church's grounds also include a number of graves for a Japanese ship that sank locally during the First World War.
The first rescue where the crew received silver medals was in the rescue of 27 (some say 33) people who were on board the 1878-built Loch Shiel which had ran into rocks off Thorn Island. Two lifeboat crew members and the honorary secretary received silver medals. It was said that the lifeboat was unable to reach them but these brave people managed to reach them.
The rescue is particularly noteworthy as it is described as Wales' "Whisky Galore". The Loch Shiel was carrying goods from Scotland to Adelaide and included gunpowder, beer and 7,500 (some say 7,000) cases of Glasgow whisky. Much of this was never recovered. Some of the bottles are still amongst the wreck which are described as "undrinkable", but much of the cargo was only partially recovered by the customs men. It was said that one local drank himself to death on the 100 proof whiskey. In 1999, bottles of beer from the wreck were auctioned for £1000 per bottle.
The next award was a bronze medal awarded to Coxswain James Watkins for rescuing 28 people on the 26 November 1929 from the single-screw steamship Molesley which had been caught by a sudden wind change and a poor decision by its captain. James Watkins went on to be awarded both a silver medal for rescuing 6 people in 1944 from the motor boat Thor and a year later another bronze medal for a difficult rescue of nine people from the steamer Walter L M Russ. (This steamer had been seized from the Germans and sank on the 15 July before it could be renamed the Empire Concourse.)
More recently, Coxswain William John Rees Holmes has been awarded two bronze medals. The first was in 1977 when the tanker Donna Marike was thought to be about to explode and the lifeboat stood by her in December 1976. The second bronze medal was for rescuing three people from the fishing boat Cairnsmore on 1 December 1978.
In 1997 a third coxswain, Jeremy R. Rees, and his crew were awarded another bronze medal for rescuing four people after their motor boat, Dale Princess, was blown onto cliffs on Skomer Island. The rescue was made in gale force winds and stormy seas.