The earliest part of the house dates from the sixteenth century, and was substantially rebuilt around 1620 for Sir Charles Hele. The Hele family held the house until 1716, when the estate passed to the Bulteels. Additions were made to the house in both the early and the late eighteenth century. The house was heavilly remodelled in the Gothic style in 1835 by Mr J C Bulteel, which obliterated the early and late eighteenth century classical work and added castellations.
In 1878 the architect Richard Norman Shaw undertook extensive building works for Henry Bingham Mildmay, remodelling and extending the house, while retaining the sixteenth/seventeenth century house to the south west.
Flete House was used by the City of Plymouth as a maternity hospital during and after the Second World War. At the time the Second World War broke out, the estate was in the hands of Lieutenant-Colonel, the Lord Mildmay of Flete. The first baby to be born at Flete arrived on Monday July 14th 1941, and by the end of 1941 there had been a further 124 births. Lord Mildmay remained in residence at the House until his death in 1947 and was apparently often seen in the Wards. Dave Hill of the pop group Slade was one of the babies born at the House.
The hospital closed 8 May 1958, less than a month after the last baby was born there. Flete House was returned to the Mildmay family in 1959.
More recently, the Hall was owned by the Country Houses Association until it went into liquidation in 2003.