Between 1664 and 1674 a Richard Slaney paid Hearth Tax on a building on the site of the present-day Pitzhanger Manor for 16 hearths. This provides a rough indication of the (considerable) size of the property as it then was.
In 1711, the occupants John and Mary Wilmer gave away their eldest daughter Grizell to be married to Johnathan Gurnell. He went on to make his fortune, first as a merchant and later as a co-founder of the city bank Gurnell, Hoare, and Harman. It was through this marriage that the house then passed to his only surviving son Thomas Gurnell, who bought Pits Hanger Manor Farm (sometimes spelt Pitts Hanger on old maps) in 1765. With the plainer 'manor house' of Pits Hanger (Farm) Manor standing near the centre of the modern Meadvale Road in the present suburb of Pitshanger (often referred to locally as Pitshanger Village), his grander existing house, a mile to the south in Ealing, became known as Pitshanger Place.
Upon the death of Thomas Gurnell, his son Johnathan II inherited the house. On his death in 1791, ownership passed to his young daughter (but was held in trust). The house was let out until 1799, and the trustees decided to sell it.
By the 1790s John Soane had a successful architectural practice in London, holding the post of architect to the Bank of England. In 1794 he, his wife and their two young sons moved into 12 Lincoln's Inn Fields (now part of Sir John Soane's Museum) in central London, an address that doubled as his architecture office and which was therefore also used by his staff.
In early 1800 Soane decided to acquire a family home to the west of London. At first he planned to have it purpose built, but on 21 July 1800 he visited Pitzhanger, which he had heard was available, and seeing its potential offered the trustees £4,500 for the whole estate of 28 acres. This was accepted on the first day of the following month. Soane referred to it as Pitzhanger Manor-house.
Soane worked vigorously on the designs of the new house, and over a hundred designs for it still exist and are held by Sir John Soane's Museum. He planned for the demolition of the older part of the house and many of the outbuildings; however, he retained the two-storey south wing designed by George Dance in part because of admiration for their interiors and in part in respect for Dance, his first employer. Demolition work started in 1800 and most of the rebuilding was complete by late 1803.
Completed in 1804, the central section of the house uses many typical Soane features: curved ceilings, inset mirrors, false doors, and wooden paneling with many cupboards. Soane continued the building to the east with a servants' wing (perhaps an adaptation of existing buildings) and romantic ruins. (All the buildings in this eastern part of the site were demolished in or around 1901.) The building is remarkably similar to his main London home at Lincoln's Inn Fields (now the Soane Museum). Much of his collection of paintings and classical antiquities now at the museum was housed in Pitzhanger Manor.
Soane sold the house in 1810 and it then passed through several hands until in 1843 it became home to the daughters of Britain's only assassinated Prime Minister, Spencer Perceval.
Since Soane's time, the house has been referred to variously as The Manor, or Pitshanger Manor, but has now formally reverted to the name given to it by Soane, spelt with a Z.
In 1900 the house was acquired by Ealing Urban District Council in the year before it became a Municipal Borough for a total of £40,000 pounds, a quarter of which came from the Middlesex County Council. Its new function was to serve as a Free Public Library. However, work on converting the building did not start until after the death of its last resident, Frederika Perceval in May 1901. The important part of the work was to build a ground-floor extension with a pitched slate roof, on the west of the 'Eating Room'. However, this magnificent room was all that remained of George Dance's original design. Therefore the Council had its chief surveyor Charles Jones design the extension next to the existing Breakfast Room. As Dance had given the room windows with a tall aspect, topped by semicircular bonded gauge brick arches, it made sense to remove the glazing and frames so as to open them up. This provided three large arched pedestrian openings into the newly created extension. To avoid a clash of architectural styles, Jones specified that the new extension be an almost mirror image of its neighbour, which is clearly visible through the connecting arches. With the high ceilings and matching plaster moulding and colour scheme, this helps create the illusion of one elegant, pleasantly proportioned and spacious reading area. Finally, as access from this part of the building to the main library stock and issuing area required going up and down many steps which exists along the passage way (which winds northward through the house), a new entrance was built out on the east-side of the breakfast room with 'Reading Room' emblazoned across its Portland stone lintel.
On the north side of the house: Jones had the servants' quarters demolished and removed some ornamental faux Roman ruins. The building to house the new lending library was constructed on the space so cleared. To complement the rest of the house it had the same arched windows. The lintel of the Portland stone surround of the portico was inscribed 'Lending Library'. It was opened to the public in April 1902. In 1938-40 the lending library block was replaced by a new, slightly larger building.
The Library moved out in 1984 and in 1985 the restoration work began. Analysis of the structure and paint layers were used to recreate an authentic period look to the build. It open to the public once again in January 1987 as the London Borough of Ealing's main museum and the PM Gallery & House, showing contemporary art. The house is a Grade I listed building.