A later owner built a large main wing in the Federal style; the only remnant of the original house is the small kitchen wing. It has since become the property of the Historical Society of Quaker Hill and Pawling, which uses the house as its headquarters and to display exhibits related to local history, particularly the life of pioneering radio broadcaster and executive Lowell Thomas, who lived near Pawling for the later years of his life. It was added to the National Register of Historic Places in 1980.
The oldest part of the house is the eastern extension known today as the kitchen wing, a one-and-a-half-story frame structure on a stone foundation. It has a low-pitched gabled shingled roof, pierced by a brick chimney at the gable end. The five-bay south-facing front elevation appears as one story due to the windows and colonnade added later. Earlier stone and frame additions have since been removed. An original rendering vat is in the basement.
The main block, added later, is a five-by-three-bay two-story frame house lined with brick. Its most distinctive Federal style feature is the main doorway, flanked by sidelights, fluted pilasters and topped with a rectangular transom window. Smaller versions of the pilasters flank the Palladian window immediately above the doorway. All the other windows are rectangular symmetrical and topped with projecting cornices. A columned Greek Revival portico runs the length of the first story.
On the west gable there is a Palladian opening with grilled quarter-round openings on either side and topped with a pediment. The rear entrance uses a Dutch door and lacks the transom but is otherwise identical to the front. Three chimneys rise from the main block's roof, two in the west and one in the east near the join with the kitchen wing.
Inside, the two wings present a contrast. The kitchen wing has little decoration beyond the carved wooden fireplace mantel. The main block is more lavishly decorated, with marble mantels and casings on two of its four fireplaces. All the windows and doors are trimmed in carved wood; the rooms also have similarly carved wainscoting and ceiling cornices.
The upper story, mainly given over to bedrooms, also has several fireplaces, all with similarly detailed and painted wooden casings and mantels. The attic is unfinished.
The property was first settled in the late 1730s by William Prendergast, a tenant farmer who leased southeast of the nascent settlement of Pawling from the Philipse family, the area's dominant landowners. He built the small house that became the kitchen wing in 1740, adding other buildings later and connecting the two through a stone passageway.
In 1766 he became a leader in the tenant uprising known as the Dutchess County Anti-Rent War, a revolt against the quit-rents left over from the feudal system left in place by Dutch colonists in the region which made it hard for tenants to eventually purchase their land. It was quelled by troops called in from Poughkeepsie. Prendergast was tried, convicted and sentenced to hang for treason. His wife made a personal appeal to the colonial governor, Sir Harry Moore and he was spared at the last minute by a gubernatorial reprieve and later a royal pardon. The Prendergast family left for Chautauqua County shortly thereafter as it was outgrowing the house.
Kane, an Irish immigrant, bought the house later that year. When the Revolution started, he was initially on the Patriot side, as he had been elected to the Provisional Congress of New York in 1775. After the first year of the war, he switched sides and became a Loyalist, convinced the cause was hopeless. Accordingly, his house and property were confiscated by the New York State Legislature, and the following year, in September 1778, George Washington moved in when the Continental Army wintered in the area, where they could move on either New England or New York City at short notice. Kane retreated to the safety of British lines for the remainder of the war, while his family went to Nova Scotia. He received a lifelong pension from the British in 1783, when the war ended, and returned to the Pawling area. Since he could not legally live in or repurchase his home, he lived the remainder of his life with his children.
A later owner in the early 19th century demolished all but the original kitchen wing and built the current Federal structure. It went through a variety of later uses, as an inn and a rental property owned by the local bank, but returned to single-family dwelling status late in the century. In 1946 one owner added electricity; it is not known when modern heating and plumbing were installed. In the late 20th century, after it had been listed on the National Register, it was acquired by the Historical Society of Quaker Hill and Pawling and converted to its present use.