Joel built the house in 1814. It was made from ash, oak, and poplar trees that he chopped down when clearing the property, built in a New England Federal style. In total, it is 2 1/2 stories. The first floor features two parlors and a hall. The second floor has 3 bedrooms. Both these floors have a rear porch that allowed a view of the Ohio River. There is also a basement.
It passed through the family until the final owner, Harriet Scribner, sold it to the local Daughters of the American Revolution, who made sure to keep it as Harriet left it when she died later that year.. In 1977 it was put on the National Register of Historic Places.
It is now open only twice a year, in May and the first Sunday of December. However, all 3rd Grade schoolchildren of Floyd County, Indiana visit it.