The reason locals call them the Pelhams is that they all once belonged to Thomas Pell. The Village of Pelham, Pelham Manor, Pellville, North Pelham, Pelhamwood, Pelham Gardens, Bartow-Pell Mansion and Pelham Bay, Bronx are named for Pell. Thomas Pell bought the area and some islands from the Siwanoy Indians.
Bartow-Pell Mansion is the plantation mansion that was owned by Pell and his descendants. It is a Greek Revival Mansion.
There is some dispute amongst locals who claim that there are two origins of the name Pelham. One origin for the name says that Thomas Pell named Pelham after his tutor, named Pelham Burton. Another taught in Bronx schools says that Pelham really means Pell's Town in the English style, with the word "Ham" meaning village or town.