A person's 'weak point' is any characteristic that can be exploited by another for some purpose. This weakness does not just have to be physical; it can be psychological, moral or situational.
In "The Art of War" by Sun Tzu, much is talked about of finding and exploiting an opponent's weaknesses to achieve victory. This principle equally applies outside of warfare and is useful in any conflict or interaction.
Physical weakness is a common trait people focus on. This could be a lack of physical strength, a bad knee or a debilitating disease. A person can also have a less visible physical weakness, such as an addiction to gambling.
There are social weaknesses as well. A person may have much of their self-worth wrapped up in their monetary worth or social standing, and this personal investment in such things may constitute a weakness that can be leveraged.
Note that a person's "weakness" can be very subjective. A sociopath may view any person with emotional ties to others as a sign of weakness, whereas society as a whole sees that as normal and healthy.
Weaknesses need not be exploited for nefarious reasons, either. For example, bringing a coffee lover a latte when asking for a favor may work wonders.