Tennis, ping-pong, badminton, squash and golf are all fun sports that can be played with two players. Basketball is another great option that has a variety of two-player games including Horse, Around the World, All-Star Shootout and Twenty One.
While tennis, ping-pong and badminton can all be played indoors or outdoors, golf is only played outdoors. Squash is normally played indoors, though some outdoor squash courts do exist. Volleyball and paddle ball are great two-player sport games to play at the beach.
All of these games work well for two players, but they can easily incorporate more players if needed.