Different species may employ different methods, but all of them are based on one or more senses (after all, this is how the organism gather information about the environment). The recognition may happen by chemical signature (smell), by having a distinctive shape (sight), by emitting certain sounds (hearing), or even by behaviour patterns. Often a combination of the two is used.
Among human beings, the sense of sight is usually in charge of recognizing other members of the same species, with maybe the subconscious help of smell. In particular, the human brain has a disproportionate amount of processing power dedicated to finely analyze the features of a human face. This is why we are able to distinguish basically all six billions of human beings from each other, and a human being from a similar species like some anthropomorphic ape, with only a quick glance.
Intra-species recognition systems are often subtle. For example, ornithologists have great difficulty in distinguishing the chiffchaff from the willow warbler by eye, and there is no evidence that the birds themselves can do so other than by the different songs of the male. Sometimes, too, intra-species recognition is fallible: in many species of frog, the males are not uncommonly seen copulating with females of the wrong species or even with inanimate objects.