According to the Stamp Echo Blog, the first adhesive postage stamp was issued by the United Kingdom of Great Britain and Ireland on May 1, 1840. The Stamp Echo Blog goes on to say that the first stamps were called "penny blacks."
These stamps are not very rare due to the practice of affixing the stamps directly to documents, which were then saved. The penny black also had a large production run totalling over 68 million stamps, which makes them more common today. Wikipedia states that the United Kingdom is the only country today that omits its name from their postage stamps.