The term derives from burlesque, around 1930, where the slang term "banana" referred to a comedian.
In the I Love Lucy show, Lucille Ball was the top banana, and Vivian Vance was the second banana.