Born in Peru, Indiana, he graduated from Northwestern University in 1912 with a degree in music and hit the Vaudeville circuit. In 1914 he met Chic Johnson, who was advertising himself as the "Greatest Ragtime Pianist in the Midwest."
The pair met when they were hired as musicians in the same band. When the band broke up "Ole" Olsen and "Chic" Johnson formed a comedy team. They really did not have a set act but found themselves booked into a small Chicago nightclub as part of "Mike Fritzol’s Frolics." When it came time for their turn in the show, unannounced and not particularly welcome, the brave pair pushed a piano onstage. Johnson seated himself at the keyboard and began to plunk out a ragtime tune. Olsen joined in with his violin and started singing, making up comical lyrics as he went along. The pair began to exchange "patter," mostly insults and the soon-to-be-famous "Olsen and Johnson" team emerged.
Ole Olsen was married twice. He had three children with his first wife, Lillian; J. C., Joy, and Moya. They were later divorced. His son, J. C. committed suicide. Moya married William P. Lear of Learjet fame in 1942. Ole was involved in a serious automobile accident in 1950 and recuperated at the Lear home. In June 1961 Ole married Eileen Maria Osthoff, a dancer and choreographer he had known for eight years.
Ole's ambition was to make people laugh. He is remembered for the quote, "May you live as long as you want to, and laugh as long as you live". Those words are engraved on his headstone. Olsen died at the age of 70 and is interred in Las Vegas, Nevada in a grave ajoining Chic Johnson's.