Koller started his football training as a goalkeeper, but was converted to striker by the time he started his professional career with Czech giants AC Sparta Praha. In 1996, Koller caught the eye of Belgian football and signed with the club KSC Lokeren. After a successful three-year personal campaign, in which he managed to finish as Belgian first division top-scorer in his last season at Lokeren, Koller was appointed by Belgian giants RSC Anderlecht. A brilliant performance in his debut season earned him the Belgian Golden Shoe in 2000 and just a year later he was bought by German side Borussia Dortmund.
During Koller's spell at Dortmund (in which the club conquered the Bundesliga trophy in the 2001-2002 season), his training as a keeper saw some use in the 2002-03 season. In one Bundesliga match at Bayern Munich, Borussia Dortmund's keeper Jens Lehmann was sent off about midway through the second half, and Koller moved from striker to keeper after he had already scored once in the first half. He kept a clean sheet for the rest of the match against good opportunities by Michael Ballack and others, and was named the Bundesliga's top keeper of the week for his performance. Nevertheless Dortmund lost this important match.
In a surprising move, Koller signed with French side AS Monaco in 2006 but a two-season disappointing campaign, despite earning a decent scoring record, forced him to move back to Germany to play with 1. FC Nuremberg. Unfortunately for Koller (who wasn't the only Czech in the team as he was partnered with Tomas Galásek and Jaromír Blazek), the Nuremberg-based club displayed a poor performance and was relegated to 2.Bundesliga at the end of the 2007-2008 season. On June 23 Koller was transferred to Russian club Krylya Sovetov Samara in a deal worth 1 million euros.
His best performance to date in a major tournament was in the Euro 2004 when his side reached semi-finals and he scored two goals, not to mention the pivotal partnership that he forged with fellow striker Milan Baros. In the 2006 World Cup held at Germany, Koller suffered a minor thigh injury during the group stage debut game against the United States on June 12 2006, not long after the opening goal he scored. His injury was a major blow for the Czech Republic, which lost their next two matches without him.
Just about into his third month with Nuremberg, Koller announced that he would retire from the Czech national team after Euro 2008. He currently has 194 goals in his professional career and scored a total of 55 goals in his international career, including a crucial headed goal in their deciding Euro 2008 Group A match against Turkey. However, his goal didn't prove good enough as the Czechs conceded 3 goals in the last 15 minutes to throw away a 2-0 lead, lose the match 3-2 and thus send Koller into international retirement in a dramatic way.