In 1984, Tomkins first made the Australian National team in eight man boat (8+). In 1986, the Australian eight won a gold medal at the World Rowing Championships and at the 1988 Summer Olympics, the Australian eight finished fourth.
In 1990, Tomkins, with Nick Green, Sam Patten and Mike McKay, began racing the coxless four. Their success was immediate. They won the 1990 and 1991 World Championships. With Andrew Cooper replacing Sam Patten, they followed up with a gold medal at the 1992 Summer Olympics in Barcelona. The crew's success gained them the nickname Oarsome Foursome.
In 1998, the Oarsome Foursome won the four with coxswain, and Tomkins and Nicholas Green won the pair with coxswain. In 1999, the boat would go on to try out, but lose the 1999 Australian selection trials in the four without. Tomkins and Drew Ginn decided to switch to the coxless pairs and they won the 1999 World Championships. This win made Tomkins the first man to win a gold medal at the World Championships in each of the 5 sweep rowing events.
Tomkins and Ginn had planned to row the pair at the 2000 Summer Olympics in Sydney, but while in Europe preparing for the games, a severe back injury requiring surgery sidelined Ginn. On short notice, Tomkins teamed with team alternate Matthew Long, (who had to switch from the port side to the starboard side of the boat) and they raced a surprising third at the Lucerne World Cup race. Tomkins and Long were selected to represent Australia, and at the Olympics, they finished third behind France and the United States, just 1.3 seconds out of first place.
Returning to the coxless pairs in 2002, Ginn and Tomkins beat the heavily favoured British crew of Matthew Pinsent and James Cracknell in a world cup race before finishing a close fourth at the World Championships. In 2003, Ginn and Tomkins reversed the prior years results winning the World Championship, with Pinset and Cracknell getting fourth. In 2004, Pinset and Cracknell moved to the coxless four to better their chances for a medal. Ginn and Tomkins would go on to win the coxless pairs at the 2004 Summer Olympics, leading at every mark, beating Croatia by 2 seconds, with South Africa claiming the bronze.
Tomkins was selected in the mens eight for the 2007 Rowing World Championships in Munich, placing a disappointing 4th. He competed in his fifth Olympic Games at Beijing following the disqualification of the Russian Federation boat as a result of a doping scandal. Tomkins was the Australian flag bearer at the Opening Ceremony. Tomkins and his crew finished last in the men's Eight A Final and 6th overall at the Beijing Games.
James Tomkins was announced as the 2008 Victorian Father of the Year by the Father's Day Council of Victoria Inc.