He was born in Melbourne, Victoria, and was educated at Monash University, Melbourne, where he graduated in arts and education. He worked as a teacher in government schools and as a taxi-driver before entering politics.
A leading member of the right-wing faction of the ALP, Ray defeated a left-wing Senator, Jean Melzer, in a ballot for a place on the party's Senate ticket. This caused great bitterness in the Socialist Left faction, which has disliked Ray ever since. He was elected to the Senate at the October 1980 election, taking his seat on 1 July 1981.
In the Hawke Labor government Ray was Minister for Home Affairs 1987-88, Minister for Immigration, Local Government and Ethnic Affairs 1988-90 (with a seat in the Cabinet), and Minister for Defence 1990-96. In this portfolio he strongly supported Hawke's decision to send Australian forces to support the United Nations in the Gulf War, despite bitter opposition from the left.
After the defeat of the Keating government in 1996 election, Ray did not seek election to the Opposition Shadow Ministry, but remained a powerful figure in the Labor Party, acting as a key factional supporter of Opposition Leader Kim Beazley 1996-2001. His influence diminished when Mark Latham became Leader in 2003. He did not stand for re-election at the 2007 election, which meant that his Senate term was due to expire on 30 June 2008. He formally resigned from Parliament on 5 May 2008, having spent two more days in the Senate as a member of government than as a member of the Opposition. The casual vacancy was filled by Jacinta Collins.