He was involved in possibly the best era ever for Aberdeen who, under the management of Sir Alex Ferguson, won the Cup Winners' Cup in 1983 with a 2–1 win over Real Madrid in which the then 19 year old Black scored the first goal. Black's main asset was his balletic heading ability and scoring goals. He made 180 appearances (30 as substitute) and scored 70 goals for Aberdeen between 1981 and 1986 before joining FC Metz in France.
After five seasons (1986–91) with Metz, during which he won the 1988 French Cup, Black was forced to retire prematurely from the game due to a chronic back problem. In all, he made 85 appearances, scoring 26 goals. It is from his time in France that Black became fluent in French.
Since then he has coached and managed most notably as assistant manager at Celtic and manager at Coventry City and Motherwell. Black joined Motherwell in October 2001. He was appointed Coventry manager in January 2004 following the resignation of Gary McAllister. Oddly, Black was sacked from Coventry on the back of a 5–2 win against Gillingham for "being too good for this club". Black joined Birmingham City in July 2004, as assistant manager to Steve Bruce. He managed one game as caretaker manager, after Bruce joined Wigan Athletic, before leaving St Andrews in November 2007.