In 2000 he was released from a 17-year prison sentence. Upon his release, he immediately assumed the kaicho position in the Fourth Kenryu-kai, and he became a formal member of the , then headed by Kuwata. Inoue became Kuwata's adopted son in May, 2003.
In August, 2005, Inoue was promoted from (eldest son) to the 4th kumicho of the Yamaken-gumi. At this time, he became a formal member of the Sixth Yamaguchi-gumi. He assumed the position of kambu of the Yamaguchi-gumi in September, 2005.