The pair formed during a time of personal turmoil in Ministry, which due to drug abuse and record label disagreements nearly ended the group. Upon an invitation from one Dave Palmer (of the jazz group Earl Harvin Trio), they performed an experimental set at a small rave in Austin, TX on Halloween, 2001. The recorded set was later released as Halloween Party on Ipecac Recordings in 2003.
Pink Anvil made another live appearance again, this time at the yearly New Years Eve Ipecac bash in San Francisco, CA at Great American Music Hall. This show was also recorded, but has yet to be released.
Max Brody has stated that plans for a third album were slightly recorded, though not completed as Barker had moved to Los Angeles in 2004. It is unknown if either of the two recordings will see the light of day.