He is considered a member of the "first generation" of American free improvisers, reflecting influences from European free players including, among others, Derek Bailey and Evan Parker. In 1978, Kaiser founded the Metalanguage Records label with Larry Ochs (Rova Saxophone Quartet) and Greg Goodman.
Kaiser was a member of the aggregation French Frith Kaiser Thompson, together with fellow experimental musicians John French, Fred Frith, and English folk-rocker Richard Thompson. They recorded two eclectic albums, Live, Love, Larf & Loaf (1987) and Invisible Means (1990).
In 1991, Kaiser went to Madagascar with fellow guitarist David Lindley, where they spent two weeks recording music with Malagasy musicians. Three volumes of this music were released by Shanachie under the title A World Out of Time.
Since 1998, Kaiser has been collaborating with trumpeter Wadada Leo Smith in the "Yo Miles!" project, releasing a series of tributes to Miles Davis's 1970s electric music. This shifting aggregation has included musicians from the worlds of rock (guitarists Nels Cline and Mike Keneally, drummer Steve Smith), jazz (saxophonists Greg Osby and John Tchicai), and Indian classical music (tabla player Zakir Hussain).
In 2001, Kaiser spent two and a half months in Antarctica on a National Science Foundation Antarctic Artists and Writers Program grant. He has subsequently returned to work as a research diver. His underwater camerawork was featured in two Werner Herzog documentary films, The Wild Blue Yonder (2005) and Encounters at the End of the World (2007), which he also produced.