The bispectrum is a statistic used to search for nonlinear interactions. The Fourier transform of the second-order cumulant, i.e., the autocorrelation function, is the traditional power spectrum. The Fourier transform of C3(t1,t2) (third-order cumulant) is called bispectrum or bispectral density. They fall in the category of Higher Order Spectra, or Polyspectra and provide supplementary information to the power spectrum. The third order polyspectrum (bispectrum) is the easiest to compute, and hence the most popular.
The difference with measuring coherence (coherence analysis is an extensively used method to study the correlations in frequency domain, between two simultaneously measured signals) is the need for both input and output measurements by estimating two auto-spectra and one cross spectrum. On the other hand, bicoherence is an auto-quantity, i.e. it can be computed from a single signal. The coherence function provides a quantification of deviations from linearity in the system which lies between the input and output measurement sensors. The bicoherence measures the proportion of the signal energy at any bifrequency that is quadratically phase coupled.