Definitions

# Deviance (statistics)

In statistics, deviance is a quality of fit statistic for a model that is often used for statistical hypothesis testing.

The deviance for a model M0 is defined as

$D\left(y\right) = -2 \left[log lbrace p\left(y|hat theta_0\right)rbrace -log lbrace p\left(y|hat theta_s\right)rbrace \right].,$

Here $hat theta_0$ denotes the fitted values of the parameters in the model M0, while $hat theta_s$ denotes the fitted parameters for the "full model": both sets of fitted values are implicitly functions of the observations y. Here the full model is a model with a parameter for every observation so that the data is fit exactly. This expression is simply −2 times the log-likelihood ratio of the reduced model compared to the full model. The deviance is used to compare two models - in particular in the case of generalized linear models where it has a similar role to residual variance from ANOVA in linear models.

Suppose in the framework of the GLM, we have two nested models, M1 and M2. In particular, suppose that M1 contains a set of the parameters in M2, and k additional parameters. Then, under the null hypothesis that M2 is the true model, the difference between the deviances for the two models follows an approximate chi-squared distribution with k-degrees of freedom. Note that here both models M1 and M2 would be subsets of the full model used to define the zero of the deviance criterion.