Less technically, in statistics a loss function represents the loss (cost in money or loss in utility in some other sense) associated with an estimate being "wrong" (different from either a desired or a true value) as a function of a measure of the degree of wrongness (generally the difference between the estimated value and the true or desired value.)
Both Frequentist and Bayesian statistical theory involve calculating statistics in such a way as to minimize the expected loss observed from being wrong given a set of assumptions about the data and ones loss function. Sound statistical practice requires selecting an estimator consistent with the actual loss experienced in the context of a particular applied problem. Thus, in the applied use of loss functions, selecting which statistical method to use to model an applied problem depends on knowing the losses that will be experienced from being wrong under the problem's particular circumstances, which results in the introduction of an element of teleology into problems of scientific decision-making .
A common example involves estimating "location". Under typical statistical assumptions, the mean or average is the statistic for estimating location that minimizes the expected loss experienced under the Taguchi or squared-error loss function, while the median is the estimator that minimizes expected loss experienced under the absolute-difference loss function. Still different estimators would be optimal under other, less common circumstances.
Loss functions in economics are typically expressed in monetary terms. For example:
where k is some arbitrary constant.
A loss function satisfies the definition of a random variable so we can establish a cumulative distribution function and an expected value. However, more commonly, the loss function is expressed as a function of some other random variable. For example, the time that a light bulb operates before failure is a random variable and we can specify the loss, arising from having to cope in the dark and/or replace the bulb, as a function of failure time.
The expected loss (sometimes known as risk) is:
One of the consequences of Bayesian inference is that in addition to experimental data, the loss function does not in itself wholly determine a decision. What is important is the relationship between the loss function and the prior probability. So it is possible to have two different loss functions which lead to the same decision when the prior probability distributions associated with each compensate for the details of each loss function.
Combining the three elements of the prior probability, the data, and the loss function then allows decisions to be based on maximizing the subjective expected utility, a concept introduced by Leonard J. Savage.
The use of a quadratic loss function is common, for example when using least squares techniques or Taguchi methods. It is often more mathematically tractable than other loss functions because of the properties of variances, as well as being symmetric: an error above the target causes the same loss as the same magnitude of error below the target. If the target is t, then a quadratic loss function is