Sample mean and sample covariance are statistics computed from a collection of data, thought of as being random.
Sample mean and covariance
Given a random sample
-dimensional random variable
random variables with the same distribution
), the sample mean
In coordinates, writing the vectors as columns,
the entries of the sample mean are
The sample covariance of is the by matrix with the entries given by
The sample mean and the sample covariance matrix are unbiased estimates of the mean and the covariance matrix of the random variable . The reason why the sample covariance matrix has in the denominator rather than is essentially that the population mean is not known and is replaced by the sample mean . If the population mean is known, the analogous unbiased estimate
with the population mean indeed does have . This is an example why in probability and statistics it is essential to distinguish between upper case letters (random variables) and lower case letters (realizations of the random variables).
The maximum likelihood estimate of the covariance
for the Gaussian distribution case has as well. The difference of course diminishes for large .
In a weighted sample, each vector is assigned a weight . Without loss of generality, assume that the weights are normalized:
(If they are not, divide the weights by their sum.)
Then the weighted mean and the weighted covariance matrix are given by
If all weights are the same, , the weighted mean and covariance reduce to the sample mean and covariance above.