Page 88 - 4660

Estimation

independently from a one-dimensional population P(x), it is possible to construct some basic
estimators for the moments and central moments of P(x). In this section, we investigate the
estimating properties of the common sample statistics early presented. In fact, expectation
values and variances of these sample statistics can be computed without prior knowledge of the
functional form of the population; they depend only on the sample size N and certain moments
and central moments of P(x).

Population mean. Let us suppose that the parent population P(x) has mean µ and variance
2
2
σ . An obvious estimator ˆµ of the population mean is the sample mean ¯x. Provided µ and σ are
both finite, we may apply the central limit theorem directly to obtain exact expressions, valid for
samples of any size N, for the expectation value and variance of ¯x. From parts (i) and (ii) of the
central limit theorem we immediately obtain

σ 2
E(¯ x) = µ, Var(¯ x) = . (2.5)
N
Thus we see that ¯x is an unbiased estimator of µ. Moreover, we note that the standard error in
√
¯ x is σ/ N, and so the sampling distribution of ¯x becomes more tightly centred around µ as the
sample size N increases. Indeed, since Var(¯x) → 0 as N → ∞, ¯x is also a consistent estimator of
µ.
In the limit of large N, we may in fact obtain an approximate form for the full sampling
distribution of ¯x. Part (iii) of the central limit theorem tells us immediately that, for large N, the
sampling distribution of ¯x is given approximately by the Gaussian form
[ 2 ]
1 (¯x − µ)
P(¯x|µ, σ) ≈ √ exp − 2 .
2πσ /N 2σ /N
2
Note that this does not depend on the form of the original parent population. If, however, the
parent population is in fact Gaussian then this result is exact for samples of any size N.

2
Population variance. An estimator for the population variance σ is not so straightforward to
define as one for the mean. Complications arise because, in many cases, the true mean of the
population µ is not known. Nevertheless, let us begin by considering the case where in fact µ is
known. In this event, a useful estimator is
( )
N N
1 ∑ 1 ∑
2
2
ˆ 2
σ = (x i − µ) = x 2 i − µ . (2.6)
N N
i=1 i=1
Example 2.3. Show that σ ˆ 2 is an unbiased and consistent estimator of the
2
population variance σ . ,
ˆ 2
Solution. The expectation value of σ is given by
( )
N
1 ∑
2
2
2
2
2
ˆ 2
E(σ ) = E x 2 i − µ = E(x ) − µ = µ 2 − µ = σ ,
i
N
i=1
from which we see that the estimator is unbiased. The variance of the estimator is
[ ]
N
1 ∑ 1 1
2
2
2
ˆ 2
Var(σ ) = V x 2 + Var(µ ) Var(x ) = (µ 4 − µ ),
2
i
i
N 2 N N
i=1
4
2
2
2
2
2
in which we have used that fact that Var(µ ) = 0 and Var(x ) = E(x ) − (E(x )) = µ 4 − µ ,
i i i 2
ˆ 2
ˆ 2
where µ r is the r-th population moment. Since σ is unbiased and Var(σ ) → 0 as N → ∞,
2
showing that it is also a consistent estimator of σ , the result is established.
88