Page 99 - 4660

2.3. Hypothesis testing

So far we have concentrated on using a data sample to obtain a number or a set of numbers.
These numbers may be estimated values for the moments or central moments of the population
from which the sample was drawn or, more generally, the values of some parameters a in an
assumed model for the data. Sometimes, however, one wishes to use the data to give a ’yes’ or
’no’ answer to a particular question. For example, one might wish to know whether some
assumed model does, in fact, provide a good fit to the data, or whether two parameters have the
same value.

Simple and composite hypotheses

In order to use data to answer questions of this sort, the question must be posed precisely. This is
done by first asserting that some hypothesis is true. The hypothesis under consideration is
traditionally called the null hypothesis and is denoted by H 0 . In particular, this usually specifies
some form P(x|H 0 ) for the probability density function from which the data x are drawn. If the
hypothesis determines the PDF uniquely, then it is said to be a simple hypothesis. If, however,
the hypothesis determines the functional form of the PDF but not the values of certain
parameters a on which it depends then it is called a composite hypothesis.
One decides whether to accept or reject the null hypothesis H0 by performing some
statistical test, as described below in subsection. In fact, formally one uses a statistical test to
decide between the null hypothesis H 0 and the alternative hypothesis H 1 . We define the latter to
¯
be the complement H 0 of the null hypothesis within some restricted hypothesis space known (or
assumed) in advance. Hence, rejection of H 0 implies acceptance of H 1 , and vice versa.
As an example, let us consider the case in which a sample x is drawn from a Gaussian
2
distribution with a known variance σ but with an unknown mean µ. If one adopts the null
hypothesis H 0 that µ = 0 which we write as H 0 : µ = 0, then the corresponding alternative
hypothesis must be H 1 : µ ̸= 0. Note that, in this case, H 0 is a simple hypothesis whereas H 1 is a
composite hypothesis. If, however, one adopted the null hypothesis H 0 : µ < 0 then the
alternative hypothesis would be H 1 : µ ≥ 0, so that both H 0 and H 1 would be composite
hypotheses. Very occasionally both H 0 and H 1 will be simple hypotheses. In our illustration, this
would occur, for example, if one knew in advance that the mean µ of the Gaussian distribution
were equal to either zero or unity. In this case, if one adopted the null hypothesis H 0 : µ = 0 then
the alternative hypothesis would be H 1 : µ = 1.

Statistical tests

In our discussion of hypothesis testing we will restrict our attention to cases in which the null
hypothesis H 0 is simple (see above). We begin by constructing a test statistic t(x) from the data
sample. Although, in general, the test statistic need not be just a (scalar) number, and could be a
multi-dimensional (vector) quantity, we will restrict our attention to the former case. Like any
statistic, t(x) will be a random variable. Moreover, given the simple null hypothesis H 0
concerning the PDF from which the sample was drawn, we may determine (in principle) the
sampling distribution P(t|H 0 ) of the test statistic. A typical example of such a sampling
distribution is shown in Fig. 3.1.
One defines for t a rejection region containing some fraction α of the total probability. For
example, the (one-tailed) rejection region could consist of values of t greater than some value t crit ,

99