Printer-friendly versionPrinter-friendly version

Let's take a look at another example that involves calculating the power of a hypothesis test.

iq logoExample

Let X denote the IQ of a randomly selected adult American. Assume, a bit unrealistically, that X is normally distributed with unknown mean μ and standard deviation 16. Take a random sample of n = 16 students, so that, after setting the probability of committing a Type I error at α = 0.05, we can test the null hypothesis H0μ = 100 against the alternative hypothesis that HAμ > 100.

What is the power of the hypothesis test if the true population mean were μ = 108?

Solution. Setting α, the probability of committing a Type I error, to 0.05, implies that we should reject the null hypothesis when the test statistic Z ≥ 1.645, or equivalently, when the observed sample mean is 106.58 or greater:

drawing

because we transform the test statistic Z to the sample mean by way of:

eqn

Now, that implies that the power, that is, the probability of rejecting the null hypothesis, when μ = 108 is 0.6406 as calculated here (recalling that Φ(z) is standard notation for the cumulative distribution function of the standard normal random variable):

eqn

and illustrated here:

drawing

In summary, we have determined that we have (only) a 64.06% chance of rejecting the null hypothesis H0μ = 100 in favor of the alternative hypothesis HAμ > 100 if the true unknown population mean is in reality μ = 108.

What is the power of the hypothesis test if the true population mean were μ = 112?

Solution. Because we are setting α, the probability of committing a Type I error, to 0.05, we again reject the null hypothesis when the test statistic Z ≥ 1.645, or equivalently, when the observed sample mean is 106.58 or greater. That means that the probability of rejecting the null hypothesis, when μ = 112 is 0.9131 as calculated here:

eqn

and illustrated here:

drawing

In summary, we have determined that we now have a 91.31% chance of rejecting the null hypothesis H0μ = 100 in favor of the alternative hypothesis HAμ > 100 if the true unknown population mean is in reality μ = 112. Hmm.... it should make sense that the probability of rejecting the null hypothesis is larger for values of the mean, such as 112, that are far away from the assumed mean under the null hypothesis.

What is the power of the hypothesis test if the true population mean were μ = 116?

Solution. Again, because we are setting α, the probability of committing a Type I error, to 0.05, we reject the null hypothesis when the test statistic Z ≥ 1.645, or equivalently, when the observed sample mean is 106.58 or greater. That means that the probability of rejecting the null hypothesis, when μ = 116 is 0.9909 as calculated here:

eqn

and illustrated here:

drawing

In summary, we have determined that, in this case, we have a 99.09% chance of rejecting the null hypothesis H0μ = 100 in favor of the alternative hypothesis HAμ > 100 if the true unknown population mean is in reality μ = 116. The probability of rejecting the null hypothesis is the largest yet of those we calculated, because the mean, 116, is the farthest away from the assumed mean under the null hypothesis.

Are you growing weary of this? Tongue out Let's summarize a few things we've learned from engaging in this exercise:

(1) First and foremost, my instructor can be tedious at times..... errrr, I mean, first and foremost, the power of a hypothesis test depends on the value of the parameter being investigated. In the above, example, the power of the hypothesis test depends on the value of the mean μ

(2) As the actual mean μ moves further away from the value of the mean μ = 100 under the null hypothesis, the power of the hypothesis test increases. 

It's that first point that leads us to what is called the power function of the hypothesis test. If you go back and take a look, you'll see that in each case our calculation of the power involved a step that looks like this:

\(\text{Power }  =1 - \Phi (z) \)  where  \(z = \frac{106.58 - \mu}{16 / \sqrt{16}}  \)

That is, if we use the standard notation K(μ) to denote the power function, as it depends on μ, we have:

\[K(\mu) = 1- \Phi \left( \frac{106.58 - \mu}{16 / \sqrt{16}}  \right) \]

So, the reality is your instructor could have been a whole lot more tedious by calculating the power for every possible value of μ under the alternative hypothesis! What we can do instead is create a plot of the power function, with the mean μ on the horizontal axis and the power K(μ) on the vertical axis. Doing so, we get a plot in this case that looks like this:

plot

Now, what can we learn from this plot? Well:

(1) We can see that α (the probability of a Type I error), β (the probability of a Type II error) , and K(μ) are all represented on a power function plot, as illustrated here: 

plot

(2) We can see that the probability of a Type I error is α = K(100) = 0.05, that is, the probability of rejecting the null hypothesis when the null hypothesis is true is 0.05.

(3) We can see the power of a test K(μ), as well as the probability of a Type II error β(μ), for each possible value of μ.

(4) We can see that β(μ) = 1−K(μ) and vice versa, that is, K(μ) = 1−β(μ).

(5) And we can see graphically that, indeed, as the actual mean μ moves further away from the null mean μ = 100, the power of the hypothesis test increases.

Now, what would do you suppose would happen to the power of our hypothesis test if we were to change our willingness to commit a Type I error? Would the power for a given value of μ increase, decrease, or remain unchanged?  Suppose, for example, that we wanted to set α = 0.01 instead of α = 0.05? Let's return to our example to explore this question.

iq testExample (continued)

Let X denote the IQ of a randomly selected adult American. Assume, a bit unrealistically, that X is normally distributed with unknown mean μ and standard deviation 16. Take a random sample of n = 16 students, so that, after setting the probability of committing a Type I error at α = 0.01, we can test the null hypothesis H0μ = 100 against the alternative hypothesis that HAμ > 100.

What is the power of the hypothesis test if the true population mean were μ = 108?

Solution. Setting α, the probability of committing a Type I error, to 0.01, implies that we should reject the null hypothesis when the test statistic Z ≥ 2.326, or equivalently, when the observed sample mean is 109.304 or greater:

distribution

because:

\[\bar{x} = \mu + z \left( \frac{\sigma}{\sqrt{n}} \right) =100 + 2.326\left( \frac{16}{\sqrt{16}} \right)=109.304 \]

That means that the probability of rejecting the null hypothesis, when μ = 108 is 0.3722 as calculated here:

eqn

So, the power when μ = 108 and α = 0.01 is smaller (0.3722) than the power when μ = 108 and α = 0.05 (0.6406)! Perhaps we can see this graphically:

drawing

By the way, we could again alternatively look at the glass as being half-empty. In that case, the probability of a Type II error when μ = 108 and α = 0.01 is 1−0.3722 = 0.6278. In this case, the probability of a Type II error is greater than the probability of a Type II error when μ = 108 and α = 0.05.

All of this can be seen graphically by plotting the two power functions, one where α = 0.01 and the other where α = 0.05, simultaneously. Doing so, we get a plot that looks like this:

plot

This last example illustrates that, providing the sample size n remains unchanged, a decrease in α causes an increase in β, and at least theoretically, if not practically, a decrease in β causes an increase in α. It turns out that the only way that α and β can be decreased simultaneously is by increasing the sample size n.