6.5 - Power

The probability of rejecting the null hypothesis, given that the null hypothesis is false, is known as power. In other words, power is the probability of correctly rejecting \(H_0\).

Power
\(Power = 1-\beta\)
\(\beta\) = probability of committing a Type II Error.

The power of a test can be increased in a number of ways, for example (1) increasing the sample size; (2) when testing a mean or difference in means, decreasing the sample standard deviation(s); (3) increasing the effect size; or (4) increasing the alpha level. 

The relationship between \(\alpha\) and \(\beta\): If the sample size is fixed, then decreasing \(\alpha\) will increase \(\beta\). If we want both \(\alpha\) and \(\beta\) to decrease, then we should increase the sample size.

On Your Own Section

Question 1
If the power of a statistical test is increased, for example by increasing the sample size, how does the probability of a Type II error change?

\(Power=P(rejecting\;H_{0}\mid H_{0}\;false)\)

The probability of committing a Type II error is known as \(\beta\).

\(\beta=P(failing\;to\;reject\;H_0\mid H_0\;false)\)

\(Power+\beta=1\)

\(Power=1-\beta\)

If power increases then \(\beta\) must decrease. So, if the power of a statistical test is increased, for example by increasing the sample size, the probability of committing a Type II error decreases.

Question 2
When the data indicate that we cannot reject the null hypothesis, does it mean that we can accept the null hypothesis? For example, when the p-value computed from is .12, we fail to reject the null hypothesis at \(\alpha=.05\). Can we say that the data support the null hypothesis?

No. When we perform a hypothesis test, we only set the size of Type I error (i.e., \(\alpha\)) and guard against it. Thus, we can only present the strength of evidence against the null hypothesis. We can sidestep the concern about Type II error if the conclusion never mentions that the null hypothesis is accepted. When the null hypothesis cannot be rejected, there are two possible cases:

1) the null hypothesis is true

2) the sample size is not large enough to reject the null hypothesis (i.e., statistical power is too low)

Question 3
A study was conducted by a retail store to determine if the majority of their customers were under the age of 25. The results showed that \(\widehat{p}=.48\), the null hypothesis was not rejected, the company concluded that they did not have evidence that the majority of their customers were under the age of 25. But, in reality, the proportion of all of their customers (i.e., the population) is actually \(p=.53\). Did this research study result in a Type I error, Type II error, or correct decision?

The result of the study was to fail to reject the null hypothesis. In reality, the null hypothesis was false. This is a Type II error.

Question 4
A university conducted a hypothesis test to determine if their average SAT-Math score was greater than the national average of 500. They collected a sample of \(n=800\) students and found that \(\overline{x}=506\). The t test statistic was 1.70 and \(p=.045\) therefore they rejected the null hypothesis and concluded that the mean SAT-Math score at their university was higher than the national average. However, in reality, in the population of all students at the university, the mean SAT-Math score is 503. Did this research study result in a Type I error, Type II error, or correct decision?
This study came to a correct conclusion. They rejected the null hypothesis and concluded that \(\mu>500\) when in reality \(\mu=503\) which is greater than 500.