Published on *STAT 414 / 415* (https://onlinecourses.science.psu.edu/stat414)

In the previous lesson, we learned that one of the primary uses of an estimated regression line:

\[\hat{y}=\hat{\alpha}+\hat{\beta}(x-\bar{x})\]

is to determine whether or not a linear relationship exists between the predictor *x* and the response *y*. In that lesson, we learned how to calculate a confidence interval for the slope parameter *β *as a way of determining whether a linear relationship does exist. In this lesson, we'll learn learn two other primary uses of an estimated regression line:

(1) If we are interested in knowing the value of the mean response *E*(*Y*) = *μ _{Y}* for a given value

(2) If we are interested in knowing the value of a new observation *Y*_{n+1} for a given value *x* of the predictor, we'll learn how to calculate a **prediction interval for the new observation **** Y_{n+1}**.

We have gotten so good with deriving confidence intervals for various parameters, let's just jump right in and state (and prove) the result.

\[\hat{y} \pm t_{\alpha/2,n-2}\sqrt{MSE} \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}\] |

**Proof. **We know from our work in the previous lesson that a point estimate of the mean *μ _{Y}*

**\[\hat{y}=\hat{\alpha}+\hat{\beta}(x-\bar{x})\]**

Now, recall that:

** **\[\hat{\alpha} \sim N\left(\alpha,\dfrac{\sigma^2}{n}\right)\] and \[\hat{\beta}\sim N\left(\beta,\dfrac{\sigma^2}{\sum_{i=1}^n (x_i-\bar{x})^2}\right)\] and **\[\dfrac{n\hat{\sigma}^2}{\sigma^2}\sim \chi^2_{(n-2)}\]**

are independent. Therefore, \[\hat{Y}\]is a linear combination of independent normal random variables with mean:

\[E(\hat{y})=E[\hat{\alpha}+\hat{\beta}(x-\bar{x})]=E(\hat{\alpha})+(x-\bar{x})E(\hat{\beta})=\alpha+\beta(x-\bar{x})=\mu_Y\]

and variance:

\[Var(\hat{y})=Var[\hat{\alpha}+\hat{\beta}(x-\bar{x})]=Var(\hat{\alpha})+(x-\bar{x})^2 Var(\hat{\beta})=\dfrac{\sigma^2}{n}+\dfrac{(x-\bar{x})^2\sigma^2}{\sum(x_i-\bar{x})^2}=\sigma^2\left[\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}\right]\]

The first equality holds by **the definition of \[\hat{Y}\]. The second equality holds because \[\hat{\alpha}\] and \[\hat{\beta}\] are independent. The third equality comes from the distributions of** \[\hat{\alpha}\] and \[\hat{\beta}\] that are recalled above. And, the last equality comes from simple algebra. Putting it all together, we have:

\[\hat{Y} \sim N\left(\mu_Y, \sigma^2\left[\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}\right]\right)\]

Now, the definition of a *T* random variable tells us that:

\[T=\dfrac{\dfrac{\hat{Y}-\mu_Y}{\sigma \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}}}{\sqrt{\dfrac{n\hat{\sigma}^2}{\sigma^2}/(n-2)}}=\dfrac{\hat{Y}-\mu_Y}{\sqrt{MSE} \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}} \sim t_{n-2}\]

So, finding the confidence interval for *μ _{Y}*

\[P\left(-t_{\alpha/2,n-2} \leq \dfrac{\hat{Y}-\mu_Y}{\sqrt{MSE} \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}} \leq +t_{\alpha/2,n-2}\right)=1-\alpha\]

Upon doing the manipulation, we get that a (1−*α*)100% confidence interval for *μ _{Y}*

\[\hat{y} \pm t_{\alpha/2,n-2}\sqrt{MSE} \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}\]

as was to be proved.

The eruptions of Old Faithful Geyser in Yellowstone National Park, Wyoming are quite regular (and hence its name). Rangers post the predicted time until the next eruption (*y*, in minutes) based on the duration of the previous eruption (*x*, in minutes). Using the data collected on 107 eruptions [1] from a park geologist, R. A. Hutchinson, what is the mean time until the next eruption if the previous eruption lasted 4.8 minutes? lasted 3.5 minutes? (Photo credit: Tony Lehrman)

**Solution.** The easiest (and most practical!) way of calculating the confidence interval for the mean is to let Minitab do the work for us. Here's what the resulting analysis looks like:

That is, we can be 95% confident that, if the previous eruption lasted 4.8 minutes, then the mean time until the next eruption is between 83.286 and 87.484 minutes. And, we can be 95% confident that, if the previous eruption lasted 3.5 minutes, then the mean time until the next eruption is between 70.140 and 72.703 minutes.

Let's do one of the calculations by hand, though. When the previous eruption lasted *x* = 4.8 minutes, then the predicted time until the next eruption is:

\[\hat{y}=33.828 + 10.741(4.8)=85.385\]

Now, we can use Minitab or a probability calculator to determine that *t*_{0.025,105} = 1.9828. We can also use Minitab to determine that *MSE* equals 44.6 (it is rounded to 45 in the above output), the mean duration is 3.46075 minutes, and:

\[\sum\limits_{i=1}^n (x_i-\bar{x})^2=113.835\]

Putting it all together, we get:

\[85.385 \pm 1.9828 \sqrt{44.66} \sqrt{\dfrac{1}{107}+\dfrac{(4.8-3.46075)^2}{113.835}}\]

which simplifies to this:

\[85.385 \pm 2.099\]

and finally this:

\[(83.286,87.484)\]

as we (thankfully) obtained previously using Minitab. Incidentally, you might note that the length of the confidence interval for *μ _{Y}* when

\[87.484-83.286=4.198\]

and the length of the confidence interval when *x* = 3.5 is:

\[72.703-70.140=2.563\]

Hmmm. That suggests that the confidence interval is narrower when the *x* value is close to the mean of all of the *x* values. That is, in fact, one generalization, among others, that we can make about the length of the confidence interval for *μ _{Y}*.

If we take a look at the formula for the confidence interval for *μ _{Y}*:

\[\hat{y} \pm t_{\alpha/2,n-2}\sqrt{MSE} \sqrt{\dfrac{1}{n}+\dfrac{(x-\bar{x})^2}{\sum(x_i-\bar{x})^2}}\]

we can determine four ways in which we can get a narrow confidence interval for *μ _{Y}*. We can:

(1) Estimate the mean *μ _{Y}* at the mean of the predictor values. That's because when \[x=\bar{x}\], the term circled in

contributes nothing to the length of the interval. That is, the shortest confidence interval for *μ _{Y}* occurs when \[x=\bar{x}\].

(2) Decrease the confidence level. That's because, the smaller the confidence level, the smaller the term circled in **blue**:

(3) Increase the sample size. That's because, the larger the sample size, the larger the term circled in **blue**:

and therefore the shorter the length of the interval.

(4) Choose predictor values *x _{i}* so that they are quite spread out. That's because, the more spread out the predictor values, the larger the term circled in

and therefore the shorter the length of the interval.

On the previous page, we focused our attention on deriving a confidence interval for the mean *μ _{Y}* at

is:x = x_{n+1} \[\hat{y}_{n+1} \pm t_{\alpha/2,n-2}\sqrt{MSE} \sqrt{1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}}\] |

**Proof. **First,** **recall that:

**\[Y_{n+1} \sim N(\alpha+\beta(x_{n+1}-\bar{x}),\sigma^2)\] **and** **\[\hat{\alpha} \sim N\left(\alpha,\dfrac{\sigma^2}{n}\right)\] and \[\hat{\beta}\sim N\left(\beta,\dfrac{\sigma^2}{\sum_{i=1}^n (x_i-\bar{x})^2}\right)\] and **\[\dfrac{n\hat{\sigma}^2}{\sigma^2}=\dfrac{(n-2)MSE}{\sigma^2}\sim \chi^2_{(n-2)}\]**

are independent. Therefore:

\[W=Y_{n+1}-\hat{Y}_{n+1}=Y_{n+1}-\hat{\alpha}-\hat{\beta}(x_{n+1}-\bar{x})\]

is a linear combination of independent normal random variables with mean:

and variance:

The first equality holds by **the definition of W. The second equality holds because \[Y_{n+1}\], \(\hat{\alpha}\) and \[\hat{\beta}\] are independent. The third equality comes from the distributions of**

\[W=(Y_{n+1}-\hat{Y}_{n+1})\sim N\left(0,\sigma^2\left[1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}\right]\right)\]

Now, the definition of a *T* random variable tells us that:

\[T=\dfrac{\dfrac{(Y_{n+1}-\hat{Y}_{n+1})-0}{\sqrt{\sigma^2\left(1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}\right)}}}{\sqrt{\dfrac{n\hat{\sigma}^2}{\sigma^2}/(n-2)}}=\dfrac{(Y_{n+1}-\hat{Y}_{n+1})}{\sqrt{MSE}\sqrt{1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}}} \sim t_{n-2}\]

So, finding the prediction interval for *Y*_{n+1}** **again reduces to manipulating the quantity inside the parentheses of a probability statement:

\[P\left(-t_{\alpha/2,n-2} \leq \dfrac{(Y_{n+1}-\hat{Y}_{n+1})}{\sqrt{MSE}\sqrt{1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}}} \leq +t_{\alpha/2,n-2}\right)=1-\alpha\]

Upon doing the manipulation, we get that a (1−*α*)100% prediction interval for *Y*_{n+1}** **is:

\[\hat{y}_{n+1} \pm t_{\alpha/2,n-2}\sqrt{MSE} \sqrt{1+\dfrac{1}{n}+\dfrac{(x_{n+1}-\bar{x})^2}{\sum(x_i-\bar{x})^2}}\]

as was to be proved.

The eruptions of Old Faithful Geyser in Yellowstone National Park, Wyoming are quite regular (and hence its name). Rangers post the predicted time until the next eruption (*y*, in minutes) based on the duration of the previous eruption (*x*, in minutes). Using the data collected on 107 eruptions [1] from a park geologist, R. A. Hutchinson, what is the **predicted time** until the next eruption if the previous eruption lasted 4.8 minutes? lasted 3.5 minutes?

**Solution.** Again, the easiest (and most practical!) way of calculating the prediction interval for the new observation is to let Minitab do the work for us. Here's what the resulting analysis looks like:

That is, we can be 95% confident that, if the previous eruption lasted 4.8 minutes, then the time until the next eruption is between 71.969 and 98.801minutes. And, we can be 95% confident that, if the previous eruption lasted 3.5 minutes, then the time until the next eruption is between 58.109 and 84.734 minutes.

Let's do one of the calculations by hand, though. When the previous eruption lasted *x* = 4.8 minutes, then the predicted time until the next eruption is:

\[\hat{y}=33.828 + 10.741(4.8)=85.385\]

Now, we can use Minitab or a probability calculator to determine that *t*_{0.025,105} = 1.9828. We can also use Minitab to determine that *MSE* equals 44.6 (it is rounded to 45 in the above output), the mean duration is 3.46075 minutes, and:

\[\sum\limits_{i=1}^n (x_i-\bar{x})^2=113.835\]

Putting it all together, we get:

\[85.385 \pm 1.9828 \sqrt{44.66} \sqrt{1+\dfrac{1}{107}+\dfrac{(4.8-3.46075)^2}{113.835}}\]

which simplifies to this:

\[85.385 \pm 13.416\]

and finally this:

\[(71.969,98.801)\]

as we (thankfully) obtained previously using Minitab. Incidentally, you might note that the length of the confidence interval for *μ _{Y}* when

\[87.484-83.286=4.198\]

and the length of the prediction interval when *x* = 4.8 is:

\[98.801-71.969=26.832\]

Hmmm. I wonder if that means that the confidence interval will always be narrower than the prediction interval? That is indeed the case. Let's take note of that, as well as a few other things.

(1) For a given value *x* of the predictor variable, and confidence level (1−*α*), the prediction interval for a new observation *Y*_{n+1} is always longer than the corresponding confidence interval for the mean *μ _{Y}*. That's because the prediction interval has an extra term (

(2) The prediction interval for a new observation *Y*_{n+1 }can be made to be narrower in the same ways that we can make the confidence interval for the mean *μ _{Y }*narrower. That is, we can make a prediction interval for a new observation

(3) We cannot make the standard error of the prediction for *Y*_{n+1 }approach 0, as we can for the standard error of the estimate for *μ _{Y}*. That's again because the prediction interval has an extra term (

For any sizeable data set, and even for the small ones, you'll definitely want to use Minitab to calculate the confidence and/or prediction intervals for you. To do so:

(1) Under the **Stat** menu, select **Stat**, and then select **Regression**:

(2) In the pop-up window that appears, in the box labeled **Response**, specify the response and, in the box labeled **Predictors**, specify the predictor variable:

Then, click on the **Options...** button.

(3) In the pop-up window that appears, in the box labeled **Prediction intervals for new observations**, type the value of the predictor variable for which you'd like a confidence interval and/or prediction interval:

In the box labeled **Confidence level**, type your desired confidence level. (The default is 95.) Then, select **OK**.

(4) And, select **OK** on the main pop-up window. The output should appear in the session window. The first part of the output should look something like this:

while the second part of the output, which contains the requested intervals, should look something like this:

**Links:**

[1] https://onlinecourses.science.psu.edu/stat414/sites/onlinecourses.science.psu.edu.stat414/files/lesson36/OldFaithful.TXT