Chapter 8: Introduction to linear regressionContent check — Statistics (OpenIntro), Chapter 8: Introduction to linear regression

If a model underestimates an observation, then the model estimate is below the actual. The residual, which is the actual observation value minus the model estimate, must then be positive. The opposite is true when the model overestimates the observation: the residual is negative.

($+$⁠) First compute the predicted value based on the model:

$$\begin{array}{c}\hfill {\hat{y}}_{+}=41+0.59x_{+}=41+0.59\times 85.0=91.15\end{array}$$

Then the residual is given by

$$\begin{array}{c}\hfill e_{+}=y_{+}-{\hat{y}}_{+}=98.6-91.15=7.45\end{array}$$

This was close to the earlier estimate of 7.

($△$⁠) ${\hat{y}}_{△}=41+0.59x_{△}=97.3$⁠. $e_{△}=y_{△}-{\hat{y}}_{△}=-3.3$⁠, close to the estimate of -4.

We’ll leave it to you to draw the lines. In general, the lines you draw should be close to most points and reflect overall trends in the data.

The residual plot will show randomly distributed residuals around 0. The variance is also approximately constant.

The residuals will show a fan shape, with higher variability for smaller $x$⁠. There will also be many points on the right above the line. There is trouble with the model being fit here.

Strong relationship, but a straight line would not fit the data.

Strong relationship, and a linear fit would be reasonable.

Weak relationship, and trying a linear fit would be reasonable.

Moderate relationship, but a straight line would not fit the data.

Strong relationship, and a linear fit would be reasonable.

Weak relationship, and trying a linear fit would be reasonable.

Exam 2 since there is less of a scatter in the plot of final exam grade versus exam 2. Notice that the relationship between Exam 1 and the Final Exam appears to be slightly nonlinear.

Exam 2 and the final are relatively close to each other chronologically, or Exam 2 may be cumulative so has greater similarities in material to the final exam. Answers may vary.

$r=-0.7$ $\to$ (4).

$r=0.45$ $\to$ (3).

$r=0.06$ $\to$ (1).

$r=0.92$ $\to$ (2).

The relationship is positive, weak, and possibly linear. However, there do appear to be some anomalous observations along the left where several students have the same height that is notably far from the cloud of the other points. Additionally, there are many students who appear not to have driven a car, and they are represented by a set of points along the bottom of the scatterplot.

There is no obvious explanation why simply being tall should lead a person to drive faster. However, one confounding factor is gender. Males tend to be taller than females on average, and personal experiences (anecdotal) may suggest they drive faster. If we were to follow-up on this suspicion, we would find that sociological studies confirm this suspicion.

Males are taller on average and they drive faster. The gender variable is indeed an important confounding variable.

There is a somewhat weak, positive, possibly linear relationship between the distance traveled and travel time. There is clustering near the lower left corner that we should take special note of.

Changing the units will not change the form, direction or strength of the relationship between the two variables. If longer distances measured in miles are associated with longer travel time measured in minutes, longer distances measured in kilometers will be associated with longer travel time measured in hours.

Changing units doesn’t affect correlation: $r=0.636$⁠.

There is a moderate, positive, and linear relationship between shoulder girth and height.

Changing the units, even if just for one of the variables, will not change the form, direction or strength of the relationship between the two variables.

In each part, we can write the husband ages as a linear function of the wife ages.

$ag{e}_H=ag{e}_W+3$⁠.

$ag{e}_H=ag{e}_W-2$⁠.

$ag{e}_H=2\times ag{e}_W$⁠.
Since the slopes are positive and these are perfect linear relationships, the correlation will be exactly 1 in all three parts. An alternative way to gain insight into this solution is to create a mock data set, e.g. 5 women aged 26, 27, 28, 29, and 30, then find the husband ages for each wife in each part and create a scatterplot.

Larger family incomes are associated with lower amounts of aid, so the correlation will be negative. Using a computer, the correlation can be computed: -0.499.

The trend appears to be linear, the data fall around the line with no obvious outliers, the variance is roughly constant. These are also not time series observations. Least squares regression can be applied to these data.

Compute the slope using the summary statistics from Figure 8.13:

$$\begin{array}{c}\hfill b_1=\frac{s_y}{s_x}R=\frac{\text{5,460}}{\text{63,200}}(-0.499)=-0.0431\end{array}$$

About $R^2={(-0.97)}^2=0.94$ or 94% of the variation is explained by the linear model.

Correlation: no units. Intercept: kg. Slope: kg/cm.

Over-estimate. Since the residual is calculated as $observed-predicted$⁠, a negative residual means that the predicted value is higher than the observed value.

There is a positive, very strong, linear association between the number of tourists and spending.

Explanatory: number of tourists (in thousands). Response: spending (in millions of US dollars).

We can predict spending for a given number of tourists using a regression line. This may be useful information for determining how much the country may want to spend in advertising abroad, or to forecast expected revenues from tourism.

Even though the relationship appears linear in the scatterplot, the residual plot actually shows a nonlinear relationship. This is not a contradiction: residual plots can show divergences from linearity that can be difficult to see in a scatterplot. A simple linear model is inadequate for modeling these data. It is also important to consider that these data are observed sequentially, which means there may be a hidden structure not evident in the current plots but that is important to consider.

First calculate the slope: $b_1=$ $R\times s_y/s_x=$ $0.636\times 113/99=$ $0.726$⁠. Next, make use of the fact that the regression line passes through the point $(\bar{x},\bar{y})$⁠: $\bar{y}=b_0+b_1\times \bar{x}$⁠. Plug in $\bar{x}$⁠, $\bar{y}$⁠, and $b_1$⁠, and solve for $b_0$⁠: 51. Solution: $\widehat{traveltime}=$ $51+$ $0.726\times distance$⁠.

$b_1$⁠: For each additional mile in distance, the model predicts an additional 0.726 minutes in travel time. $b_0$⁠: When the distance traveled is 0 miles, the travel time is expected to be 51 minutes. It does not make sense to have a travel distance of 0 miles in this context. Here, the $y$⁠-intercept serves only to adjust the height of the line and is meaningless by itself.

$R^2={0.636}^2=0.40$⁠. About 40% of the variability in travel time is accounted for by the model, i.e. explained by the distance traveled.

$\widehat{traveltime}=$ $51+$ $0.726\times distance=$ $51+$ $0.726\times 103\approx$ $126$ minutes. (Note: we should be cautious in our predictions with this model since we have not yet evaluated whether it is a well-fit model.)

$e_i=y_i-{\hat{y}}_i=168-126=42$ minutes. A positive residual means that the model underestimates the travel time.

No, this calculation would require extrapolation.

$\widehat{murder}=$ $-$ $29.901+$ $2.559\times poverty\%$⁠.

Expected murder rate in metropolitan areas with no poverty is -29. 901 per million. This is obviously not a meaningful value, it just serves to adjust the height of the regression line.

For each additional percentage increase in poverty, we expect murders per million to be higher on average by 2.559.

Poverty level explains 70.52% of the variability in murder rates in metropolitan areas.

$\sqrt{0.7052}=0.8398$⁠.

There is an outlier in the bottom right. Since it is far from the center of the data, it is a point with high leverage. It is also an influential point since, without that observation, the regression line would have a very different slope.

There is an outlier in the bottom right. Since it is far from the center of the data, it is a point with high leverage. However, it does not appear to be affecting the line much, so it is not an influential point.

The observation is in the center of the data (in the x-axis direction), so this point does not have high leverage. This means the point won’t have much effect on the slope of the line and so is not an influential point.

There is a negative, moderate-to-strong, somewhat linear relationship between percent of families who own their home and the percent of the population living in urban areas in 2010. There is one outlier: a state where 100% of the population is urban. The variability in the percent of homeownership also increases as we move from left to right in the plot.

The outlier is located in the bottom right corner, horizontally far from the center of the other points, so it is a point with high leverage. It is an influential point since excluding this point from the analysis would greatly affect the slope of the regression line.

We will provide two considerations. Each of these points would have very high leverage on any least-squares regression line, and years with such high unemployment may not help us understand what would happen in other years where the unemployment is only modestly high. On the other hand, these are exceptional cases, and we would be discarding important information if we exclude them from a final analysis.

We look in the second row corresponding to the family income variable. We see the point estimate of the slope of the line is -0.0431, the standard error of this estimate is 0.0108, and the $t$⁠-test statistic is $T=-3.98$⁠. The p-value corresponds exactly to the two-sided test we are interested in: 0.0002. The p-value is so small that we reject the null hypothesis and conclude that family income and financial aid at Elmhurst College for freshman entering in the year 2011 are negatively correlated and the true slope parameter is indeed less than 0, just as we believed in Example 8.21.

The relationship is positive, moderate-to-strong, and linear. There are a few outliers but no points that appear to be influential.

$\widehat{weight}=$ $-$ $105.0113+$ $1.0176\times height$⁠.
Slope: For each additional centimeter in height, the model predicts the average weight to be 1.0176 additional kilograms (about 2.2 pounds).
Intercept: People who are 0 centimeters tall are expected to weigh - 105.0113 kilograms. This is obviously not possible. Here, the $y$⁠- intercept serves only to adjust the height of the line and is meaningless by itself.

$H_0$⁠: The true slope coefficient of height is zero (${\beta }_1=0$⁠).
$H_A$⁠: The true slope coefficient of height is different than zero (${\beta }_1\ne 0$⁠).
The p-value for the two-sided alternative hypothesis (${\beta }_1\ne 0$⁠) is incredibly small, so we reject $H_0$⁠. The data provide convincing evidence that height and weight are positively correlated. The true slope parameter is indeed greater than 0.

$R^2={0.72}^2=0.52$⁠. Approximately 52% of the variability in weight can be explained by the height of individuals.

$H_0$⁠: ${\beta }_1=0$⁠. $H_A$⁠: ${\beta }_1\ne 0$⁠. The p-value, as reported in the table, is incredibly small and is smaller than 0.05, so we reject $H_0$⁠. The data provide convincing evidence that wives’ and husbands’ heights are positively correlated.

${\widehat{height}}_W=$ $43.5755+$ $0.2863\times heigh{t}_H$⁠.

Slope: For each additional inch in husband’s height, the average wife’s height is expected to be an additional 0.2863 inches on average. Intercept: Men who are 0 inches tall are expected to have wives who are, on average, 43.5755 inches tall. The intercept here is meaningless, and it serves only to adjust the height of the line.

The slope is positive, so $r$ must also be positive. $r=\sqrt{0.09}=0.30$⁠.

63.33. Since $R^2$ is low, the prediction based on this regression model is not very reliable.

No, we should avoid extrapolating.

$H_0:{\beta }_1=0;H_A:{\beta }_1\ne 0$

The p-value for this test is approximately 0, therefore we reject $H_0$⁠. The data provide convincing evidence that poverty percentage is a significant predictor of murder rate.

$n=20,df=18,T_{18}^{\ast }=2.10$⁠; $2.559\pm 2.10\times 0.390=$ $(1.74,3.378)$⁠; For each percentage point poverty is higher, murder rate is expected to be higher on average by 1.74 to 3.378 per million.

Yes, we rejected $H_0$ and the confidence interval does not include 0.

True.

False, correlation is a measure of the linear association between any two numerical variables.

The point estimate and standard error are $b_1=0.9112$ and $SE=0.0259$⁠. We can compute a T-score: $T=$ $(0.9112-1)/0.0259=$ $-$ $3.43$⁠. Using $df=168$⁠, the p-value is about 0.001, which is less than $\alpha =0.05$⁠. That is, the data provide strong evidence that the average difference between husbands’ and wives’ ages has actually changed over time.

${\widehat{age}}_W=$ $1.5740+$ $0.9112\times ag{e}_H$⁠.

Slope: For each additional year in husband’s age, the model predicts an additional 0.9112 years in wife’s age. This means that wives’ ages tend to be lower for later ages, suggesting the average gap of husband and wife age is larger for older people. Intercept: Men who are 0 years old are expected to have wives who are on average 1.5740 years old. The intercept here is meaningless and serves only to adjust the height of the line.

$R=\sqrt{0.88}=0.94$⁠. The regression of wives’ ages on husbands’ ages has a positive slope, so the correlation coefficient will be positive.

${\widehat{age}}_W=$ $1.5740+$ $0.9112\times 55=$ $51.69$⁠. Since $R^2$ is pretty high, the prediction based on this regression model is reliable.

No, we shouldn’t use the same model to predict an 85 year old man’s wife’s age. This would require extrapolation. The scatterplot from an earlier exercise shows that husbands in this data set are approximately 20 to 65 years old. The regression model may not be reasonable outside of this range.

There is an upwards trend. However, the variability is higher for higher calorie counts, and it looks like there might be two clusters of observations above and below the line on the right, so we should be cautious about fitting a linear model to these data.

$r=-0.72\to (2)$

$r=0.07\to (4)$

$r=0.86\to (1)$

$r=0.99\to (3)$