Course Outline
-
segmentGetting Started (Don't Skip This Part)
-
segmentStatistics and Data Science: A Modeling Approach
-
segmentPART I: EXPLORING VARIATION
-
segmentChapter 1 - Welcome to Statistics: A Modeling Approach
-
segmentChapter 2 - Understanding Data
-
segmentChapter 3 - Examining Distributions
-
segmentChapter 4 - Explaining Variation
-
segmentPART II: MODELING VARIATION
-
segmentChapter 5 - A Simple Model
-
segmentChapter 6 - Quantifying Error
-
segmentChapter 7 - Adding an Explanatory Variable to the Model
-
segmentChapter 8 - Models with a Quantitative Explanatory Variable
-
8.1 Groups versus Quantitative Explanatory Variables
-
-
segmentPART III: EVALUATING MODELS
-
segmentChapter 9 - The Logic of Inference
-
segmentChapter 10 - Model Comparison with F
-
segmentChapter 11 - Parameter Estimation and Confidence Intervals
-
segmentPART IV: MULTIVARIATE MODELS
-
segmentChapter 12 - Introduction to Multivariate Models
-
segmentChapter 13 - Multivariate Model Comparisons
-
segmentFinishing Up (Don't Skip This Part!)
-
segmentResources
list College / Advanced Statistics and Data Science (ABCD)
Chapter 8 - Models With a Quantitative Explanatory Variable
In the previous chapter we figured out how to add an explanatory variable to a model. We explained thumb length using sex, and using height (coded as short/tall, or short/medium/tall). We noticed two things: First, when we add more groups, we reduce left over error, explaining more variation in our outcome variable.
But, second, when we add more groups we use more degrees of freedom. If we keep going like this, we certainly will explain lots of variation in our data, but our explanation will not be very elegant. Imagine if we had a “group” for every individual in our sample. We would explain all the variability, but have a completely useless model in terms of understanding the Data Generating Process (DGP).
Let’s consider another approach: instead of using height in inches to divide students into groups (e.g., short or tall), what if we just model height as a continuous variable? Earlier in the course we learned how to visualize this approach in a scatterplot. In this chapter we will figure out how to extend our models to accommodate quantitative explanatory variables.
The models we develop in this chapter are usually called regression models. Before we start, though, note that the core ideas behind these new models are exactly the same as those we have developed for the grouping models. A model still yields a single predicted score for each observation, based on some mathematical function of the explanatory variable.
Further, in regression models, we still use residuals—the difference between the predicted and observed score to measure error around the model; and we still use the sum of squared deviations from the model predictions as a measure of model fit. And, we still use PRE to indicate the proportion reduction in error of the regression model compared with the empty model.
8.1 Groups versus Quantitative Explanatory Variables
The Two-Group Model: Review
Let’s start with our TinyFingers
data set. In the graph below, we illustrate the two-group approach to modeling this data. Just as a reminder, we would specify this model—the Height2Group
model—as a two-parameter model, with one parameter being the mean thumb length for short people, the other, the increment to add on for tall people.
We can represent this two-group model like this:
\[Y_{i}=b_{0}+b_{1}X_{i}+e_{i}\]
Or like this:
\[{Thumb}_{i}=b_{0}+b_{1}{Height2Group}_{i}+e_{i}\]
Although \(X_{i}\) is coded this way by R, you could, in theory, code it different ways. R has coded the first group (in this case, the short group) as 0 and the next group (the tall group) as 1. Doing this makes the interpretation of \(b_{1}\) straightforward: it is the estimated increment in thumb size from short people to tall people.
Note that the increment to add on for short people would be negative: by starting with \(b_{0}\) as the mean for tall people, \(b_{1}\) would represent the decrement when going from the mean for tall people to the mean for short people.
In general, the reason we code 0 for short and 1 for tall in this situation is simply that it makes the interpretation easier to keep in our brains! It’s easier to think of an increment than a decrement, for some reason. If we did decide to switch the coding, though, we would probably want to reverse the order in the graph to make the graph more consistent with the model.
We are comparing the Height2Group
model to the empty model, which is represented in the graph above with the blue horizontal line that goes all the way across the plot. The predicted score for everyone under the empty model is the Grand Mean.
The SS Total for the Height2Group
model is the sum of squared deviations from the empty model. The SS Error for the Height2Group
model is the deviation of each score from its group mean (Short
or Tall
).
We can see just by inspecting the graph above that the two-group model will yield better predictions than the empty model. Knowing whether someone is short or tall would help us make a more accurate prediction of their thumb length than if we didn’t know which group they were in (in which case we would just use the empty model).
The Continuous Model
Now let’s compare what happens when we use Height as a quantitative variable instead of as a grouping variable.
The scatterplot below (right) shows Height
on the x-axis measured in inches (the explanatory variable) and thumb length measured in millimeters on the y-axis (the outcome variable). We have included the Height2Group
plot on the left, for comparison.
The same six data points are represented in the graph on the left (which uses Height2Group
as the explanatory variable) as in the graph on the right, (which uses Height
in inches). Whereas the points on the left are divided into two groups, the points on the right are spread out continuously over the x-axis, with each person’s height and thumb length represented as a point in two-dimensional space.
The empty model can be represented the same way in both graphs, by a horizontal line through the Grand Mean.
The empty model of the outcome variable is exactly the same no matter how you code the explanatory variable: it is just the mean of the six data points. Sum of squared deviations around the empty model, thus, is the same in both cases.
We get a sense from the scatterplot that knowing someone’s exact height would help us make a better prediction of their thumb length than just knowing whether they are short or tall. But where, in the scatterplot, is the model?
When the explanatory variable is categorical, as with Height2Group
, we can use the group means as our model (as reviewed above). But when the explanatory variable is quantitative, as with Height
in inches, we can’t use the group means to predict new scores because there are no groups!