10.2 Using Bootstrapping to Construct a Confidence Interval

By now you should have a sense of what a sampling distribution is, and some intuition as to why we need sampling distributions to construct confidence intervals. Most of our ideas about sampling distributions have been developed through simulations. Although simulations help us to understand how all this works, they may not be as practical in a data analysis situation.

Simulations require us to make some assumptions about the DGP. For example, we have assumed in previous sections that the population distribution is normal in shape, and that it has a particular standard deviation, which we have estimated from our sample. We generated sampling distributions based on these assumptions. But not all DGPs are normal, so there are going to be plenty of situations where we don’t want to make this assumption.

The rise of cheap and fast computers has made popular an alternative approach to creating sampling distributions. This approach is called resampling, or bootstrapping. One cool thing about resampling techniques is that they use only the data you collect in your sample. So, unlike simulation, they don’t require you to make up anything else.

How Bootstrapping Works

We have already used the resample() function in R. We used it previously to create a sampling distribution of the means of samples of 24 die rolls. We created a vector with the numbers 1 to 6, then rigged up a DGP in which each number had an equal chance of being resampled.

Bootstrapping works the same way, but instead of making up the distribution to sample from, we resample from the actual data we have collected. Let’s start by explaining the process using our TinyFingers data set of six thumb lengths. Then we will apply the same techniques to our actual sample of 157 thumb lengths.

Here are the six thumb lengths from the TinyFingers data set again:

TinyFingers$Thumb

[1] 56 60 61 63 64 68

Let’s start by reviewing what the resample() command does.

resample(TinyFingers$Thumb, 6)

Here is the output of this command.

[1] 60 60 63 56 61 61

What resample() does here is take a new random sample of six observations from our data set of six observations. It samples with replacement, meaning that when it samples the first number, it then puts it back so it can be sampled again.

You can see that all of the numbers in the resampled data came from the original data set. But, some of the numbers in the original data were not selected (e.g. 68), while others (e.g. 60) were selected twice. It’s a random process, meaning each number has an equal chance of being selected each time a number is selected.

Bootstrapping a Sampling Distribution from our Fingers Data Set

Let’s now use resample() to bootstrap a sampling distribution of means we can use to help us interpret the mean thumb length we observed in our sample of 157 students.

Let’s start by creating just five bootstrapped sample means by running this code. We’ll save the bootstrapped means in a new data frame called bootSDoM. Note that all of the five means are based on our original data points.

Modify the code now to create a sampling distribution of 1,000 means, then plot the means of this bootstrapped sampling distribution as a histogram.

Comparing the Bootstrapped Sampling Distribution with the Simulated Sampling Distribution

The histogram of our bootstrapped distribution of 1,000 means certainly looks like a sampling distribution. It’s normal in shape, and is centered on our observed sample mean of 60.1. This makes sense given that all the numbers sampled to go into these means came from our sample with a mean of 60.1.

Let’s compare this bootstrapped distribution to a simulated distribution of 1,000 means centered on the same mean and based on estimates derived from the same sample of 157 students: a mean of 60.1 (Thumb.stats$mean) and a standard deviation of 8.73 (Thumb.stats$sd). Save the resulting sampling distribution in a data frame called simSDoM. (You have done this before; we are just asking you to do it again.)

Now compare the two sampling distributions of means of samples of n=157. On the left is our bootstrapped sampling distribution. On the right is the simulated sampling distribution.

Go ahead and get the favstats() for Fingers$Thumb (the original variable with 157 thumb lengths), bootSDoM$mean, and simSDoM$mean.

Using the Bootstrapped Sampling Distribution to Construct the 95% Confidence Interval

We have established that the bootstrapped sampling distribution is nearly identical to the simulated sampling distribution. Although we used different methods to create them, we end up in the same place. Just based on the fact that the two distributions have similar standard errors, we can guess that the confidence interval we would construct from the bootstrapped distribution would be roughly the same as what we got from the simulated distribution.

The code window below finds the confidence interval using the simulated sampling distribution centered at the sample mean. As before, we sort the means in descending order and then print the 25th largest mean and 975th largest mean. These correspond to the upper and lower bounds of the confidence interval.

Add code to find the confidence interval using the bootstrapped distribution. See how the confidence intervals compare using the two different distributions.

[1] 61.51474

[1] 58.81467

[1] 61.40159

[1] 58.66962

The two sets of confidence intervals are very close! When we use simulation, we get a confidence interval of 58.8 to 61.5, and when we use bootstrapping, we get a confidence interval of 58.7 to 61.4. The numbers you got with your random simulations and bootstrapping may be a little different, but are probably pretty close.