Sampling a Population

The average height of a female US army recruit

Lets think about the concrete example of the HEIGHT.CSV data set.

Say we want to answer the question:

What is the mean height of female US army recruits?

The population
The population is the set of heights ofall womenwho have joined the US army.

The sample
We’ll use the data from 2010 (dataframehumanF_2010) as a sample. These data comprise 116 height measurements of army recruits. Whilst this is a decent amount of data it is still a small fraction of all the female recruits who have ever joined the US army.

The sample gives uspartial informationabout the population.

Estimating the mean of a population

If you had to guess how to estimate the mean of a population you may try using the mean of the sample. Congratulations! This would be a great guess. There is even statistical theory that supports using the mean of a sample.

Using the mean of our sample from 2010 gives us an answer to the question above:

Female US army recruits are 1.628 m tall on average.

But… we could have used the data from 2011 as the sample (dataframehumanF_2011).

Using the mean of data from 2011 gives us another answer to the question above:

Female US army recruits are 1.632 m tall on average.

We have two different answers! Which is the right answer?

The answer is that both answers are correct, allowing for someuncertainty.

Estimating uncertainty in the mean

Our estimates of the population mean have some uncertainty because the sample only givespartial informationabout the population.

To quantify the uncertainty in the estimated mean of the population mean we could

1. take several samples,
2. calculate the mean of each
3. calculate the spread of these means. This spread will be a measure of uncertainty. Two common measures of uncertainty are:
   • thestandard error of the mean: the standard deviation of the estimated means
   • the95% confidence interval: the 2.5% and 97.5% quantiles of the estimated means

There’s a problem: sampling the population several times is nearly always going to be impractical and costly.

Thankfully there are solutions: methods have been developed to quantify uncertainty from a single sample. Here are two approaches.

Method 1: Bootstrapping

Bootstrapping is a very powerful technique that was originally proposed byEfron (1979). The idea is very simple: rather than resampling the population you resample the data you have, and then calculate the uncertainty in the means from these resamples. This approach turns out to be surprisingly good.

Bootstrap algorithm to estimate the uncertainty in a population's mean
===================================================

1. Resample the data set with replacement
2. Calculate the mean of the resampled data
3. Repeat steps 1 and 2 many times (e.g. 1000 times)
4. The standard deviation of the reampled means is an estimate of the standard error of the mean

Aside: There are several variation on the bootstrapping algorithm. The above algorithm is the percentile method. Puthet al.(2015)discuss the different bootstrap algorithms commonly used.

One way to bootstrap in R is to use thesample()function.

# Resample (with replacement) the numbers 1 to 10 
sample(c(1:10), replace=TRUE)

 [1] 6 2 3 1 7 1 1 2 4 8

There are 10 numbers that we are sampling . Thissample()command generates a new set of 10 numbers by randomly picking values from the original data. Thereplace=TRUEargument means that a single observation can be picked more than once (you can see that some numbers appear more than once in the resampled data). This is very important. Without thereplace=TRUEyour resample would be the same as the original data, which we don’t want.

# Resample (without replacement) the numbers 1 to 10 
sample(c(1:10), replace=FALSE)

 [1]  3  6  9  8 10  4  1  2  5  7

We can use thissample()command to generate a simple bootstrap estimate for the uncertainty in estimating the population mean. We will resample the 2010 data 10000 times.

# Code to perform a simple bootstrap for an estimate of the mean of a population

# Create a variable that will contain the means of the resampled data
resample_mean87 = array(NA, dim=10000)

# Use a for loop to resample the data 10000 times using 2010 data
for (i in 1:10000) {
  resample = sample(humanF_2010$HEIGHT, replace=T)
  # Save the mean of the resampled data in the array
  resample_mean87[i] = mean(resample)
}

# Calculate the standard deviation of the resampled means
sd(resample_mean87)

[1] 0.005537942

Our bootstrap estimate for the standard error of the mean is 0.0055 m. Using this result we can now say that:

Female US army recruits are 1.628 +/- 0.006 m tall, on average.

We can do the same for the sample from 2011, which gives

Female US army recruits are 1.632 +/- 0.002 m tall, on average.

Now that we’ve estimated the standard error on the mean of our two means we can see that they are consistent to within the uncertainty.

As a rule of thumb: two estimates are consistent if the estimate’s uncertainties overlap (uncertainty must be measured by a 95% confidence interval or twice the standard error)

In this case, twice the standard error gives the ranges 1.617 - 1.639 m and 1.628 - 1.636 m. These two ranges overlap, indicating consistency in our estimates of the population.

# A histogram of the resampled means 
hist(resample_mean87, n=50)

# Calculate standard error of the mean from 2010 
# using equation from statistical theory

n = nrow(humanF_2010)            # The number of observations in humanF
sd(humanF_2010$HEIGHT) / sqrt(n) # Formula for the standard error of the mean

[1] 0.005537021