Skip to Main Content

Sample Sizes: Maths and Stats

Overview of Sample Sizes

One of the important decisions we need to make when hypothesis testing is determining how much data we need in order to accurately make assumptions about a population. Oftentimes, it is not feasible, practical or worthwhile to test a whole population due to limitations such as time, money, feasibility, etc. and so we need to instead use a smaller sample...but not too small: if the sample is too small the study may produce unreliable results. Taking samples of our populations to test is still a perfectly fine way of drawing conclusions about the population, and precise sampling techniques are required to calculate the ideal sample size in specific research scenarios as one technique may not be as appropriate to use compared to another.

Sampling involves simply taking number of observations from a larger population in order to estimate the characteristics of that whole population. One very important aspect is deciding just how large your sample should be - if it is too small it would not be representative of the whole population, and if it is too big the study comes too complex - and there are two main ways to do this: the precision-based method, and the power-based method.

Sample size calculations can be performed on pretty much all quantitative studies, and should be done before you gather your data for your analyses. Of course, when it comes to actually performing statistical analyses, you have to work with what you have - so make sure what you have is the best to work with!

Guide contents

The tabs of this guide will support you in understanding sample sizes. The sections are organised as follows:

  • Precision-Based Sample Sizing - with what precision is it important to you to estimate the proportion, mean difference, etc?
  • Power-Based Sample Sizing - with what degree of certainty is it important to you to estimate the proportion, mean difference, etc?
  • Using Software - examples on how to use various statistical softwares and languages to perform different sample size calculations.

You can use online sample size calculators to calculate the required sample size, however this guide will assist you in understanding these calculations and how to perform them yourself, both by hand and using software.

Using the Confidence Intervals

When we use precision-based sample sizing, we are wanting the estimation to have a certain degree of precision, and for this we make use of the confidence intervals.

Recall that the formula for the 95% confidence interval for normally distributed data is given by 

95 per cent confidence interval is equal to x bar plus minus z multiplied by s over the square root of n

where:

  •  x bar denotes the sample mean 
  • z  is the confidence level value 
  • s is the sample standard deviation
  • n is the sample size 

 

Therefore, if you specify the 95% confidence interval you have, this formula can be rearranged to find the sample size n.

Cochran's formula

You can use the Cochran formula to calculate sample sizes at the 95% confidence interval when your population is very large, however a modification can be made to the calculation when your population is smaller.

 

Large Populations

The formula for Cochran is given as:

n equals Z squared multiplied by p multiplied by one minus p all divided by e squared

where:

  • n is the minimum sample size
  • Z is the Z-score (can be found from the Z-table
  • p is the estimated ratio of the population having the desired attribute (if you are unsure of this, you can check other studies which are similar to yours to see what other researchers have put. A common value is p=0.5)
  • e is the margin of error

 

Be aware that the Cochran formula will sometimes not give a whole number - you will need to round up to find the real minimum sample size you should use.

 

Smaller Populations

If the population is smaller we need to use a slightly different formula to calculate the appropriate sample size:

n equals n zero divided by the entirety of one plus n zero minus one divided by large N

where:

  • n is our sample size to be found
  • n0 is the Cochran sample size recommendation (calculated using the formula for large sample sizes above)
  • N is the population size

 

Again, this calculation may not provide an integer answer so would need to be rounded up if this were to happen.

Yamane's Formula

Yamane's formula is another way of calculating the sample size at 95% confidence. It is given by:

n equals the fraction N over one plus N e squared

where:

  • n is the sample size to be found
  • N is the population size
  • e is the margin of error

Using Power

Whereas precision-based sample size estimating makes use of confidence intervals, power-based makes use of power, which is calculated by:

Power equals one minus beta

where β is the probability that the null hypothesis is rejected when it is false. In other words, the power is the probability of you not making a type II error.

It is important to measure the power of your study as a higher power implies a higher likelihood that your results are valid - a study with low power (for example, a value of less than 0.8) implies that your test has a only a small chance of producing valid results, and these may be influenced by random or systematic error.

Since both power and sample size are positively related. Power can also be used to determine appropriate sample sizing: if you know the power, significance level, research design and effect size, you can calculate the best size for your sample. 

 

Continuous Outcomes

It is very easy to use power to calculate sample size when using quantitative data, however the type of test performed affects the way we need to calculate the sample size. For the example of the unpaired t-test, we can use the formula:

n equals function of alpha and beta multiplied by 2 s over delta squared

where:

  • n is the sample size we are looking for
  • α is the significance level
  • β is the probability that the null hypothesis is rejected when it is false, β=1-Power
  • δ is the smallest difference in means you regard as important to be able to detect
  • s is the standard deviation
  • f(α, β) is the value calculated by the table below:
α, β 0.05 0.1 0.2 0.5
0.05 13.0 10.5 7.9 3.8
0.1 17.8 14.9 11.7 6.6

 

Using Software

Using software to perform complex calculations is desirable as it tends to save time and reduces the risk of human error. Below are some examples of how to calculate sample sizes for different tests using different softwares and languages.

Minitab

Minitab uses power-based sample sizing to calculate the right sample size to use. To choose the sample size for a t-test:

  • Click Stat 
  • Select Power and Sample Size
  • Select the type of t-test you are performing (for example, 2-sample t for a paired-samples t-test)
  • In the Differences section, enter the difference value required to detect 
  • In the Power Values section, enter the desired power (this tends to be 0.8 or 0.9)
  • In the Standard Deviation section, enter the required standard deviation for your test
  • Click OK

 

To calculate the sample size for an ANOVA:

  • Click Stat 
  • Select Power and Sample Size
  • Select the type of ANOVA you are performing (for example, one-way ANOVA)
  • In the Number of levels section, enter the number of different values or categories (for example, the number of treatments) 
  • In the Values of the Maximum Difference Between Means section, enter the required value
  • In the Power Values section, enter the desired power (this is commonly 0.8 or 0.9)
  • In the Standard Deviation section, enter the required standard deviation for your test
  • Click OK

The important thing with Minitab is to leave the desired field blank - so here, it is sample size. If, for example, you wish to calculate the power instead, you can leave this blank and fill in the other fields as appropriate.

SPSS

To calculate the required sample size for Pearson's correlation in SPSS:

  • Click Analyze at the top of SPSS
  • Click Power Analysis 
  • Select Correlations and then Pearson Product Moment
  • It is recommended that you click Reset and ensure the following:
    • For Estimate, have Sample Size selected
    • For Single Power Value, input the desired power (this is commonly 0.8 or 0.9)
    • For Pearson Correlation Parameter, enter the expected correlation/expected r-value of the variables you are using. If you are unsure of this, you can see what the correlation was for a previous study which is similar to yours
    • For Null Value, leave as 0
    • Check the Use Bias-Correlation Formula in the Power Estimate box
    • For Test Direction, have the required Nondirectional (Two-Sided) Analysis or Directional (One-Sided) Analysis box checked as appropriate. Unless you can justify that your test is one-sided, you should put two-sided for this
    • For Significance Level, enter the required significance level (alpha level) (this tends to be 0.05)
  • Click OK

The value for the sample size will appear in the table SPSS generates under the N column.

SAS

You can use PROC Power to calculate the desired sample size. Note that SAS defaults the significance level to 0.05 so if you need to change that you will need to override that default.

To calculate the sample size for a two-sample t-test, write the following and substitute the ____ with the required input as according to your study:

proc power;

twosamplemeans test=diff

groupmeans = ____

stddev = ____

npergroup = .

power = ____;

run;

Be aware that the value SAS generates will be the number of participants in each group, not the overall total sample size.

R

To calculate the sample size for a correlation analysis:

  • Create your variables and find the r-value using cor()
  • Write the following, substituting the ___ for the values you are using:

pwr -> pwer.r.test

pwr.r.test (r = ___, sig.level = ___, power = ___)

 

To calculate the sample size for a logistic regression, write the following, substituting the ___ for the values you are using:

wp.logistic(p0 = ___, p1 = ___, alpha = ___, power = ___, alternative = "two.sided", family = "normal"

where p0 is the probability of observing a value of 1 for the dependent variable when the independent variable is 0, and p1 is the probability of observing a value of 1 for the dependent variable when then independent variable is 1.

 

Bear in mind again that R may not provide integer values for these answers, so it may be the case that you will need to round up in order to find the right sample size.

G*Power

G*Power is a statistical software used to calculate power, however it can also be used to calculate the sample size of t-, z- and F-tests using power-based calculations.

To calculate the sample size for a multiple regression:

  • Select F tests in the Test Family drop down menu
  • Select Linear Multiple Regression: Fixed Model, R2 increase in the Statistical Test drop-down menu
  • Select A Priori: Compute Required Sample Size - Given α, power, and effect size in the Type of Power Analysis drop-down
  • Select Determine => if it has not been selected already, and input the following:
    • For Effect Size, input the required effect size
    • For α Error Prob, input the required significance level (alpha level) (this tends to be 0.05)
    • For Power, input the power level (this is commonly 0.8 or 0.9)
    • For Number of Tested Predictors, input the required number
    • For Total Number of Predictors, input the number of independent variables you are using
  • Click Calculate

 

To calculate the sample size for a paired samples t-test:

  • Select t tests in the Test Family drop down menu
  • Select Means: Difference Between Two Dependent Means (Matched Pairs) in the Statistical Test drop-down menu
  • Select A Priori: Compute Required Sample Size - Given α, power, and effect size in the Type of Power Analysis drop-down
  • Select Determine => if it has not been selected already, and input the following:
    • For Effect Size, input the required effect size
    • For α Error Prob, input the required significance level (alpha level) (this tends to be 0.05)
    • For Power, input the power level (this is commonly 0.8 or 0.9)
  • Click Calculate