Skip to Main Content

Hypothesis Testing: Maths and Stats

Fast facts

  • First, consider your research question, what it is exactly  you are testing for. From that, determine your null and alternative hypotheses.
  • Identify what types of variables you have and what your significance level should be.
  • Test the assumptions to determine whether you need a parametric or non-parametric test, and find the right one to use.
  • Compare the p-value to the significance level to see whether you need to reject or fail to reject the null hypothesis.
  • Link the results back to your research question.

Hypothesis Testing

Hypothesis testing is the statistical process of using evidence and reasoning to draw conclusions about data, by making use of data samples. They involve creating null and alternative hypotheses which are always mutually exclusive.


Research Questions

When hypothesis testing, it is good to explicitly write out your research question (or questions) for your reference, so you can keep track of what it is exactly you want to test for. From your research question(s), you can then form your null hypothesis (NH, or H0) and alternative hypothesis (AH, Ha or H1). 

  • The null hypothesis assumes that there exists no difference, association or relationship between the variables you have chosen to investigate. 
  • The alternative hypothesis assumes that that there does exist a difference, association or relationship between the variables you have chosen.

Bear in mind that each research question will need its own null and alternative hypotheses, so if you have four research questions to investigate you will need in total four null and four alternative hypotheses.

Then, you can collect the data you need in order to test your research question. You may end up needing to gather many different types of data and use different methods to do so, such as using questionnaires, interviews, observations, etc.


Significance Level

You need to decide on your significance level before continuing, which is written as α (pronounced 'alpha'). This is the threshold below which you can come to the conclusion that the null hypothesis becomes rejected or fails to be rejected. Typically, we take a significance level to be 5% or 0.05, however you can also choose α to be 1% or 0.01, or 10% or 0.1. It all is determined by how strong you wish your evidence to be.


Types of Variables

After this, you can think about what different types of variables you have. Are they all categorical, quantitative or a mixture of both? Identifying what kinds of variables you have to work with will help you to decide what statistical test is best to use.


Categorical Variables

Categorical variables represent groups of things and can take one of a fixed number of values. Ordinal variables are categorical variables with a set order to them (for example small, medium, big; or first, second, third, etc.). Nominal variables represent group names (for example names of places, brands, colours, species, etc.). Binary variables are data with strictly one of two mutually exclusive options (for example yes/no, heads/tails, left/right, etc.)


Quantitative Variables

Quantitative variables, as opposed to categorical variables, represent quantities of things. Continuous (or ratio) variables represent measures and can take non-integer values (for example weight, distance, speed, etc.). Discrete variables represent counts and can only take integer values (for example the number of fish in a pond, the number of cars passing through a junction, etc.)


Choosing the Right Test to Use



There are two types of tests you can choose from in order to test your hypotheses: parametric and non-parametric. Which you end up using will be determined by whether or not your data satisfies the following assumptions:

  • Independence - the variables involved are not related. 
  • Homogeneity of variance - the variance in each group involved in the test is similar among all variables.
  • Normality - the distribution of the data points of the variables are normally distributed. This can be determined visually using a histogram, or by using a test such as the Shapiro-Wilk test.

Note that the assumption of normality only applies to quantitative data.

If your data meets all of these assumptions then your data is parametric and you can use a parametric test - if one of these assumptions is violated, you will need to consider your options.

There are many different types of hypothesis tests you can do, and once you know what type of variables you have and have tested these assumptions, you can search for the type of test you need. 



Comparing the p-value to the significance level is how we decide whether to reject or fail to reject the null hypothesis: when the p-value is lower than α then we reject the null in favour of the alternative, and when the p-value is greater than or equal to α then we fail to reject the null. In other words:

p less than or equal to alpha implies we reject the null hypothesis


p greater than alpha implies we fail to reject the null hypothesis in favour of the alternative hypothesis

Note that we do not say that we accept the null hypothesis - we only reject or fail to reject. This is because a lack of evidence of something existing does not mean that it exists: accepting something implies certainty, whereas a statistical test is used to report a lack of uncertainty. Therefore, when we reject the null hypothesis we say that the result is not statistically significant whereas when we fail to reject the null hypothesis we say that the result is significant.


Drawing Conclusions

In order to properly report the results of the statistical test performed, you always need to link it back to the context - don't just say "we fail to/reject the null hypothesis" and leave it there! Think about what the results are telling you about your research question and what conclusions would be appropriate to draw.