Hypothesis testing is the statistical process of using evidence and reasoning to draw conclusions about data, by making use of data samples. They involve creating null and alternative hypotheses which are always mutually exclusive.
When hypothesis testing, it is good to explicitly write out your research question (or questions) for your reference, so you can keep track of what it is exactly you want to test for. From your research question(s), you can then form your null hypothesis (NH, or H0) and alternative hypothesis (AH, Ha or H1).
Bear in mind that each research question will need its own null and alternative hypotheses, so if you have four research questions to investigate you will need in total four null and four alternative hypotheses.
Then, you can collect the data you need in order to test your research question. You may end up needing to gather many different types of data and use different methods to do so, such as using questionnaires, interviews, observations, etc.
You need to decide on your significance level before continuing, which is written as α (pronounced 'alpha'). This is the threshold below which you can come to the conclusion that the null hypothesis becomes rejected or fails to be rejected. Typically, we take a significance level to be 5% or 0.05, however you can also choose α to be 1% or 0.01, or 10% or 0.1. It all is determined by how strong you wish your evidence to be.
After this, you can think about what different types of variables you have. Are they all categorical, quantitative or a mixture of both? Identifying what kinds of variables you have to work with will help you to decide what statistical test is best to use.
Categorical variables represent groups of things and can take one of a fixed number of values. Ordinal variables are categorical variables with a set order to them (for example small, medium, big; or first, second, third, etc.). Nominal variables represent group names (for example names of places, brands, colours, species, etc.). Binary variables are data with strictly one of two mutually exclusive options (for example yes/no, heads/tails, left/right, etc.)
Quantitative variables, as opposed to categorical variables, represent quantities of things. Continuous (or ratio) variables represent measures and can take non-integer values (for example weight, distance, speed, etc.). Discrete variables represent counts and can only take integer values (for example the number of fish in a pond, the number of cars passing through a junction, etc.)
There are two types of tests you can choose from in order to test your hypotheses: parametric and non-parametric. Which you end up using will be determined by whether or not your data satisfies the following assumptions:
Note that the assumption of normality only applies to quantitative data.
If your data meets all of these assumptions then your data is parametric and you can use a parametric test - if one of these assumptions is violated, you will need to consider your options.
There are many different types of hypothesis tests you can do, and once you know what type of variables you have and have tested these assumptions, you can search for the type of test you need.
Comparing the p-value to the significance level is how we decide whether to reject or fail to reject the null hypothesis: when the p-value is lower than α then we reject the null in favour of the alternative, and when the p-value is greater than or equal to α then we fail to reject the null. In other words:
and
Note that we do not say that we accept the null hypothesis - we only reject or fail to reject. This is because a lack of evidence of something existing does not mean that it exists: accepting something implies certainty, whereas a statistical test is used to report a lack of uncertainty. Therefore, when we reject the null hypothesis we say that the result is not statistically significant whereas when we fail to reject the null hypothesis we say that the result is significant.
In order to properly report the results of the statistical test performed, you always need to link it back to the context - don't just say "we fail to/reject the null hypothesis" and leave it there! Think about what the results are telling you about your research question and what conclusions would be appropriate to draw.
