What is hypothesis testing?
Hypothesis testing in statistics is a way to test sample(s) with a population parameter. You can use hypothesis testing to test the data whether the data you gathered is a meaningful result. More specifically, how probable is your sample data compared to population parameters or to other samples.
The null hypothesis, (\(H_0\)), represents the status quo. Meaning the evidence collected is not strong enough to warrant a change.
The alternative hypothesis, (\(H_1\) or \(H_A\)), represents what is required to make a change to the status quo. That means there is statistical evidence using probability that it is different from the status quo.
For example, you believe your internet is slower than the average. You decide to collect the data on your connection speed every day for a month. The null hypothesis would be that there is no significant difference between your connection speed and the average. And the alternative hypothesis would be that your internet is slower.
So what is considered convincing evidence? To consider what evidence is convincing we use a test statistic to decide whether to reject the null hypothesis or not. The distribution is divided up into two regions, one to reject the null hypothesis and one that remains the same. These regions are determined by the significance level or critical value.
The test statistic depends on the distribution. For normal or approximately normal distributions, we can use the z- or t-statistics.
The significance level or critical value, splits the rejection region from the non-rejection region. For example, a significance level \(\alpha =0.05) in a two-tailed test for a normal approximation results in a critical value of 1.96 (found using z-table).
Two-tailed test: The diagrams above represents a two-tailed hypothesis test. This means that there is a rejection region at the two extreme ends of the distribution. The null and alternative hypothesis is stated as the following when comparing it to the population mean:
\begin{align} H_0 &= \mu \\ H_1 &\neq \mu \end{align}
One-tailed test:
There are two different one-tailed tests. One where the rejection region is in the upper extreme of the distribution and the other the lower extreme:
The upper one-tailed test is usually stated:
\begin{align} H_0 &\leq \mu \\ H_1 &> \mu \end{align}
The lower one-tailed test:
\begin{align} H_0 &\geq \mu \\ H_1 &< \mu \end{align}
In some text, the \(\leq\) and \(\geq\) signs are replaced with the \(=\) sign.
Making a decision. The decision during hypothesis testing is whether to keep the status quo or there is evidence to consider the alternative hypothesis. If the test statistic falls inside the rejection region, then there is evidence to reject the status quo. This does not mean that the alternative hypothesis is true population parameter, it just means there is evidence that this sample is different from the population parameter.
Example 1: A formal hypothesis test is to be conducted using the claim that the mean height of men is equal to 174.1 cm. What is the null and alternative hypothesis?
Solution: Since the test is to see if the population mean is equal to 174.1 cm. Any mean that is significantly higher or lower will reject the claim. This is a two-tailed test because there are rejection regions both in the upper and lower extremes.
The null hypothesis or status quo is that the mean height of men is equal to 174.1 cm.
\[H_0=174.1\,cm\]
The alternative hypothesis is that there is evidence that there is a mean height statistically higher or lower than 174.1 cm.
\[H_1\neq 174.1\,cm\]
Example 2: Fewer than 95% of adults have a cell phone. In a poll of 1128 adults, 87% said they have a cell phone.
Solution: In this example, the status quo is that fewer than 95% of adults have a cell phone. The sample of 1128 adults shows a mean that may is lower. You want to test if this sample mean is significantly lower to warrant a change to the population mean of 95%.
You are performing a lower one-tailed test because you want to show this sample is significantly lower. A value significantly higher does not challenge this claim for this test. Therefore, the null and alternative hypothesis are (note that the population mean, 95%, is used and not the sample mean):
\begin{align} H_0 &\geq 95\% \\ H_1 &< 95\% \end{align}
Example 3: A survey of online reports that 68% of college administration believe that their online education courses are as good as or superior that utilize traditional face-to-face instruction. You collect a survey that shows that 75% of college administration makes such a claim.
Solution: You are testing that your sample is showing a significant increase to the original claim. This is an upper one-tailed test since the rejection region of the claim will be values that are statistically higher.
\begin{align} H_0 &\leq 68\% \\ H_1 &> 68\% \end{align}
The significance level \(\alpha\) for a hypothesis test is the probability used as the cutoff for what constitutes significant evidence against the null hypothesis.
For example, a significance level of \(\alpha=0.05\) in an upper one-tailed test in a normal distribution represents the following rejection region
The critical value of \(z=1.645\) was found using the z-table looking up the top 5% of the normal distribution.
However, a significance level of \(\alpha=0.05\) during a two-tailed test means that the upper and lower rejections regions total 5%. That means on each side there is a rejection region of 2.5% or 0.025.
The critical value \(z=\pm 1.96\) was found by looking for the upper and lower 2.5% of the z-table.
Note that determining if it is a one-tailed or two-tailed test when setting up the alternative hypothesis is important to setting up the rejection region and finding the critical value.
You may think "Why don't I just the rejection region as large as possible, so I can verify that it is different?" Or "Let's make the rejection region as small as possible, so the status quo remains." This is not the best method to claim something is statistically different because of the error that is created. There are two types of errors: Type I and Type II.
Type I error: The mistake of rejecting the null hypothesis when it is actually true. \(\alpha\), same as the significance level, is the type I error. So \(\alpha=0.05\) means that there is a 5% chance of rejecting the status quo, when in fact the status quo was the right decision.
Type II error: The mistake of failing to reject the null hypothesis when it is actually false. Type II error is represented by the symbol \(\beta\) and can be calculated using the Power of a Test.
As a result, there are four outcomes possible. Two of those outcomes are correct, and two are errors.
\(\alpha\) and \(\beta\) are inversely related (as one increases, the other decreases). Therefore, it is impossible to reduce both errors to 0. In an actual study, you will be choosing your own significance level, so keep type I and I errors in mind.
Example 1: There is a medicine that claims to cure a disease. Describe the type I and II errors.
| Null Hypothesis | \(H_0\) is true | \(H_0\) is false |
| Medicine cures the disease | The drug works and is not rejected | Type I error: the medicine cures the disease but the drug is rejected |
| Medicine does not cure the disease | Type II error: the medicine does not cure the disease but is not rejected | The drug does not work and is rejected. |
Example 2: You decide to attend an online lecture because it will discuss a topic you are not familiar with.
| Null Hypothesis | \(H_0\) is true | \(H_0\) is false |
| You attend lecture | You attend lecture and it contains something you do not know. | Type I error: you attend the lecture but the lecture does not discuss the topic you do not know. |
| You do not attend lecture | Type II error: you miss the lecture but the topic you are not familiar with was discussed. | You miss the lecture and the topic was not discussed. |
Each possibility contains a probability value depending on the information provided.
Statistics by Matthew Cheung. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.