An introduction to the concept of the p-value, in the context of one-sample Z tests for the population mean. Much of the underlying logic holds for other tests as well. I discuss what a p-value is, then using simulation I illustrate its distribution when the null hypothesis is true and when the null hypothesis is false. I then give a rough guideline of what different p-values mean in terms of evidence against the null hypothesis.
2 thoughts on “6.6 What is a p-value?”
Can you elaborate a little on what exactly is a p-value distribution?
Is not the p-value a probability of X > sample-stat , for example?
I don’t understand how the p-value can be uniform, if these probabilities can areas under the standard normal curve.
What is the exact probability statement is the p-value?
By the way, your videos as excellent. I really appreciate your effort.
thank you for any time you have,
Mathematically showing that the p-value has a continuous uniform distribution under the null hypothesis (for a continuous test statistic) is a little too tricky for me to show here. The argument is very strongly related to the probability integral transformation, if you’d like to look that up. It’s something that might be covered in a first or second mathematical statistics course.
The exact probability statement depends on the test. Suppose we are carrying out a Z test on a single mean (Which would be appropriate if we were sampling from a normally distributed population where sigma is known.)
We will test the null hypothesis H_0: mu = mu_0. Let the random variable that is the test statistic be represented by Z, and let z_obs be the observed value of the test statistic.
Then for the alternative: H_a: mu > mu_0, p-value = P(Z >= z_obs | mu = mu_0)
For the alternative: H_a: mu < mu_0, p-value = P(Z <= z_obs | mu = mu_0) For the alternative: H_a: mu != mu_0, p-value = 2*P(Z >= |z_obs| | mu = mu_0) (there are other ways of expressing this one).