The t-test is used to find out if two data sets are significantly different from each other. The t-test can be used to compare samples before and after a treatment or to compare two different ways of treatments in order to understand if a specific treatment can lead to significantly improvements.
The confidence interval describes the range that may contain the true difference between means of the two groups. It's usally given for a 95% specific reliability level. The confidence interval can be considered as an margin of error around the most likely value. In the same way as the p-value, a confidence interval can be used to show statistical significance (null hypothesis value included or not), but in addition it shows also the range of the tested difference value.
Example of t-test comparing two groups x and y
x = c(6.6,5.3,2.2,2.3,7.1,1.6,6.1,2.0,2.5,3.7)
y = c(6.3,14.7,10.3,7.8,3.7,5.8,4.5,7.8,11.1,10.8)
t.test(x,y)
Welch Two Sample t-test
data: x and y
t = -3.4068, df = 15.045, p-value = 0.003889
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-7.054604 -1.625396
sample estimates:
mean of x mean of y
3.94 8.28
The p-value < 0.05 shows a strong evidence for a difference between data set x and y.
Instead of using the p-value, we can make the same conclusions using the confidence interval:
As the confidence interval of the true difference does not include 0, there is strong evidence that the means of x and y are different.
(95% CI and 0.05 p-value referring to the same significance level)
Welch's t-test: adjustment for differences in variance of the two groups. It corrects the number of degree of freedom (df).