Overlapping Confidence Intervals

Perhaps you’ve seen a claim like this in an applied paper: “the estimated effect for Group A is statistically significant, but the estimated effect for Group B is not; this treatment helps As but not Bs.” But this reasoning is flawed . To see why, consider the following example from Gelman & Stern . We have data from two independent samples: Group A and Group B. For Group A our estimated effect is 25 with a standard error of 10, yielding an approximate 95% confidence interval of \(25 \pm 20\) or