Alternative effect size measurements corresponding to the two sample t-test

Fri, 05 Dec 2014 00:00:00 +0000

If interest is in comparing the means of two (normally distributed) samples it is common practise to perform a two-sample t-test and report the corresponding p-value. Nevertheless, it has been widely criticized that the p-value does not provide a measure for the magnitude of the mean effect (e.g., Browne (2010)). This report provides an overview of existing alternatives recently published in the scientific literature that provide a more meaningful measurement of the effect size. Browne (2010) introduced closed form equations to translate a significant t-test p-value and sample size into the probability of one treatment being more successful than another on a per individual basis. This term was afterwards denoted as win probability by Hayter (2013) and he demonstrated the interpretation as “what would happen if a single future observation were to be taken from either of the two treatments, with attention being directed towards which treatment would win by providing the better value.” In addition Hayter (2013) introduced the corresponding confidence interval as well as the odds of X being greater than Y. He further introduced the transformation into Cohens effect size and the corresponding confidence intervals.

You can find the pdf of a detailed article here

effect size | AKitsche Blog

Alternative effect size measurements corresponding to the two sample t-test