In simple terms, effect size is the quantifiable differences between two population parameters. While frequently overlooked in many statistical readings, effect size can provide powerful insight into understanding the effectiveness of particular products or methodologies beyond capabilities of simple hypothesis testing, particularly because of its independence from sample size.
Statistics are used every day when analysing experiments, relating results to research and understanding some of the patterns that exist around us. Many statistical results report on whether their findings are significant, meaning if their results are due to an underlying phenomena as opposed to have being caused by pure chance. While it is powerful to understand whether the patterns, we observe are due to luck or not, something that is commonly overlooked is the extent of the patterns that we are observing. This is referred to as the effect size.
Effect size aims to measure the magnitude of the relationships that exist in the data. While statistical significance can allow one to determine whether a relationship does exist, effect size is what measures the extent of this relationship.
As there are many different types of relationships one could be interested in understanding, there exist many effect size measures as well. Each with their own appropriate contexts, interpretations and limitations. This paper aims to introduce a handful of common effect measures to familiarise one with what they are, and how they work.
From this paper, there is also a brief literature review in which the statistical practices were examined from a variety of non-numerical fields were observed. This was aimed to observe how non-statisticians handle data and discuss their results. In short, it was found that whilst significance was very often reported on, effect size measures were often not reported on, or was poorly reported on, indicated a lack of understanding of the basis of the measure.
Furthermore, in improving understandings of effect size, a simulation study underwent in which the behaviours of effect size measures are observed as their sample sizes increase. This showed that whilst effect size measures are thought of to be independent of sample size, in reality larger sample sizes are more able to accurately estimate the population parameter.
This is an interesting insight, as not only did a larger sample size have a higher rate of accurate estimations, but that a power relationship existed between the two. The relationship found is in the form of an inverse square relationship, and thus any increase in the sample size lead to much lower variance across the estimated effect sizes. This can imply that the larger the sample size, the tighter the estimates bounds around the population’s true parameter is.
The University of Newcastle