Occasionally, doctoral students are challenged on the validity of using parametric statistics to analyze summative scale scores. I’m referring to a scale score that is derived by averaging (or summing) many Likert-type survey questions to measure an underlying construct like “emotional intelligence” for example. So, for example, let’s say you have 10 survey questions measured on a 5-point Likert-Type scale like: 1=Strongly Disagree; 2=Disagree; 3=Neutral; 4=Agree, and; 5=Strongly Agree. The idea is that each survey question measures some facet of the underlying construct (e.g. emotional intelligence) and that by averaging all of the questions, you get a valid and reliable measure of emotional intelligence. Of course work needs to be done to establish validity and reliability, but for the sake of this discussion, let’s assume we have a valid and reliable instrument of this type.
Parametric statistics refers to statistical tests like the t-test, ANOVA, and linear regression analysis. Those statistical tests are called parametric because they are based upon an underlying probability distribution (e.g. the Normal Distribution) which has parameters (e.g. the mean and standard deviation). Sometimes, empirical data do not meet the strict assumptions of parametric tests (e.g. normal distribution, no outliers etc.). In that case, remedial steps are undertaken such as transformations of the data or use of non-parametric statistics. But, for the sake of this discussion, let’s assume the summative scale score and other data (e.g. independent or dependent variables to be compared with the summative scale score) meet the assumptions for the parametric statistical analysis. Personally, I believe it is appropriate to apply parametric statistics in this case. Apparently thousands of other researchers agree with me because you will see parametric statistics applied to summative scale scores in thousands of published articles in high caliber peer reviewed journals.
If I understand correctly, those who question or oppose the use of parametric statistics for analysis of summative scale scores are concerned that the scale scores, while measured on a “continuous measurement scale”, are not truly measured on an interval or ratio measurement scale, and technically, parametric statistics do assume interval or ratio measurement scales. I think their concern goes like this: The individual survey questions that make up the summative scale score are measured on an ordinal measurement scale. With ordinal measures, we can’t necessarily say that the quantitative difference between “Strongly Disagree” (for example) and “Disagree”, is the same quantitative difference between “Agree” and “Strongly Agree”. So, the question becomes, when you derive a summative scale score from multiple Likert-type survey questions, does the resultant summative scale score have the same problem as the Likert-type questions?
For example, taking the average of 10 Likert-type survey questions measured on a 5-point scale would produce a measurement with a “continuous measurement scale”, meaning it can take on fractional values like 1.13, 2.56…, 4.87. One might ask, is the difference between 1.00 and 2.00 the same as the difference between 3.00 and 4.00? I take the position that if there is a difference (i.e. the summative scale score is not truly interval or ratio), it is likely to be relatively insignificant. In other words, the results will still produce valid and meaningful results most of the time (in my opinion). I think that averaging out over multiple survey questions kind of smooths that issue out, making it a relatively small concern. Perhaps the worst case scenario is, report this potential threat to validity as a limitation of the study.
