Obtaining Superscripts to Affix to Means That Are Not
Significantly Different From Each Other

Gerard E. Dallal, PhD

[To explain the concepts in a straightforward manner, I'm being a bit loose with my language. I am using the word similar to indicate not shown to be different statistically or within sampling variability of each other.]

When statistical program packages report the results of a multiple comparisons procedure, the output is usually in the form of a list of pairwise comparisons along with an indication whether each comparison is statistically significant. When these results are summarized for publication, standard practice is to present a table of mean with various superscripts attached and a comment such as,

This procedure is widely used. Nevertheless, at the time of this writing (November 2007; the last version was written in 2003 and, before that, March 2000!), none of the major statistical packages--SAS, SPSS, SYSTAT--provides the superscripts automatically. The analyst must deduce them from the table of P values. The one exception is the MEANS statement of SAS's GLM procedure, which can be used only when the number of observations is the same for each group or treatment. Since the computer software refuses to do the work, the analyst is left to translate the list of pairwise differences into a set of superscripts so that those not judged different from each other share a superscript while those judged different do not have a superscript in common.

By way of example, consider a set of four groups--A,B,C,D--where A was judged different from B and B was judged different from D. A brute force approach might use a different superscript for each possible comparison, eliminating those superscripts where the pair is judged significantly different. There are six possible comparisons--AB, AC, AD, BC, BD, CD--so the brute force approach would start with six superscripts

Aabc Bade Cbdf Dcef ,

where the superscript a indicates that A & B are similar, the superscript b indicates that A & C are similar, and so on. The superscripts a and e would be eliminated--a because A & B were judged different and e because B & D were judged different. This leaves
Abc Bd Cbdf Dcf.

This is a true description of the differences between the groups, but it is awkward when you consider that the same set of differences can be written

Aa Bb Cab Da.

In both cases, A & B do not share a superscript, nor do B & D. However, every other combination does share a superscript. The second expression is much easier to interpret because

There is a straightforward way to obtain the simpler expression. A computer program to generate the superscripts is now available. The procedure takes sets of similar treatments and divides them if they contain pairs of treatments have been shown to be different.

(Never!) Attaching Superscripts To Singletons

Some researchers have attached unique superscripts to single means that are judged to be different from all other means. For example, suppose when comparing four treatment means,

Some researchers make the mistake of attaching a superscript to D, expressing the differences as
Aa Ba Ca Db
rather than the much preferable
Aa Ba Ca D.

I find superscripts affixed to a single mean to be the worst kind of visual clutter. They invite the reader to look for matches that don't exist. It's similar to reading an article that includes a symbol indicating a footnote and being unable to find the footnote! Without such superscripts, unique means stand unadorned and the absence of any superscript trumpets a mean's uniqueness. For this reason, I never use superscripts that would be attached to only one mean.

Copyright © 2000 Gerard E. Dallal