In wine tastings, in which several tasters (judges) taste several wines, it is important to insure objectivity to the extent possible. This is usually accomplished by holding the tast- ing “blind,” i.e., covering the bottles so that the tasters do not know which wine is in which bottle. At some agreed upon point in the proceedings, the tasters reveal what they think about the various bottles. Ideally, this revelation would take place by secret ballot, lest a taster’s choices be influenced by what he or she hears another taster say. But in any event, there are two standard ways of rating the wines. The older method is to assign them “grades” on a scale of, say, up to 100 points (Parker) or up to 20 points as in the famous face-off between California wines and French Bordeaux wines in 1976 (see Ashenfelter et al., 2007). As Ashenfelter at al. point out, this has the distinct disadvantage that a judge with greater dispersion in his or her grades will have a greater influence on the average score that each wine achieves.

A preferable method for rating the wines is to rank them, i.e., rank the most favored wine “1”, the second most favored wine “2”, etc. The winner, that is to say the wine that is liked “on the average” best by the group, is the one that achieves the lowest rank total. Of course, there will always be a wine that has a lower rank total than the other wines (with the proviso that there may be more than one wine tied for the lowest rank total) and we need to know whether this lowest rank total could have occurred by chance, even if there were no difference among the wines. Fortunately, a statistical significance test exists for the lowest rank total, originally due to Kramer (1956) and discussed in Quandt (2006).

But there are occasions when knowing which is the most favored wine and whether its performance is statistically significant is not all that we want to know. The typical case in point is when there are two types of wines being tasted, as in the California vs. French face-off, where we want to know whether the wines of type A are together and on the whole more favored than the wines of type B. The question we would want to ask then is whether the sum of the ranksums over the subset of wines comprising type A is significantly smaller than the sum of the ranksums over the wines comprising type B. The ranksums for the wines in any subgroup are not independent of each other and the distribution of the sum of the ranksums under the null hypothesis that the two groups are identical is not obvious. In the next section we produce critical values for the sum of ranksums by employing Monte Carlo experiments.