The objective of this research is to devise a reliable and accurate scoring algorithm for awarding medals at a major U.S. wine competition that was described earlier by Hodgson (2008). The new algorithm is comprised of a two-tiered method that considers an award on the basis of a total medal score averaged across four judges per each evaluated wine; and a reliability algorithm that grades the degree of inter-judge reliability according to the following criteria: < 70% = Poor or Unacceptable; 70%–79% = Fair; 80%–89% = Good; and 90%–100% = Excellent agreement (Cicchetti, 2001).The method has advantages over a previous scoring method that based its decision upon incorrect estimates of majority opinion; and unacceptable levels of inter-judge agreement.