Contact
AAWE
Economics Department
New York University
19 W. 4th Street, 6FL
New York, NY 10012, U.S.A.
Tel: (212) 992-8083
Fax: (212) 995-4186
E-Mail: karl.storchmann@nyu.edu
The objective of this research is to devise a reliable and accurate scoring algorithm for awarding medals at a major U.S. wine competition that was described earlier by Hodgson (2008). The new algorithm is comprised of a two-tiered method that considers an award on the basis of a total medal score averaged across four judges per each evaluated wine; and a reliability algorithm that grades the degree of inter-judge reliability according to the following criteria: < 70% = Poor or Unacceptable; 70%–79% = Fair; 80%–89% = Good; and 90%–100% = Excellent agreement (Cicchetti, 2001).The method has advantages over a previous scoring method that based its decision upon incorrect estimates of majority opinion; and unacceptable levels of inter-judge agreement.
AAWE
Economics Department
New York University
19 W. 4th Street, 6FL
New York, NY 10012, U.S.A.
Tel: (212) 992-8083
Fax: (212) 995-4186
E-Mail: karl.storchmann@nyu.edu