Assessing accuracy for multi-class classification when subclasses are involved

Classifications that involve subclasses are common in many applied fields. “Compound multi-class classification” refers to the settings which involve three or more main classes and at least one of the main classes has multiple subclasses. In this paper, we propose an accuracy metric proper for “compound M[jls-end-space/]-class classification,” namely “hypervolume under compound (Formula presented) manifold (HUMC,M)[jls-end-space/].” The proposed HUMC,M evaluates the overall accuracy of a biomarker measured on continuous scale correctly identifying M main classes without requiring specification of an ordering in terms of marker values for subclasses relative to each other within each main class. The probabilistic interpretation of HUMC,M is analytically derived. A network-based computing algorithm which enables efficient computation of the empirical estimate of HUMC,M is developed. Non-parametric bootstrap percentile confidence intervals of HUMC,M are assessed through extensive simulation studies. Lastly, a real data example is included to illustrate the usage of our proposed method.