Measuring Relative Agreement: Echocardiographer Versus Computer

The purpose of this article is to define a single descriptive measure of agreement that provides insight into the question: Does an autonomous computer algorithm dedicated to a measurement task perform better or worse than trained observers relative to a continuous gold standard measurement? We have defined a new measure τ, 0 ≤ τ ≤ 1, called the relative agreement, which measures the relative distance between three measurements from three distinct raters to the line of equality or agreement (i.e., the 45° line passing through the origin). We will illustrate that τ is independent of the correlation coefficient or other measures of association or location that are commonly misused or misinterpreted as measures of agreement. An example of the relative agreement τ is given to an echocardiographic imaging (EI) study. The goal of the EI study is directed toward fully autonomous boundary detection of the epicardial and endocardial surfaces to provide consistent measures of chamber diameter, chamber area, fractional area change, and left ventricular myocardial area.