Human and machine performance in an inspection task

Inspectors inspected a batch of 162 metal cylindrical items. These were later inspected by a prototype inspection device so that performance of human and machine could be compared. Receiver operating characteristic (ROC) curves were plotted for each of four types of fault for the inspection device. The performance of each inspector was compared with these ROC curves to assess overall performance. Inspectors performed significantly better than did the prototype machine, largely because of the more sophisticated decision-making capabilities of humans. The inspection device could locate most faults but was unable to classify them as acceptable or rejectable with the same consistency as inspectors.