TY - GEN
T1 - Truthing, testing and evaluation issues in complex systems
AU - Setlur, Srirangaraj
AU - Govindaraju, Venu
AU - Lawson, Alfred
N1 - Publisher Copyright:
© 2001 IEEE.
PY - 2001
Y1 - 2001
N2 - This paper describes the issues involved in the design of a system for evaluating improvements in the performance of a real-time address recognition system being used by the United States Postal Service for processing mail-piece images. Evaluation of the performance of recognition systems is normally carried out by measuring the per-formance of the system on a representative sample of images. Designing a comprehensive and valid testing scenario is a complex task that requires careful attention. Sampling live mail-stream to generate a deck of images representative of the general mail-stream for testing, truthing (generating reference data on a significant number of images), grading and evaluation, and designing tools to facilitate these functions are important topics that need to be addressed. This paper describes the efforts of the United States Postal Service and CEDAR towards developing an infrastructure for sampling, truthing and testing of mailstream images.
AB - This paper describes the issues involved in the design of a system for evaluating improvements in the performance of a real-time address recognition system being used by the United States Postal Service for processing mail-piece images. Evaluation of the performance of recognition systems is normally carried out by measuring the per-formance of the system on a representative sample of images. Designing a comprehensive and valid testing scenario is a complex task that requires careful attention. Sampling live mail-stream to generate a deck of images representative of the general mail-stream for testing, truthing (generating reference data on a significant number of images), grading and evaluation, and designing tools to facilitate these functions are important topics that need to be addressed. This paper describes the efforts of the United States Postal Service and CEDAR towards developing an infrastructure for sampling, truthing and testing of mailstream images.
UR - https://www.scopus.com/pages/publications/33646752743
U2 - 10.1109/ICDAR.2001.953975
DO - 10.1109/ICDAR.2001.953975
M3 - Conference contribution
AN - SCOPUS:33646752743
T3 - Proceedings of the International Conference on Document Analysis and Recognition, ICDAR
SP - 1205
EP - 1214
BT - Proceedings - 6th International Conference on Document Analysis and Recognition, ICDAR 2001
PB - IEEE Computer Society
T2 - 6th International Conference on Document Analysis and Recognition, ICDAR 2001
Y2 - 10 September 2001 through 13 September 2001
ER -