RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA

Jiajie Li; Brian R. Quaranto; Chenhui Xu; Ishan Mishra; Ruiyang Qin; Dancheng Liu; Peter C.W. Kim; Jinjun Xiong

RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA

Jiajie Li
, Brian R. Quaranto
, Chenhui Xu
, Ishan Mishra
, Ruiyang Qin
, Dancheng Liu
, Peter C.W. Kim
, Jinjun Xiong

SUNY Buffalo
Indian Institute of Technology Jodhpur
University of Notre Dame

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks.

Original language	English
Title of host publication	13th International Conference on Learning Representations, ICLR 2025
Publisher	International Conference on Learning Representations, ICLR
Pages	46751-46763
Number of pages	13
ISBN (Electronic)	9798331320850
State	Published - 2025
Event	13th International Conference on Learning Representations, ICLR 2025 - Singapore, Singapore Duration: Apr 24 2025 → Apr 28 2025

Publication series

Name	13th International Conference on Learning Representations, ICLR 2025

Conference

Conference	13th International Conference on Learning Representations, ICLR 2025
Country/Territory	Singapore
City	Singapore
Period	04/24/25 → 04/28/25

Cite this

Li, J., Quaranto, B. R., Xu, C., Mishra, I., Qin, R., Liu, D., Kim, P. C. W., & Xiong, J. (2025). RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA. In 13th International Conference on Learning Representations, ICLR 2025 (pp. 46751-46763). (13th International Conference on Learning Representations, ICLR 2025). International Conference on Learning Representations, ICLR.

@inproceedings{711bd59fab4b4f2cb061d2f61c6cf4c2,

title = "RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA",

abstract = "We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks.",

author = "Jiajie Li and Quaranto, \{Brian R.\} and Chenhui Xu and Ishan Mishra and Ruiyang Qin and Dancheng Liu and Kim, \{Peter C.W.\} and Jinjun Xiong",

note = "Publisher Copyright: {\textcopyright} 2025 13th International Conference on Learning Representations, ICLR 2025. All rights reserved.; 13th International Conference on Learning Representations, ICLR 2025 ; Conference date: 24-04-2025 Through 28-04-2025",

year = "2025",

language = "English",

series = "13th International Conference on Learning Representations, ICLR 2025",

publisher = "International Conference on Learning Representations, ICLR",

pages = "46751--46763",

booktitle = "13th International Conference on Learning Representations, ICLR 2025",

}

Li, J, Quaranto, BR, Xu, C, Mishra, I, Qin, R, Liu, D, Kim, PCW & Xiong, J 2025, RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA. in 13th International Conference on Learning Representations, ICLR 2025. 13th International Conference on Learning Representations, ICLR 2025, International Conference on Learning Representations, ICLR, pp. 46751-46763, 13th International Conference on Learning Representations, ICLR 2025, Singapore, Singapore, 04/24/25.

RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA. / Li, Jiajie; Quaranto, Brian R.; Xu, Chenhui et al.
13th International Conference on Learning Representations, ICLR 2025. International Conference on Learning Representations, ICLR, 2025. p. 46751-46763 (13th International Conference on Learning Representations, ICLR 2025).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - RECOGNIZE ANY SURGICAL OBJECT

T2 - 13th International Conference on Learning Representations, ICLR 2025

AU - Li, Jiajie

AU - Quaranto, Brian R.

AU - Xu, Chenhui

AU - Mishra, Ishan

AU - Qin, Ruiyang

AU - Liu, Dancheng

AU - Kim, Peter C.W.

AU - Xiong, Jinjun

PY - 2025

Y1 - 2025

N2 - We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks.

AB - We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks.

UR - https://www.scopus.com/pages/publications/105010210005

M3 - Conference contribution

AN - SCOPUS:105010210005

T3 - 13th International Conference on Learning Representations, ICLR 2025

SP - 46751

EP - 46763

BT - 13th International Conference on Learning Representations, ICLR 2025

PB - International Conference on Learning Representations, ICLR

Y2 - 24 April 2025 through 28 April 2025

ER -

RECOGNIZE ANY SURGICAL OBJECT: UNLEASHING THE POWER OF WEAKLY-SUPERVISED DATA

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this