Skip to main navigation Skip to search Skip to main content

Fre-STformer: A Frequency-Based Spatio-Temporal Transformer for UAV Human Action Recognition

  • Wuhan University
  • Hubei University

Research output: Contribution to journal › Article › peer-review

Abstract

Recent advances in unmanned aerial vehicles (UAVs) have expanded their applications in human action recognition. However, existing methods struggle with background interference from wide field-of-view and UAV-induced jitter. In this work, we propose the single stage Frequency-SpatioTemporal Transformer (Fre-STformer) model, which accurately recognizes human actions via mining feature variation patterns in both actions and backgrounds. Specifically, the proposed frequency-based spatio-temporal (Fre-ST) module addresses two key challenges in UAV-based action recognition: spatial-frequency joint encoding suppresses background interference through spectral feature decomposition, while channel-level difference modeling stabilizes motion representations against UAV jitter via channel-wise correlation learning. Finally, by incorporating the Fre-ST module, our Fre-STformer parses global spatio-temporal action features end-to-end more effectively, avoiding additional pre-processing, e.g. action region segmentation. Extensive experiments conducted on three typical low-altitude UAV datasets, i.e. UAV Human, NEC Drone and UAV-human-night. Results show that our Fre-STformer outperforms the state-of-the-arts on the first two datasets by 3.5% and 3.8% in Top-1 accuracy, respectively. In addition, on the third UAV-human-night dataset, the Top-1 accuracy boosts by at least 1.0%, validating the generalization of our Fre-STformer in low-light environments.

Original languageEnglish
Pages (from-to)8207-8219
Number of pages13
JournalIEEE Transactions on Multimedia
Volume28
DOIs
StatePublished - 2026

Keywords

  • UAV action recognition
  • channel-level difference modeling
  • frequency-based spatio-temporal representation

Fingerprint

Dive into the research topics of 'Fre-STformer: A Frequency-Based Spatio-Temporal Transformer for UAV Human Action Recognition'. Together they form a unique fingerprint.

Cite this