Abstract
Recent advances in unmanned aerial vehicles (UAVs) have expanded their applications in human action recognition. However, existing methods struggle with background interference from wide field-of-view and UAV-induced jitter. In this work, we propose the single stage Frequency-SpatioTemporal Transformer (Fre-STformer) model, which accurately recognizes human actions via mining feature variation patterns in both actions and backgrounds. Specifically, the proposed frequency-based spatio-temporal (Fre-ST) module addresses two key challenges in UAV-based action recognition: spatial-frequency joint encoding suppresses background interference through spectral feature decomposition, while channel-level difference modeling stabilizes motion representations against UAV jitter via channel-wise correlation learning. Finally, by incorporating the Fre-ST module, our Fre-STformer parses global spatio-temporal action features end-to-end more effectively, avoiding additional pre-processing, e.g. action region segmentation. Extensive experiments conducted on three typical low-altitude UAV datasets, i.e. UAV Human, NEC Drone and UAV-human-night. Results show that our Fre-STformer outperforms the state-of-the-arts on the first two datasets by 3.5% and 3.8% in Top-1 accuracy, respectively. In addition, on the third UAV-human-night dataset, the Top-1 accuracy boosts by at least 1.0%, validating the generalization of our Fre-STformer in low-light environments.
| Original language | English |
|---|---|
| Pages (from-to) | 8207-8219 |
| Number of pages | 13 |
| Journal | IEEE Transactions on Multimedia |
| Volume | 28 |
| DOIs | |
| State | Published - 2026 |
Keywords
- UAV action recognition
- channel-level difference modeling
- frequency-based spatio-temporal representation
Fingerprint
Dive into the research topics of 'Fre-STformer: A Frequency-Based Spatio-Temporal Transformer for UAV Human Action Recognition'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver