University of Waterloo

Structure-aware Mamba-transformer hybrid model for hyperspectral image classification

Tina Liu

June 27th, 2025 – 12:00-1:00pm, EC4-2101A

Hyperspectral image (HSI) classification underpins applications ranging from environmental monitoring and precision agriculture to urban planning and mineral exploration, yet conventional convolutional networks capture only local patterns and transformers—while adept at global context—remain computationally heavy and still struggle with the very long‐range spectral–spatial dependencies latent in hundreds of contiguous bands. Leveraging the recent Mamba state-space model, which offers linear-time sequence processing, we introduce a structure-aware state-fusion mechanism that explicitly encodes neighbouring spectral and spatial relationships within the latent state, reducing redundancy and strengthening representations. Building on this foundation, we insert a lightweight self-attention block solely in the final layer of a Mamba backbone, yielding a hybrid Mamba–Transformer architecture that balances efficiency and global context modeling. Tested on three public benchmarks (Indian Pines, Pavia University, and WHU-Hi-HanChuan), the proposed network matches or exceeds state-of-the-art Transformer and Mamba variants, underscoring the promise of combining state-space and attention mechanisms for accurate, efficient HSI classification.