Segregation and Context Aggregation Network for Real-time Cloud Segmentation (Papers Track)
Yijie Li (Northwestern University); Hewei Wang (Carnegie Mellon University); Jiayi Zhang (University of Nottingham Ningbo China); Jinjiang You (Carnegie Mellon University); Jinfeng Xu (The University of Hong Kong); Puzhen Wu (Cornell University); Yunzhong Xiao (Carnegie Mellon University); Soumyabrata Dev (University College Dublin)
Abstract
Cloud segmentation from intensity images is a pivotal task in atmospheric science and computer vision, aiding weather forecasting and climate analysis. Ground-based sky/cloud segmentation extracts clouds from images for further feature analysis. Existing methods struggle to balance segmentation accuracy and computational efficiency, limiting real-world deployment on edge devices, so we introduce SCANet, a novel lightweight cloud segmentation model featuring Segregation and Context Aggregation Module (SCAM), which refines rough segmentation maps into weighted sky and cloud features processed separately. SCANet achieves state-of-the-art performance while drastically reducing computational complexity. SCANet-large (4.29M) achieves comparable accuracy to state-of-the-art methods with 70.9% fewer parameters. Meanwhile, SCANet-lite (90K) delivers 1390 fps in FP16, surpassing real-time standards. Additionally, we propose an efficient pre-training strategy that enhances performance even without ImageNet pre-training.