This work examines the expression mechanism of channel attention from the perspective of frequency domain, and then innovatively designs an extremely simple and effective channel attention mechanism based on DCT frequency domain. It mathematically proves that the most widely-used channel attention is a special case of attention mechanism. Most importantly, its operation and deployment are extremely simple that only one line of code needs to be modified.

With the same parameters and amount of calculation, the top-1 of Imagenet is improved by 1.8% compared with the senet-50, which also verifies its efficiency in completing traditional visual tasks such as classification, segmentation and detection. It's very suitable for practical deployment in industry and academia. The work will soon be open source. Welcome to use it and send us your feedback.

Full paper at: https://arxiv.org/abs/2012.11879

This paper was supported by CCST & Shanghai Institute of advanced research