babylonpolice.com

Transformer models have made tremendous progress in various fields in recent
years. In the field of computer vision, vision transformers (ViTs) also become
strong alternatives to convolutional neural networks (ConvNets), yet they have
not been able to replace ConvNets since both have their own merits. For
instance, ViTs are good at extracting global features with attention mechanisms
while ConvNets are more efficient in modeling local relationships due to their
strong inductive bias. A natural idea that arises is to combine the strengths
of both ConvNets and ViTs to design new structures. In this paper, we propose a
new basic neural network operator named position-aware circular convolution
(ParC) and its accelerated version Fast-ParC. The ParC operator can capture
global features by using a global kernel and circular convolution while keeping
location sensitiveness by employing position embeddings. Our Fast-ParC further
reduces the O(n2) time complexity of ParC to O(n log n) using Fast Fourier
Transform. This acceleration makes it possible to use global convolution in the
early stages of models with large feature maps, yet still maintains the overall
computational cost comparable with using 3x3 or 7x7 kernels. The proposed
operation can be used in a plug-and-play manner to 1) convert ViTs to
pure-ConvNet architecture to enjoy wider hardware support and achieve higher
inference speed; 2) replacing traditional convolutions in the deep stage of
ConvNets to improve accuracy by enlarging the effective receptive field.
Experiment results show that our ParC op can effectively enlarge the receptive
field of traditional ConvNets, and adopting the proposed op benefits both ViTs
and ConvNet models on all three popular vision tasks, image classification,
object