Visual Attention Network (VAN) for Segmentation

This repo is a PyTorch implementation of applying VAN (Visual Attention Network) to semantic segmentation.

The code is an integration from VAN-Segmentation

More details can be found in Visual Attention Network.

Citation

@article{guo2022visual,
  title={Visual Attention Network},
  author={Guo, Meng-Hao and Lu, Cheng-Ze and Liu, Zheng-Ning and Cheng, Ming-Ming and Hu, Shi-Min},
  journal={arXiv preprint arXiv:2202.09741},
  year={2022}
}

Results

Notes: Pre-trained models can be found in TsingHua Cloud.

Results can be found in VAN-Segmentation

We provide evaluation results of the converted weights.

Method	Backbone	mIoU	Download
UPerNet	VAN-B2	49.35	model
UPerNet	VAN-B3	49.71	model
UPerNet	VAN-B4	51.56	model
UPerNet	VAN-B4-in22k	52.61	model
UPerNet	VAN-B5-in22k	53.11	model
UPerNet	VAN-B6-in22k	54.25	model
FPN	VAN-B0	38.65	model
FPN	VAN-B1	43.22	model
FPN	VAN-B2	46.84	model
FPN	VAN-B3	48.32	model

Preparation

Install MMSegmentation and download ADE20K according to the guidelines in MMSegmentation.

Requirement

Step 0. Install MMCV using MIM.

pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"

Step 1. Install MMSegmentation.

Case a: If you develop and run mmseg directly, install it from source:

git clone -b main https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
pip install -v -e .

Case b: If you use mmsegmentation as a dependency or third-party package, install it with pip:

pip install "mmsegmentation>=1.0.0"

Training

If you use 4 GPUs for training by default. Run:

bash tools/dist_train.sh projects/van/configs/van/van-b2_pre1k_upernet_4xb2-160k_ade20k-512x512.py 4

Evaluation

To evaluate the model, an example is:

bash tools/dist_train.sh projects/van/configs/van/van-b2_pre1k_upernet_4xb2-160k_ade20k-512x512.py work_dirs/van-b2_pre1k_upernet_4xb2-160k_ade20k-512x512/iter_160000.pth 4 --eval mIoU

FLOPs

To calculate FLOPs for a model, run:

bash tools/analysis_tools/get_flops.py projects/van/configs/van/van-b2_pre1k_upernet_4xb2-160k_ade20k-512x512.py --shape 512 512

Acknowledgment

Our implementation is mainly based on mmsegmentation, Swin-Transformer, PoolFormer, Enjoy-Hamburger and VAN-Segmentation. Thanks for their authors.

LICENSE

This repo is under the Apache-2.0 license. For commercial use, please contact the authors.