FastLatticeSuperpixels

This is the code of our paper with title "Fast Generation of Superpixels With Lattice Topology" published on TIP 2022. In this paper, we propose a new superpixel generation method which can generate superpixels with lattice topology, i.e., the generated superpixels have regular lattice topology just like pixels on images. We achieve this point by moving the boundary pixels (or blocks) with constraints to optimize the boundaries of superpixels, as shown in the following figure:

The initial superpixels are shown in (a). We adjust the boundaries of superpixels from coarse to fine by moving the multilevel blocks as shown in (b)-(d). We refine the boundaries by adjusting the boundary pixels to get the final superpixels as shown in (e)

Installation of caffe version

Caffe Installation

1. We make use of layers in 'Superpixel Samping Networks' caffe repository and add additional layers for our lattice superpixels generation:

   For the installation of 'Superpixel Samping Networks', please refer to 'https://github.com/NVlabs/ssn_superpixels'

2. Manually copy all the source files (files in lib/include and lib/src folders) to the corresponding locations in the caffe repository. In the ssn_superpixels/lib directory:

cp FastLatticeSuperpixel/lib/caffe/src/caffe/layers/* video_prop_networks/lib/caffe/src/caffe/layers/. cp FastLatticeSuperpixel/lib/caffe/include/caffe/layers/* video_prop_networks/lib/caffe/include/caffe/layers/. cp FastLatticeSuperpixel/lib/caffe/src/caffe/proto/caffe.proto video_prop_networks/lib/caffe/src/caffe/proto/caffe.proto

3 Copy all the *.py files in the FastLatticeSuperpixel/ repository to the ssn_superpixels repository

3. Install Caffe following the installation instructions. In the ssn_superpixels/lib directory:

cd video_prop_networks/lib/caffe/
mkdir build
cd build
cmake ..
make -j
cd ../../../..

Install a cython file

We use a cython script taken from 'scikit-image' for enforcing connectivity in superpixels. To compile this:

cd lib/cython/
python setup.py install --user
cd ../..

Usage: BSDS segmentation

Data download

Download the BSDS dataset into data folder:

cd data
sh get_bsds.sh
cd ..

Manually change the path of datasets in the file of config.py according to your dataset repository.

Superpixel computation

Use compute_laticce_spixels.py to compute superpixels on BSDS dataset

You can change the number of generated superpixels by setting the parameters of s_h ,s_w,and s_l in compute_laticce_spixels.py. Where s_w and s_h denotes the weight and height of the block in the lowest level of the initial superpixel, and the s_l indicates the number of layers, i.e., the size of the initial superpixel is (s_h2^(s_l-1))(s_w2^(s_l-1)), and the number of superpixels generated by an image with length H and width W is H/((s_h2^(s_l-1))W/(s_w2^(s_l-1)).

Installation of Pytorch version

Pytorch Installation

We also provide a simple pytorch version of our algorithm. In order to run this pytorch code, each cuda file needs to be compiled first.

1. cd ./Pytorchversion/modelsA/latticesuperpixelcuda/
   python setup.py install
   

2. cd ../..
   cd pixelfeaturecuda
   python setup.py install
   

3. cd ..
   cd superpixelcolorcuda
   python setup.py install
   

Then you can use the superpixel generation algorithm in your own code. See the code in the file example.py for details.

Citation:
@article{pan2022fast,
title={Fast Generation of Superpixels With Lattice Topology},
author={Pan, Xiao and Zhou, Yuanfeng and Zhang, Yunfeng and Zhang, Caiming},
journal={IEEE Transactions on Image Processing},
volume={31},
pages={4828--4841},
year={2022},
publisher={IEEE}
} Please contact Xiao Pan([email protected]) if you have any question.