README.md

FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting

By Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, Hongsheng Li.

This repo is the official Pytorch implementation of FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting.

<<<<<<< HEAD

Introduction

Usage

Prerequisites

• Python >= 3.6
• Pytorch >= 1.0 and corresponding torchvision (https://pytorch.org/)

Install

• Clone this repo:

git clone https://github.com/ruiliu-ai/FuseFormer.git

• Install other packages:

cd FuseFormer
pip install -r requirements.txt

Training

Dataset preparation

Download datasets (YouTube-VOS and DAVIS) into the data folder.

mkdir data

Training script

python train.py -c configs/youtube-vos.json

Test

Download pre-trained model into checkpoints folder.

mkdir checkpoints

Test script

python test.py -c checkpoints/fuseformer.pth -v data/DAVIS/JPEGImages/blackswan -m data/DAVIS/Annotations/blackswan

======= Coming soon.

Introduction

fab4dcbb9e27bc1ca819b1de0006611433f0965c

Citing FuseFormer

If you find FuseFormer useful in your research, please consider citing:

@InProceedings{Liu_2021_FuseFormer,
  title={FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting},
  author={Liu, Rui and Deng, Hanming and Huang, Yangyi and Shi, Xiaoyu and Lu, Lewei and Sun, Wenxiu and Wang, Xiaogang and Li Hongsheng},
  booktitle = {International Conference on Computer Vision (ICCV)},
  year={2021}
}

<<<<<<< HEAD

Acknowledement

This code relies heavily on the video inpainting framework from spatial-temporal transformer net.

fab4dcbb9e27bc1ca819b1de0006611433f0965c