Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, and Ming-Hsuan Yang
European Conference on Computer Vision (ECCV), 2018
[Project page][Paper]
- Introduction
- Citation
- Requirements and Dependencies
- Installation
- Dataset
- Apply Pre-trained Models
- Training and Testing
- Image Processing Algorithms
Our method takes the original unprocessed and per-frame processed videos as inputs to produce a temporally consistent video. Our approach is agnostic to specific image processing algorithms applied on the original video.
If you find the code and datasets useful in your research, please cite:
@inproceedings{Lai-ECCV-2018,
author = {Lai, Wei-Sheng and Huang, Jia-Bin and Wang, Oliver and Shechtman, Eli and Yumer, Ersin and Yang, Ming-Hsuan},
title = {Learning Blind Video Temporal Consistency},
booktitle = {European Conference on Computer Vision},
year = {2018}
}
- Pytorch 0.4
- TensorboardX
- FlowNet2-Pytorch (Code and model are already included in this repository)
- LPIPS (for evaluation)
Our code is tested on Ubuntu 16.04 with cuda 9.0 and cudnn 7.0.
Download repository:
git clone https://github.com/phoenix104104/fast_blind_video_consistency.git
Compile FlowNet2 dependencies (correlation, resample, and channel norm layers):
./install.sh
Download our training and testing datasets:
cd data
./download_data.sh [train | test | all]
For example, download training data only:
./download_data.sh train
Download both training and testing data:
./download_data.sh all
You can also download the results of [Bonneel et al. 2015] and our approach:
./download_data.sh results
Download pretrained models (including FlowNet2 and our model):
cd pretrained_models
./download_models.sh
We use the following algorithms to obtain per-frame processed results:
Style transfer
- WCT: Universal Style Transfer via Feature Transforms, NIPS 2017
- Fast Neural Style Transfer: Perceptual Losses for Real-Time Style Transfer and Super-Resolution, ECCV 2016
Image Enhancement
Intrinsic Image Decomposition
Image-to-Image Translation
- CycleGAN: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV 2017
Colorization