Oyster

©2017 IFeelBloated, Oyster Python Module for VapourSynth

License

LGPL v3.0

Description

Oyster is an experimental implement of the Blocking Matching concept, designed specifically for compression artifacts removal.
according to Wikipedia, when performing block-based coding for quantization, several types of artifacts can appear.

• Ringing
• Contouring
• Posterizing
• Staircase noise
• Blockiness

and oyster handles 3 of them, ringing, staircase noise and blockiness

Requirements

• NNEDI3
• KNLMeansCL
• BM3D
• DFTTest
• FMTConv
• MVTools (floating point ver)

Function List

• Super
• Basic
• Deringing
• Destaircase
• Deblocking

Formats

• Bit Depth: 32bits floating point
• Color Space: Gray, RGB, YUV 4:4:4 (subsampled YUV formats are not supported)
• Scan Type: Progressive

Notes

• DO NOT upsample your video to YUV 4:4:4 or RGB before processing, just pass Y as a gray clip and merge the result with UV from the source clip, low-res chroma will jeopardize the correctness of weight calculation (fatal, especially to NLMeans).
• DO NOT crop your video before processing, it will destroy the macroblock boundary detecting.
• NO scene change policy provided, take Wobbly and cut each scene out and process them individually
• QUALITY: cutting edge
• PERFORMANCE: abysmal, like, literally..

Details

Super

Optional, it helps improve the precision of sub-pixel motion estimation and compensation, use it and get a quality boost or don't and get a performance boost

Super(src, pel=4)

• src
  clip to be processed
• pel
  sub-pixel precision, could be 2 or 4, 2 = precision by half a pixel, 4 = precision by quarter a pixel.

Basic

The basic estimation does a wild block matching based motion compensation, it removes all obvious artifacts and serves as the reference to all later more specific artifacts removing functions

Basic(src, super=None, radius=6, pel=4, sad=2000.0, short_time=False)

• super
  optional, clip generated by Oyster.Super
• radius
  temporal radius, frames that fall in [current frame - radius, current frame + radius] will be referenced
• sad
  SAD threshold of the motion compensation, refer to MVTools doc for more details
• short_time
  with short_time = True, starts motion estimation at the block size of 8x8 and gradually refines it to 2x2 (optimized for maximum spatial resolution), short_time = False starts motion estimation at 128x128 and gradually refines it to 4x4 (optimized for maximum frequency resolution). short_time = True is meant for Deringing and short_time = False is for Destaircase and Deblocking

Deringing

Deringing removes ringing (aka. mosquito noise) artifacts caused by lossy compression.

workflow:

• replace low frequency components of the basic estimation with low frequencies from the source clip, since ringing is high frequency artifacts
• coarse refining on basic estimation with BM3D (VBasic)
• another more delicate refining with NLMeans
• BM3D (VFinal) filtering to remove barely visible artifacts, as basic estimation only handles obvious artifacts
• replace low frequencies of the filtered result with low frequencies from the source clip yet again
• NLMeans refining again

Deringing(src, ref, radius=6, h=6.4, sigma=16.0, mse=[None, None], hard_thr=3.2, block_size=8, block_step=1, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1, lowpass=None)

• ref
  clip generated by Oyster.Basic
• h
  filtering strength of the NLMeans refining, greater value = more relaxed refining (less possible to have residual artifacts but more detail loss)
• sigma, mse, hard_thr, block_size, block_step, group_size, bm_range, bm_step, ps_num, ps_range, ps_step
  refer to BM3D doc for more details and, mse[0] is the mse value for VBasic, mse[1] for VFinal, default mse[0] = sigma * 160.0 + 1200.0, mse[1] = sigma * 120.0 + 800.0
• lowpass
  controls how lowpass filter works, refer to the sstring section in DFTTest doc for more details, default = "0.0:sigma 0.48:1024.0 1.0:1024.0"

Destaircase

block based quantization sometimes zeros out high frequency coefficients and leaves the low frequency part (almost or completely) unprocessed, which yields staircase noise, a kind of artifacts that resembles blocking, and sometimes may considered as blocking, it shows as discontinuities (aliasing) along curving edges, and Destaircase kills it

workflow:

• replace low frequency components of the basic estimation with low frequencies from the source clip like Deringing
• a threshold based limiter eliminates all small differences, discontinuities are large differences apparently
• coarse BM3D refining (VBasic)
• more delicate BM3D refining (VFinal)
• replace macroblock boundaries in the source clip with the filtered result !

Destaircase(src, ref, radius=6, sigma=16.0, mse=[None, None], hard_thr=3.2, block_size=8, block_step=1, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1, thr=0.03125, elast=0.015625, lowpass=None)

• thr
  threshold of the limiter, ranges from 0.0 (no limit) to 1.0 (no filtering), differences between the basic estimation and source clip < thr will be discarded, otherwise remain unaffected.
• elast
  ! elasticity of the threshold, ranges from 0.0 to thr.

Deblocking

generally, Destaircase + Deringing combo is enough to typical blocking artifacts, this one works in extreme cases with severe blocking artifacts, it's fairly destructive since it's designed for extreme cases, DON'T use it unless you have to, and use it extra carefully

workflow:

• Make a 100% artifacts-free copy of the input clip by appending an NLMeans filtering to the basic estimation, regardless of detail loss
• coarse BM3D refining (VBasic)
• more delicate BM3D refining (VFinal)
• replace low frequency components of both the source clip and basic estimation with low frequencies from the filtered result
• replace macroblock boundaries in the source clip with the basic estimation

Deblocking(src, ref, radius=6, h=6.4, sigma=16.0, mse=[None, None], hard_thr=3.2, block_size=8, block_step=1, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1, lowpass="0.0:0.0 0.12:1024.0 1.0:1024.0")

Demos

• Destaircase + Deringing
  

sup = Oyster.Super(clip)
ref_f = Oyster.Basic(clip, sup, short_time=False)
ref_s = Oyster.Basic(clip, sup, short_time=True)
clip = Oyster.Destaircase(clip, ref_f, block_step=2)
clip = Oyster.Deringing(clip, ref_s, block_step=2)

• Destaircase + Deringing (severe mosquito noise)
  

sup = Oyster.Super(clip)
ref_f = Oyster.Basic(clip, sup, short_time=False)
ref_s = Oyster.Basic(clip, sup, short_time=True)
clip = Oyster.Destaircase(clip, ref_f, block_step=2, lowpass="0.0:1024 1.0:1024")
clip = Oyster.Deringing(clip, ref_s, sigma=24.0, h=12.8, block_step=2, lowpass="0.0:1024 1.0:1024")

• Destaircase + Deringing (H.264/H.265 compression artifacts)
  click the image and view at full size

sup = Oyster.Super(clip)
ref_f = Oyster.Basic(clip, sup, short_time=False)
ref_s = Oyster.Basic(clip, sup, short_time=True)
clip = Oyster.Destaircase(clip, ref_f, sigma=24.0, block_step=2)
clip = Oyster.Deringing(clip, ref_s, sigma=24.0, h=9.6, block_step=2)

• Deblocking + Deringing
  

sup = Oyster.Super(clip)
ref_f = Oyster.Basic(clip, sup, short_time=False)
ref_s = Oyster.Basic(clip, sup, short_time=True)
clip = Oyster.Deblocking(clip, ref_f, block_step=2)
clip = Oyster.Deringing(clip, ref_s, sigma=24.0, h=10.8, block_step=2)