VocalSeparationAI

This repo is self-learning project that the music and its vocal converted to relative spectograms, then using these spectograms, the vocal seperation AI is trained.

Some Test Results

Music	Vocal	AI Output
Music Listen	Vocal Listen	AI Output Listen
Music Listen	Vocal Listen	AI Output Listen

The dataset

• I used DSD100 dataset to get music and vocal.
  • The Dataset
• The musics and vocals is splitted 5 sec parts and converted to spectograms. The dataset I created with spectograms
  • The Spectogram Dataset

Tools

• I used provided repo to convert from music to spectogram and spectogram to music
  • Spectogram Program
• I used pix2pix-tensorflow implementation to train the model
  • Pix2Pix

Preprocessing Details

1 - The musics and vocals are splitted to 5 sec music parts.

2 - The 5 sec parts are converted to spectogram images. Changed Values to get 255x256 images :

• Pixels per second : 51
• Bandwitdh : 205

3 - 1 pixel height is added end of the height( Don't put start ) to get image size 256x256 images.

4 - The parts that do not contain vocals is removed from dataset via removing the images has only 0 pixel values.

Training Details

• I trained 10 epochs in pix2pix implementation.

Further Improvements and Limitations

I will update this part

• pix2pixHD.
• transparency problem of spectograms.