The Dream
For years now you would have seen scenes in TV shows like CSI or movies like Blade Runner the “enhance” functionality of software that allows details to be enhanced in images that are only a blur or a few pixels in size. In Blade Runner, Deckard’s system even allowed him to look around corners.
The Reality
I have recently been testing machine learning neural network enhancers (aka super resolution) models. They resize an image while trying to maintain or enhance details without losing detail (or with losing a lot less detail than if the image was zoomed with an image editing tool using linear or bicubic zoom).
Some of my results with these models follows. I am using the following test image from here.
To best see the differences between the algorithms I recommend you open the x4 zoomed images in new tabs and switch between them.
SRCNN – Super-Resolution Convolutional Neural Network
To see the original paper on SRCNN, click here.
I am using the PyTorch script by Mirwaisse Djanbaz here.
SRRESNET
To see the original paper on SRRESNET, click here.
I am using the PyTorch script by Sagar Vinodababu here.
SRGAN – Super Resolution Generative Adversarial Network
To see the original paper on SRGAN, click here.
I am using the PyTorch script by Sagar Vinodababu here.
ESRGAN – Enhanced Super Resolution Generative Adversarial Network
I am using the PyTorch script by Xintao Wang et al here.
PSNR
I am using the PyTorch script by Xintao Wang et al here.
Real-ESRGAN
This is the best super sampler here. I am using the executable by Xintao Wang et al here.
Real-ESRNET
I am using the executable by Xintao Wang et al here.
SwinIR
Very nice results. May be equal to or better than Real-ESRGAN depending on the input image. I am using the code from this colab.
SPSR
Another method from here.
ruDALL-E Real-ESRGAN
From here.
Differences
Each of the algorithms gives different results. For an unknown source image it would probably be best to run it through them all and then see which gives you the best result. These are not the Hollywood or TV enhance magic fix just yet.
If you know of any other PyTorch implementations of super resolution I missed, let me know.
Availability
You can follow the links to the original GitHub repositories to get the software, but I have also added a simple GUI front end for these scripts in Visions of Chaos. That allows you to try the above algorithms on single images or batch process a directory of images.
Jason.