Style Transfer GANs (Generative Adversarial Networks)

Style Transfer Generative Adversarial Networks take two images and apply the style from one image to the other image. Here are some sample results from here.

Style Transfer GAN examples

For a more technical explanation of how these work, you can refer to the following papers;

Image Style Transfer Using Convolutional Neural Networks
Artistic style transfer for videos
Preserving Color in Neural Artistic Style Transfer

Ever since first seeing this technique I wanted to add it as an image processing option within Visions of Chaos.

If you only want to play around with style transfer or only have a few photos you want to experiment with, then I recommend you use an online service like DeepArt because this can be a tedious process to setup and use on your own PC.

Installing Style Transfer Prerequisites

Follow these steps to get Python, Python Libraries, CUDA and CuDNN installed.

Style Transfer in Visions of Chaos

Generate any image, then select Image->Image Processing->Style Transfer.

Visions of Chaos Style Transfer GAN

The first time you select Style Transfer it will download the 500 MB neural network model that is used for all the style transfer magic.

Start with smaller image sizes to get an idea of how long the process will take on your system before going for larger sized images.

You can also select any external image file to apply the style transfer to. So dig out those cat photos and have fun. Note that if you get tired of the limited style images that come with Visions of Chaos, you can put any image you like under the Style Transfer folder (by default this will be C:\Users\\AppData\Roaming\Visions of Chaos\Examples\TensorFlow\Style Transfer\) and use those. Grab an image of your favorite artist’s works and experiment.

For these next examples I used the following photo of Miss Marple.

Miss Marple

And applied some various transfer style images.

MC Escher Plane Filling II

Miss Marple Style Transfer GAN

A Mandelbrot fractal

Miss Marple Style Transfer GAN

Another Mandelbrot fractal

Miss Marple Style Transfer GAN

HR Giger Biomechanical Landscape

Miss Marple Style Transfer GAN

Kandinsky Composition VII

Miss Marple Style Transfer GAN

Mondrian

Miss Marple Style Transfer GAN

Monet

Miss Marple Style Transfer GAN

Picasso Les Femmes d’Alger

Miss Marple Style Transfer GAN

Picasso Seated Nude

Miss Marple Style Transfer GAN

Hokusai The Great Wave off Kanagawa

Miss Marple Style Transfer GAN

Munch The Scream

Miss Marple Style Transfer GAN

Turner The Wreck of a Transport Ship

Miss Marple Style Transfer GAN

van Gogh Starry Night

Miss Marple Style Transfer GAN

How It Works

Behind the scenes the Style Transfer uses Cameron Smith’s excellent Python script from here. After trying various Style Transfer related scripts that one gives the sharpest and most interesting results. See that link if you want to run these sort of style transfers yourself from the command line outside Visions of Chaos.

Troubleshooting

If you get a failed style transfer and an error message, here are a few things to try;
1. Smaller image size. Depending on the RAM in your PC and GPU you may have maxed out.
2. Wait 30 seconds and try again. This seems to help sometimes.
3. Reboot. If all else fails. Seems to always fix a stubborn error for me. The Cuda and/or cuDNN seem to be the main culprit. They get hung or locked or something and only a reboot will get them working again.

Jason.

One response to “Style Transfer GANs (Generative Adversarial Networks)

  1. Thanks for your explanations .
    i checked notify for new comments.
    and many thanks for voc, very good program, i’ll use it to explain things to students.have a nice day .
    Pierre-Yves

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s