Text-to-Image Summary – Part 4

This is Part 4. See Part 1, Part 2 and Part 3 for the previous posts.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.


Name: PixelDraw
Author: dribnet
Original script: https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/PixelDrawer.ipynb
Time for 512×512 on a 3090: 1 minutes 59 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Generates “pixel art” images. I had a lot of requests to add support for this one.

'a cartoon of a peacock' PixelDraw Text-to-Image
a cartoon of a peacock

'a cloudy sunset' PixelDraw Text-to-Image
a cloudy sunset

'a gorilla' PixelDraw Text-to-Image
a gorilla

'a morning landcsape' PixelDraw Text-to-Image
a morning landscape

'a watercolor painting of a castle' PixelDraw Text-to-Image
a watercolor painting of a castle

'an art deco painting of Al Pacino' PixelDraw Text-to-Image
an art deco painting of Al Pacino

'Hell' PixelDraw Text-to-Image
Hell

'Shrek' PixelDraw Text-to-Image
Shrek


Name: DirectVisions
Author: Jens Goldberg
Original script: https://colab.research.google.com/drive/127lKSsQjx-UDDUSvIkLL6mREfZ0KQu5D
Time for 512×512 on a 3090: 2 minutes 39 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Interesting detailed images. Can create huge resolution results.

'a color pencil sketch of a western town' DirectVisions Text-to-Image
a color pencil sketch of a western town

'a detailed painting of a cephalopod' DirectVisions Text-to-Image
a detailed painting of a cephalopod

'a digital rendering of an ugly face' DirectVisions Text-to-Image
a digital rendering of an ugly face

'a pencil sketch of Buzz Lightyear' DirectVisions Text-to-Image
a pencil sketch of Buzz Lightyear

'a rough seascape by Pinchus Kremegne' DirectVisions Text-to-Image
a rough seascape by Pinchus Kremegne

'a stock photo of a president' DirectVisions Text-to-Image
a stock photo of a president

'a sunset' DirectVisions Text-to-Image
a sunset

'an alien city' DirectVisions Text-to-Image
an alien city

'an alien forest by Helen Berman' DirectVisions Text-to-Image
an alien forest by Helen Berman

'an evening landscape' DirectVisions Text-to-Image
an evening landscape


Name: Pixel Direct
Author: Unknown
Original script: https://colab.research.google.com/drive/1F9ZOZnpV3uBPRDSESaAXYwzNZJQRJT75
Time for 512×512 on a 3090: 1 minutes 03 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Another “Pixel Art” script. More abstract results than the PixelDraw script above.

'a bronze sculpture of a nightmare creature' Pixel Direct Text-to-Image
a bronze sculpture of a nightmare creature

'a cartoon of Al Pacino' Pixel Direct Text-to-Image
a cartoon of Al Pacino

'a nightclub' Pixel Direct Text-to-Image
a nightclub

'a silk screen of a bouquet of flowers' Pixel Direct Text-to-Image
a silk screen of a bouquet of flowers

'an etching of a worried woman' Pixel Direct Text-to-Image
an etching of a worried woman

'an illustration of of a thunder storm' Pixel Direct Text-to-Image
an illustration of of a thunder storm


Name: FourierVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1nGNBjhbYnDHSumGPjpFHjDOsaZFAqGgF
Time for 512×512 on a 3090: 1 minutes 40 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Detailed images. The default script generates washed out pastel images, but with some gamma and brightness tweaks they can be improved (still not ideal, but better). Allows very large resolution images.

'a cathedral' FourierVisions Text-to-Image
a cathedral

'a charcoal drawing of zombies' FourierVisions Text-to-Image
a charcoal drawing of zombies

'a detailed painting of a sunset by Thomas Cantrell Dugdale' FourierVisions Text-to-Image
a detailed painting of a sunset by Thomas Cantrell Dugdale

'a ghost made of mist' FourierVisions Text-to-Image
a ghost made of mist

'a kitchen' FourierVisions Text-to-Image
a kitchen

'a movie monster' FourierVisions Text-to-Image
a movie monster

'a pencil sketch of a sad clown' FourierVisions Text-to-Image
a pencil sketch of a sad clown

'a werewolf' FourierVisions Text-to-Image
a werewolf

'an evil clown by Viktor Oliva' FourierVisions Text-to-Image
an evil clown by Viktor Oliva

'an ink drawing of an ugly monster' FourierVisions Text-to-Image
an ink drawing of an ugly monster


Name: PyramidVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1dpAS_wK34y7c6s-CatAFmBtbkjGT_erM
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Very detailed images. Not the fastest script, but gives some very nice results. Lower VRAM requirements so good for lesser spec GPUs. Definitely one of the better scripts worth exploring.

'a desert oasis' PyramidVisions Text-to-Image
a desert oasis

'a lush rainforest' PyramidVisions Text-to-Image
a lush rainforest

'a marble sculpture of an angry person' PyramidVisions Text-to-Image
a marble sculpture of an angry person

'a minimalist painting of the Amazon Rainforest' PyramidVisions Text-to-Image
a minimalist painting of the Amazon Rainforest

'a nightmare creature' PyramidVisions Text-to-Image
a nightmare creature

'a pastel of a computer made of paper' PyramidVisions Text-to-Image
a pastel of a computer made of paper

'an abstract sculpture of a sad clown' PyramidVisions Text-to-Image
an abstract sculpture of a sad clown

'an acrylic painting of an alien forest | vivid colors' PyramidVisions Text-to-Image
an acrylic painting of an alien forest | vivid colors

'Medusa' PyramidVisions Text-to-Image
Medusa

'vector art of an ugly woman' PyramidVisions Text-to-Image
vector art of an ugly woman



Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

Text-to-Image Summary – Part 3

This is Part 3. See Part 1 and Part 2 for the previous posts.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.


Name: CLIP Guided Diffusion v4
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1V66mUeJbXrTuQITvJunvnWVn96FEbSI3
Time for 512×512 on a 3090: 3 minutes 05 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Description: Another CLIP Guided Diffusion script. Locked to 512×512 resolution. Like the other CLIP Diffusion scripts, some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. When it gets a “hit” it can create very detailed impressive results, but the amount of “misses” stops it from getting a great rating. Still worth a try if you have the patience to run a large batch of images waiting for the best results. The following samples came hand picked from a large batch run of random prompt phrases.

'a forest clearing' CLIP Guided Diffusion v4 Text-to-Image
a forest clearing

'a storybook illustration of a nightmare' CLIP Guided Diffusion v4 Text-to-Image
a storybook illustration of a nightmare

'an impressionist painting of a cemetery' CLIP Guided Diffusion v4 Text-to-Image
an impressionist painting of a cemetery

'Harry Potter in the style of Rembrandt' CLIP Guided Diffusion v4 Text-to-Image
Harry Potter in the style of Rembrandt

'a detailed painting of a witch' CLIP Guided Diffusion v4 Text-to-Image
a detailed painting of a witch

'a babbling brook' CLIP Guided Diffusion v4 Text-to-Image
a babbling brook

'a desert oasis' CLIP Guided Diffusion v4 Text-to-Image
a desert oasis

'a hyperrealistic painting of an android' CLIP Guided Diffusion v4 Text-to-Image
a hyperrealistic painting of an android

'eyeballs' CLIP Guided Diffusion v4 Text-to-Image
eyeballs

'a cross stitch of Buzz Lightyear' CLIP Guided Diffusion v4 Text-to-Image
a cross stitch of Buzz Lightyear


Name: CLIP Guided Decision Transformer
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1V66mUeJbXrTuQITvJunvnWVn96FEbSI3
Time for 512×512 on a 3090: 1 minutes 13 seconds
Maximum resolution on a 24 GB 3090: Locked to 384×384
Description: Another one from Katherine Crowson. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. When it gets a “hit” it can create very detailed impressive results, but the amount of “misses” stops it from getting a great rating. The following samples came hand picked from a large batch run of random prompt phrases.
Another good point for CLIP Decsision Transformer is that it will generate a batch of images from each run. So rather than a single image for the prompt text you can specify (for example) 8 images to be generated from the prompt. This allows a much larger set of images to be quickly generated to find those great outputs in.
For these images I have enhanced the resolution 4x using Real-ESRGAN (the thumnails are the original output images and the clicked images are resized x4).

a detailed painting of a palace by Thomas Kinkade
a detailed painting of a palace by Thomas Kinkade

a drawing of Chewbacca
a drawing of Chewbacca

a forest path
a forest path

a renaissance painting of a mountain range
a renaissance painting of a mountain range

a rough seascape
a rough seascape

a rough seascape
a rough seascape

a spooky forest
a spooky forest

an oil on canvas painting of a western town
an oil on canvas painting of a western town

Frankenstein
Frankenstein

The Grand Canyon
The Grand Canyon


Name: CLIPIT
Author: dribnet
Original script: https://github.com/dribnet/clipit
Time for 512×512 on a 3090: 2 minutes 38 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Another GAN+CLIP script. Gives nice results that tend to match the prompt text more closely. This one is heavy on VAM usage.

'a happy family by Piet Mondiran' CLIPIT
a happy family by Piet Mondiran

'a landscape' CLIPIT
a landscape

'a peacock' CLIPIT
a peacock

'a tropical beach by Thomas Kinkade' CLIPIT
a tropical beach by Thomas Kinkade

'a woodcut of Dracula' CLIPIT
a woodcut of Dracula

'an ambient occlusion render of a zombie' CLIPIT
an ambient occlusion render of a zombie

'eyeballs in the style of Claude Monet' CLIPIT
eyeballs in the style of Claude Monet


Name: Art Machine
Author: Hillel Wayne
Original script: https://colab.research.google.com/drive/1n_xrgKDlGQcCF6O-eL3NOd_x4NSqAUjK
Time for 512×512 on a 3090: 4 minutes 04 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Another VQGAN+CLIP scipt.

'a charcoal drawing of a kitchen' Art Machine
a charcoal drawing of a kitchen

'a mosaic of a mountain path | CryEngine' Art Machine
a mosaic of a mountain path | CryEngine

'a silk screen of a tropical beach in the style of Kandinsky' Art Machine
a silk screen of a tropical beach in the style of Kandinsky

'a woodcut of a nightmare creature' Art Machine
a woodcut of a nightmare creature

'an illustration of of a mountainscape' Art Machine
an illustration of of a mountainscape

'an ultrafine detailed painting of a green tree frog as created by Craig Mullins' Art Machine
an ultrafine detailed painting of a green tree frog as created by Craig Mullins

'Dracula' Art Machine
Dracula

'Planets' Art Machine
Planets


Name: VQGAN+CLIP v5
Author: Max Woolf
Original script: https://colab.research.google.com/drive/1wkF67ThUz37T2_oPIuSwuO4e_-0vjaLs
Time for 512×512 on a 3090: 2 minutes 13 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Another VQGAN+CLIP scipt. More abstract results from this one.

'a desert oasis in the style of Salvador Dali' VQGAN+CLIP v5
a desert oasis in the style of Salvador Dali

'a hyperrealistic painting of a dragon' VQGAN+CLIP v5
a hyperrealistic painting of a dragon

'Big Bird' VQGAN+CLIP v5
Big Bird

'Cthulhu' VQGAN+CLIP v5
Cthulhu

'Robert DeNiro' VQGAN+CLIP v5
Robert DeNiro

'Yoda' VQGAN+CLIP v5
Yoda “hmmm, abstract I am”


Name: Zoetrope 5.5
Author: Bearsharktopusdev
Original script: https://colab.research.google.com/drive/1LpEbICv1mmta7Qqic1IcRTsRsq7UKRHM
Time for 512×512 on a 3090: 3 minutes 27 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Updated version of Zoetrope 5. Supports more VQGAN models, CLIP models and optimizers compared to Zoetrope 5.

'a cephalopod' Zoetrope 5.5 Text-to-Image
a cephalopod

'a flemish baroque of a demon' Zoetrope 5.5 Text-to-Image
a flemish baroque of a demon

'a photo of a submarine in the style of Vincent van Gogh' Zoetrope 5.5 Text-to-Image
a photo of a submarine in the style of Vincent van Gogh

'a snail' Zoetrope 5.5 Text-to-Image
a snail

'Cthulhu' Zoetrope 5.5 Text-to-Image
Cthulhu

'flesh' Zoetrope 5.5 Text-to-Image
flesh


Name: Zeta Quantize
Author: afiaka87
Original script: https://colab.research.google.com/gist/afiaka87/a97cca3b54c02209b94ff805224f9eb5/zeta_quantize.ipynb
Time for 512×512 on a 3090: 4 minutes 18 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Another VQGAN+CLIP scipt.

'a cute creature made of silver' Zeta Quantize
a cute creature made of silver

'a detailed painting of a cephalopod' Zeta Quantize
a detailed painting of a cephalopod

'a detailed painting of a ghost' Zeta Quantize
a detailed painting of a ghost

'a forest fire made of copper' Zeta Quantize
a forest fire made of copper

'a peacock' Zeta Quantize
a peacock

'a sketch of a Pokemon character in the style of Odilon Redon' Zeta Quantize
a sketch of a Pokemon character in the style of Odilon Redon

'a watercolor painting of dense woodland' Zeta Quantize
a watercolor painting of dense woodland


Name: Experimental VQGAN
Author: Various
Original script: https://colab.research.google.com/drive/1jx3klUxlGbYUwvtqzC9SYl4XZKHL3R81
Time for 512×512 on a 3090: 1 minutes 12 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Very nice smooth results from this one.

'a desert oasis in the style of Craig Mullins' Experimental VQGAN
a desert oasis in the style of Craig Mullins

'a dragon' Experimental VQGAN
a dragon

'a manga drawing of a happy alien' Experimental VQGAN
a manga drawing of a happy alien

'a nightmare' Experimental VQGAN
a nightmare

'a surrealist painting of love' Experimental VQGAN
a surrealist painting of love

'a watercolor painting of a lighthouse' Experimental VQGAN
a watercolor painting of a lighthouse

'an airbrush painting of a well kept garden by Piet Mondiran' Experimental VQGAN
an airbrush painting of a well kept garden by Piet Mondiran

'Cookie Monster' Experimental VQGAN
Cookie Monster


Name: SlideShowVisions
Author: Active Galaxy
Original script: https://colab.research.google.com/drive/1IihC4ZJvCh_tOgBVd900BzHX-ulPEFsa
Time for 512×512 on a 3090: 2 minutes 25 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Tends to give more abstract paper cutout looks.

'a happy child' SlideShowVisions
a happy child

'a house vivid colors' SlideShowVisions
a house vivid colors

'a sea monster' SlideShowVisions
a sea monster

'a thunder storm' SlideShowVisions
a thunder storm

'a tree' SlideShowVisions
a tree

'a woodcut of war' SlideShowVisions
a woodcut of war

'an engraving of zombies' SlideShowVisions
an engraving of zombies

'Han Solo' SlideShowVisions
Han Solo


Name: Quick CLIP Guided Diffusion
Author: Daniel Russell
Original script: https://colab.research.google.com/drive/1FuOobQOmDJuG7rGsMWfQa883A9r4HxEO
Time for 512×512 on a 3090: 43 seconds
Maximum resolution on a 24 GB 3090: 512×512
Description: Modified version of CLIP Guided Diffusion that gets results quicker. Option for 256×256 or 512×512 sized images. Still very hit and miss when getting images that resemble the input prompt. The following samples came from a large overnight batch run of random prompts.

'a cathedral' Quick CLIP Guided Diffusion
a cathedral

'a digital painting of a space nebula' Quick CLIP Guided Diffusion
a digital painting of a space nebula

'a lounge room' Quick CLIP Guided Diffusion
a lounge room

'a monkey | lens flare' Quick CLIP Guided Diffusion
a monkey | lens flare

'a nightmare creature' Quick CLIP Guided Diffusion
a nightmare creature

'a rough seascape' Quick CLIP Guided Diffusion
a rough seascape

'a landscape' Quick CLIP Guided Diffusion
a landscape

'an android' Quick CLIP Guided Diffusion
an android

'an attractive woman' Quick CLIP Guided Diffusion
an attractive woman

'an oil on canvas painting of a cloudy sunset' Quick CLIP Guided Diffusion
an oil on canvas painting of a cloudy sunset


Name: CLIP Guided Diffusion v5
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1QBsaDAZv8np29FPbvjffbE1eytoJcsgA
Time for 512×512 on a 3090: 3 minutes 48 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Description: Another CLIP Guided Diffusion script. Locked to 512×512 resolution. Needs less VRAM than the previous versions. The following samples came hand picked from a large batch run of random prompt phrases.

'a cityscape' CLIP Guided Diffusion v5 Text-to-Image
a cityscape

'a gorilla' CLIP Guided Diffusion v5 Text-to-Image
a gorilla

'Cthulhu by Craig Mullins' CLIP Guided Diffusion v5 Text-to-Image
Cthulhu by Craig Mullins

'computer rendering of Emporer Palpatine made of cheese by Evan Charlton' CLIP Guided Diffusion v5 Text-to-Image
computer rendering of Emporer Palpatine made of cheese by Evan Charlton

'digital art of a mountainscape as created by Persis Goodale Thurston Taylor' CLIP Guided Diffusion v5 Text-to-Image
digital art of a mountainscape as created by Persis Goodale Thurston Taylor

'a digital rendering of Chewbacca' CLIP Guided Diffusion v5 Text-to-Image
a digital rendering of Chewbacca

'an ugly person' CLIP Guided Diffusion v5 Text-to-Image
an ugly person

See this tweet for an example of using CLIP Guided Diffusion to stylize a portrait.


Name: MSE Regulized Modified
Author: jbusted
Original script: https://colab.research.google.com/drive/1gFn9u3oPOgsNzJWEFmdK-N9h_y65b8fj
Time for 512×512 on a 3090: 3 minutes 02 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Modified and updated version of the previous “MSE Regulized VQGAN+CLIP” script. Less likely to suffer the previous script’s issue of subjects floating in a purple void.

'a bronze sculpture of a planet' MSE Regulized Modified Text-to-Image
a bronze sculpture of a planet

'a cave by Asher Brown Durand' MSE Regulized Modified Text-to-Image
a cave by Asher Brown Durand

'a charcoal drawing of Emporer Palpatine' MSE Regulized Modified Text-to-Image
a charcoal drawing of Emporer Palpatine

'a cozy den' MSE Regulized Modified Text-to-Image
a cozy den

'a detailed drawing of a heart made of string by William MacTaggart' MSE Regulized Modified Text-to-Image
a detailed drawing of a heart made of string by William MacTaggart

'a digital rendering of Arnold Schwarzenegger made of metal by Muriel Brandt' MSE Regulized Modified Text-to-Image
a digital rendering of Arnold Schwarzenegger made of metal by Muriel Brandt

'a lounge room' MSE Regulized Modified Text-to-Image
a lounge room

'a palace by Jules Joseph Lefebvre' MSE Regulized Modified Text-to-Image
a palace by Jules Joseph Lefebvre

'an oil on canvas painting of a lush rainforest' MSE Regulized Modified Text-to-Image
an oil on canvas painting of a lush rainforest

'an oil on canvas painting of Cookie Monster' MSE Regulized Modified Text-to-Image
an oil on canvas painting of Cookie Monster


Name: Pixray
Author: dribnet
Original script: https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/Start_Here.ipynb
Time for 512×512 on a 3090: 1 minutes 44 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Description: Updated version of the previous “CLIPIT” script.

'a bronze sculpture of a nightmare creature' Pixray Text-to-Image
a bronze sculpture of a nightmare creature

'a fire breathing dragon by Jan Baptist Weenix' Pixray Text-to-Image
a fire breathing dragon by Jan Baptist Weenix

'a morning landscape' Pixray Text-to-Image
a morning landscape

'a surrealist sculpture of an elephant' Pixray Text-to-Image
a surrealist sculpture of an elephant

'a watercolor painting of an astronaut' Pixray Text-to-Image
a watercolor painting of an astronaut

'an oil painting of a worried woman | Rendered in Cinema4D' Pixray Text-to-Image
an oil painting of a worried woman | Rendered in Cinema4D

'an ugly creature' Pixray Text-to-Image
an ugly creature

'Dracula' Pixray Text-to-Image
Dracula

'Frankenstein' Pixray Text-to-Image
Frankenstein

'vector art of a forest clearing' Pixray Text-to-Image
vector art of a forest clearing


Name: CLIP Guided Diffusion v6
Author: Dango233
Original script: https://colab.research.google.com/drive/14xBm1aSxQLbq26-jmDJi8I1HJ4ti5ybt
Time for 512×512 on a 3090: 3 minutes 10 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Description: Latest CLIP Guided Diffusion script. The best one yet. Capable of some very nice results.

'a hyperrealistic painting of a human' CLIP Guided Diffusion v6 Text-to-Image
a hyperrealistic painting of a human

'a sketch of planets' CLIP Guided Diffusion v6 Text-to-Image
a sketch of planets

'a storybook illustration of a cloudy sunset' CLIP Guided Diffusion v6 Text-to-Image
a storybook illustration of a cloudy sunset

'a wizard | vivid colors' CLIP Guided Diffusion v6 Text-to-Image
a wizard | vivid colors

'an art deco sculpture of a planet' CLIP Guided Diffusion v6 Text-to-Image
an art deco sculpture of a planet

'an attractive man by John Linnell' CLIP Guided Diffusion v6 Text-to-Image
an attractive man by John Linnell

'an oil on canvas painting of satan' CLIP Guided Diffusion v6 Text-to-Image
an oil on canvas painting of satan

'an oil painting of a clown' CLIP Guided Diffusion v6 Text-to-Image
an oil painting of a clown

'digital art of an ugly person by Avigdor Arikha' CLIP Guided Diffusion v6 Text-to-Image
digital art of an ugly person by Avigdor Arikha

'princess in sanctuary trending on artstation photorealistic portrait of a young princess' CLIP Guided Diffusion v6 Text-to-Image
princess in sanctuary trending on artstation photorealistic portrait of a young princess


Name: CLIPDraw
Author: Kevin Frans
Original script: https://colab.research.google.com/github/kvfrans/clipdraw/blob/main/clipdraw.ipynb
Time for 512×512 on a 3090: 7 minutes 10 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Description: Generates images by a series of lines. Very abstract results.

'a cloudy sunset' CLIPDraw Text-to-Image
a cloudy sunset

'a digital painting of a rose' CLIPDraw Text-to-Image
a digital painting of a rose

'a sad clown' CLIPDraw Text-to-Image
a sad clown

'an abstract painting of Yoda' CLIPDraw Text-to-Image
an abstract painting of Yoda

'an etching of a library' CLIPDraw Text-to-Image
an etching of a library

'The Sydney Harbour Bridge' CLIPDraw Text-to-Image
The Sydney Harbour Bridge


Continued in Part 4.

Jason.

Text-to-Image Summary – Part 2

This is Part 2. See Part 1 for the previous post.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.


Name: VQGAN Gumbel
Author: Eleiber
Original script: https://colab.research.google.com/drive/1tim3xTsZXafK-A2rOUsevckdl4OitIiw
Time for 512×512 on a 3090: 3 minutes 27 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Variation using the gumbel-8192 model. Results are a bit rougher than others.

'a childs drawing of a space nebula' VQGAN Gumbel Text-to-Image
a childs drawing of a space nebula

'a movie monster in the style of Edvard Munch' VQGAN Gumbel Text-to-Image
a movie monster in the style of Edvard Munch

'a raytraced image of the Amazon Rainforest' VQGAN Gumbel Text-to-Image
a raytraced image of the Amazon Rainforest

'a tropical beach in the style of Polock' VQGAN Gumbel Text-to-Image
a tropical beach in the style of Polock

'digital art of a rose' VQGAN Gumbel Text-to-Image
digital art of a rose


Name: OpenAI DVAE+CLIP
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/10DzGECHlEnL4oeqsN-FWCkIe_sq3wVqt
Time for 512×512 on a 3090: 3 minutes 07 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Results are very colorful and more abstract. By default it gives more noisy output images but this can be disabled if you prefer.

'a dragon' OpenAI DVAE+CLIP Text-to-Image
a dragon

'a hyperrealistic painting of planets' OpenAI DVAE+CLIP Text-to-Image
a hyperrealistic painting of planets

'a mountain cabin' OpenAI DVAE+CLIP Text-to-Image
a mountain cabin

'a woodcut of a mountain range in the style of Marvel Comics' OpenAI DVAE+CLIP Text-to-Image
a woodcut of a mountain range in the style of Marvel Comics

'an angry person' OpenAI DVAE+CLIP Text-to-Image
an angry person


Name: Aphantasia
Author: Vadim Epstein
Original script: https://github.com/eps696/aphantasia
Time for 512×512 on a 3090: 1 minute 5 seconds
Maximum resolution on a 24 GB 3090: 4096×4096 or 2520×1080
Description: Different and more messy pastel abstract “Turneresque” output. I spent a few hours trying many different combinations of settings trying to get the output more coherent and deeper colors. The following samples are as good as I could push it. I give up for now. If you can do better let me know. It does support creating larger 1280×720 resolution images on a 3090 GPU.

'a marble sculpture of a computer' Aphantasia Text-to-Image
a marble sculpture of a computer

'an eyeball' Aphantasia Text-to-Image
an eyeball

'an octopus' Aphantasia Text-to-Image
an octopus

'digital art of frogs in the style of Dr Seuss' Aphantasia Text-to-Image
digital art of frogs in the style of Dr Seuss

'medusa' Aphantasia Text-to-Image
medusa


Name: Text2Image VQGAN
Author: Vadim Epstein
Original script: https://colab.research.google.com/github/eps696/aphantasia/blob/master/CLIP_VQGAN.ipynb
Time for 512×512 on a 3090: 2 minutes 8 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Allows larger sized 480p images (854×480) on a 3090 GPU.

'a digital painting of the Las Vegas strip' Text2Image VQGAN Text-to-Image
a digital painting of the Las Vegas strip

'a midnineteenth century engraving of a cute monster' Text2Image VQGAN Text-to-Image
a midnineteenth century engraving of a cute monster

'a skeleton' Text2Image VQGAN Text-to-Image
a skeleton

'an ultrafine detailed painting of a crying person' Text2Image VQGAN Text-to-Image
an ultrafine detailed painting of a crying person

'puppies' Text2Image VQGAN Text-to-Image
puppies


Name: MSE VQGAN+CLIP z+quantize
Author: jbusted
Original script: https://colab.research.google.com/drive/1gFn9u3oPOgsNzJWEFmdK-N9h_y65b8fj
Time for 512×512 on a 3090: 6 minutes 19 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Awesome crisp results. Allows larger sized 480p images (854×480) on a 3090 GPU. One of the best scripts in this list worth exploring.

'a charcoal drawing of a country town' MSE VQGAN+CLIP z+quantize Text-to-Image
a charcoal drawing of a country town

'a hyperrealistic painting of an ugly creature' MSE VQGAN+CLIP z+quantize Text-to-Image
a hyperrealistic painting of an ugly creature

'a landscape made of mist' MSE VQGAN+CLIP z+quantize Text-to-Image
a landscape made of mist

'a mosaic of christmas' MSE VQGAN+CLIP z+quantize Text-to-Image
a mosaic of christmas

'an octopus in the style of Vincent van Gogh' MSE VQGAN+CLIP z+quantize Text-to-Image
an octopus in the style of Vincent van Gogh

MSE VQGAN+CLIP z+quantize allows specifying an image as the input starting point. If you take the output and repeatedly use it as the input with some minor image stretching each frame you can get a movie zooming into the Text-to-Image output. No blending of frames or optical flow for this one, just straight combining of the 854×480 resolution frames into a movie. The VQGAN model was “vqgan_imagenet_f16_16384” and the CLIP model was “ViT-B/32”. The prompts for this movie were “hyperrealistic homer simpson”, “hyperrealistic marge simpson”, “hyperrealistic bart simpson”, “hyperrealistic lisa simpson” and “hyperrealistic maggie simpson”. The original 480p upload was badly compressed and looked terrible after YouTube compressed it, so I upscaled the 480p to 2160p (4K) in DaVinci Resolve and reuploaded to YouTube. This caused their compression to do a better encoding job so the movie is now watchable.

This next example is how MSE VQGAN+CLIP z+quantize interprets various common human phobias. Text prompts were “a hyperrealistic painting depicting acrophobia” etc. To try and smooth out the “flickering” when zooming I started using ImageMagick for zooming. ImageMagick allows sub pixel image resizing options. This movie was also originally 480p and upsized to 4K in Davinci Resolve before uploading.

I have also added some basic scripting (as in automating a series of steps rather than a Python py script) support to Visions of Chaos. Scripting allows the prompt, zoom speed, rotation and panning to be changed during the movie with smooth interpolations between them each frame.

Text-to-Image Script GUI

The following video is a test of the scripting. This video is a Powers of Ten homage with zooming in from the largest scales to the smallest scales.

Another recent addition is the ability to use a series of images as “seed images” that are processed one at a time and then combined into a movie. The following GIF of the Alien chestburster scene is an example of this. The Text-to-Image prompt was “impasto oil painting”.

This next example movie is showing a “Self-Driven” zoom movie. As in a regular zoom movie the output frames are slightly stretched and fed back into the system each frame. The self-driven difference with this movie is that the Text-to-Image prompt text is automatically changed every 2 seconds by CLIP detecting what it “sees” in the current frame. This way the movie subjects are automatically changed and steered in new directions in a totally automated way. There is no human control except me setting the initial “Rainbow colored blobs” prompt. After that it was fully automated.

By default the CLIP Image Captioning script is very good at detecting what is in an image. Using the default accuracy resulted in a zoom movie that got stuck with a single topic or subject. One got stuck on a slight variation of a prompt dealing with kites, so as the zoom movie went deeper it only showed kites. Luckily after tweaking and decreasing the accuracy of the CLIP captioning the predicitons allow the resulting subjects to drift to new topics during the movie.


Name: Monster Maker
Author: P_Hoep
Original script: https://colab.research.google.com/drive/1ZbLnt5fLS_BDfpQY-9Dh_T40pLjfqSAC
Time for 512×512 on a 3090: 2 minutes 01 seconds
Description: No longer available. I was contacted by the author who does not want it shared publicly. The colab link no longer works.

'a black and white photo of a library in the style of Rembrandt' Monster Maker Text-to-Image
a black and white photo of a library in the style of Rembrandt

'a forest fire' Monster Maker Text-to-Image
a forest fire

'a forest path' Monster Maker Text-to-Image
a forest path

'a heart made of feathers' Monster Maker Text-to-Image
a heart made of feathers

'a surrealist painting of the Las Vegas strip' Monster Maker Text-to-Image
a surrealist painting of the Las Vegas strip


Name: CLIP Guided Diffusion
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj
Time for 256×256 on a 3090: 1 minutes 35 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256
Description: This one gives very unique results compared to the other scripts. Locked to 256×256 resolution. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

'a clown' CLIP Guided Diffusion Text-to-Image
a clown

'a hyperrealistic painting of a witch' CLIP Guided Diffusion Text-to-Image
a hyperrealistic painting of a witch

'a sea monster' CLIP Guided Diffusion Text-to-Image
a sea monster

'a surrealist sculpture of an android' CLIP Guided Diffusion Text-to-Image
a surrealist sculpture of an android

'Brad Pitt' CLIP Guided Diffusion Text-to-Image
Brad Pitt

'New York City' CLIP Guided Diffusion Text-to-Image
New York City


Name: CLIP Guided Diffusion v2
Author: afiaka87
Original script: https://colab.research.google.com/github/afiaka87/clip-guided-diffusion/blob/main/colab_clip_guided_diff_hq.ipynb
Time for 256×256 on a 3090: 2 minutes 38 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256
Description: Modified CLIP Guided Diffusion with more options. This one gives very unique results compared to the other scripts. Locked to 256×256 resolution. Hopefully larger resolution versions of this script will appear in the future. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

'a digital painting of a crying person' CLIP Guided Diffusion v2 Text-to-Image
a digital painting of a crying person

'a fine art painting of heaven in the style of Edvard Munch' CLIP Guided Diffusion Text-to-Image
a fine art painting of heaven in the style of Edvard Munch

'a flemish baroque of an angry person' CLIP Guided Diffusion v2 Text-to-Image
a flemish baroque of an angry person

'a flemish baroque of hell' CLIP Guided Diffusion v2 Text-to-Image
a flemish baroque of hell

'a surrealist painting of a witch' CLIP Guided Diffusion v2 vText-to-Image
a surrealist painting of a witch

'the australian outback' CLIP Guided Diffusion v2 Text-to-Image
the australian outback


Name: CLIPRGB
Author: Jonathan Whitaker
Original script: https://colab.research.google.com/drive/1MiKaFFgau6V5QhIed5tpNdLUiSbof4nI
Time for 512×512 on a 3090: 4 minutes 51 seconds
Maximum resolution on a 24 GB 3090: 4096×4096
Description: Very early 0.1 version shows a lot of potential. Can render huge resolution images up to 4096×4096 on a 3090 so I am really looking forward to future versions of this code with sharper details.

'a digital painting of a wizard' CLIPRGB
a digital painting of a wizard

'a forest path' CLIPRGB
a forest path

'a tattoo of planets' CLIPRGB
a tattoo of planets

'a vampire' CLIPRGB
a vampire


Name: CLIP Guided Diffusion v3
Author: Michael Friesen
Original script: https://colab.research.google.com/drive/1Fl2SZvLv23MVSAHxkoiNdxPeAZwibvu1
Time for 512×512 on a 3090: 2 minutes 23 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Description: Modified CLIP Guided Diffusion that generates larger 512×512 images. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

'a cubist painting of a castle' CLIP Guided Diffusion v2 Text-to-Image
a cubist painting of a castle

'a human made of vines' CLIP Guided Diffusion Text-to-Image
a human made of vines

'a rough seascape' CLIP Guided Diffusion v2 Text-to-Image
a rough seascape

'frogs' CLIP Guided Diffusion v2 Text-to-Image
frogs

'h r giger' CLIP Guided Diffusion v2 Text-to-Image
h r giger

'a matte painting of a landscape' CLIP Guided Diffusion v2 Text-to-Image
a matte painting of a landscape


Name: Zoetrope 5
Author: Bearsharktopusdev
Original script: https://colab.research.google.com/drive/1LpEbICv1mmta7Qqic1IcRTsRsq7UKRHM
Time for 512×512 on a 3090: 2 minutes 36 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1280×720
Description: Nice crisp results. Can generates up to 720p (1280×720) resolution images on a 3090. Includes a lot of new ideas from multiple people to help improve the outputs.

'a detailed painting of a Pixar character' Zoetrope 5 Text-to-Image
a detailed painting of a Pixar character

'a futuristic city' Zoetrope 5 Text-to-Image
a futuristic city

'a planet' Zoetrope 5 Text-to-Image
a planet

'a surrealist sculpture of a sea monster' Zoetrope 5 Text-to-Image
a surrealist sculpture of a sea monster

'an art deco scultpture of a policeman' Zoetrope 5 Text-to-Image
an art deco scultpture of a policeman

'cyberpunk art of a forest fire in the style of Edvard Munch' Zoetrope 5 Text-to-Image
cyberpunk art of a forest fire in the style of Edvard Munch


Name: CLIP RGB Optimization
Author: hotgrits
Original script: https://cdn.discordapp.com/attachments/730484623028519072/871624258260987934/CLIP__RGB_Optimization_v0_3.ipynb
Time for 512×512 on a 3090: 2 minutes 50 seconds
Maximum resolution on a 24 GB 3090: 4096×4096
Description: Another CLIP RGB based script without the pixelated artefacts of the CLIPRGB script. Can render huge resolution images up to 4096×4096 on a 3090. This script gives more impressionistic textures. By default the output was a bit too dark for my liking so I have added options to tweak the gamma and contrast of the output images in the script. The gamma and contrast tweaks are only at the display stage and do not change the internal image being generated.

'a babbling brook' CLIP RGB Optimization
a babbling brook

'a movie monster' CLIP RGB Optimization
a movie monster

'an amusement park' CLIP RGB Optimization
an amusement park

'Chewbacca' CLIP RGB Optimization
Chewbacca

'Freddy Kruger in the style of Rembrandt' CLIP RGB Optimization
Freddy Kruger in the style of Rembrandt


Name: MSE Regulized VQGAN+CLIP
Author: jbusted
Original script: https://colab.research.google.com/drive/1hf1seGOZctOJUznkhJNblLluXHbWLKZh
Time for 512×512 on a 3090: 3 minutes 16 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Generates good images but they tend to be inside a grey/purple border void.

'a bronze sculpture of a heart' MSE Regulized VQGAN+CLIP
a bronze sculpture of a heart

'a cubist painting of Buzz Lightyear' MSE Regulized VQGAN+CLIP
a cubist painting of Buzz Lightyear

'a house made of string' MSE Regulized VQGAN+CLIP
a house made of string

'an art deco sculpture of a vampire' MSE Regulized VQGAN+CLIP
an art deco sculpture of a vampire

'chalk art of C-3PO' MSE Regulized VQGAN+CLIP
chalk art of C-3PO


Name: Sequential VQGAN+CLIP
Author: Jakeukalane and Avengium
Original script: https://colab.research.google.com/drive/1CcibxlLDng2yzcjLwwwSADRcisc1qVCs
Time for 512×512 on a 3090: 1 minutes 41 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Description: Really nice results and fast.

'a campfire in the style of Vincent van Gogh' Sequential VQGAN+CLIP
a campfire in the style of Vincent van Gogh

'a colorful parrot' Sequential VQGAN+CLIP
a colorful parrot

'a hyperrealistic painting of C-3PO' Sequential VQGAN+CLIP
a hyperrealistic painting of C-3PO

'an impressionist painting of Buzz Lightyear made of paper' Sequential VQGAN+CLIP
an impressionist painting of Buzz Lightyear made of paper

'New York City' Sequential VQGAN+CLIP
New York City


Name: CLIPRGB ImStack
Author: Jonathan Whitaker
Original script: https://colab.research.google.com/drive/1MCC2IwAaRNCTBUzghuG41ypAkxjJvGtq
Time for 512×512 on a 3090: 2 minutes 07 seconds
Maximum resolution on a 24 GB 3090: 2048×2048
Description: Another CLIP RGB variation. Nice results after some brightness, contrast and sharpness tweaks to the generated images. Could still be a bit sharper.

'a fine art painting of an angry person' CLIPRGB ImStack
a fine art painting of an angry person

'a fireplace in the style of Claude Monet' CLIPRGB ImStack
a fireplace in the style of Claude Monet

'a frog in the style of Beksinski' CLIPRGB ImStack
a frog in the style of Beksinski

'a nightmare creature in the style of H R Giger' CLIPRGB ImStack
a nightmare creature in the style of H R Giger

'a pointalism painting of a vampire made of copper' CLIPRGB ImStack
a pointalism painting of a vampire made of copper


Continued in Part 3 and Part 4.

Jason.