Text-to-Image Summary – Part 4

This is Part 4. There is also Part 1, Part 2, Part 3, Part 5, Part 6, Part 7 and Part 8.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.


Name: PixelDraw
Author: dribnet
Original script: https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/PixelDrawer.ipynb
Time for 512×512 on a 3090: 1 minutes 59 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Generates “pixel art” images. I had a lot of requests to add support for this one.

'a cartoon of a peacock' PixelDraw Text-to-Image
a cartoon of a peacock

'a cloudy sunset' PixelDraw Text-to-Image
a cloudy sunset

'a gorilla' PixelDraw Text-to-Image
a gorilla

'a morning landcsape' PixelDraw Text-to-Image
a morning landscape

'a watercolor painting of a castle' PixelDraw Text-to-Image
a watercolor painting of a castle

'an art deco painting of Al Pacino' PixelDraw Text-to-Image
an art deco painting of Al Pacino

'Hell' PixelDraw Text-to-Image
Hell

'Shrek' PixelDraw Text-to-Image
Shrek


Name: DirectVisions
Author: Jens Goldberg
Original script: https://colab.research.google.com/drive/127lKSsQjx-UDDUSvIkLL6mREfZ0KQu5D
Time for 512×512 on a 3090: 2 minutes 39 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 4096×4096
Description: Interesting detailed images. Can create huge resolution results.

'a color pencil sketch of a western town' DirectVisions Text-to-Image
a color pencil sketch of a western town

'a detailed painting of a cephalopod' DirectVisions Text-to-Image
a detailed painting of a cephalopod

'a digital rendering of an ugly face' DirectVisions Text-to-Image
a digital rendering of an ugly face

'a pencil sketch of Buzz Lightyear' DirectVisions Text-to-Image
a pencil sketch of Buzz Lightyear

'a rough seascape by Pinchus Kremegne' DirectVisions Text-to-Image
a rough seascape by Pinchus Kremegne

'a stock photo of a president' DirectVisions Text-to-Image
a stock photo of a president

'a sunset' DirectVisions Text-to-Image
a sunset

'an alien city' DirectVisions Text-to-Image
an alien city

'an alien forest by Helen Berman' DirectVisions Text-to-Image
an alien forest by Helen Berman

'an evening landscape' DirectVisions Text-to-Image
an evening landscape


Name: Pixel Direct
Author: Unknown
Original script: https://colab.research.google.com/drive/1F9ZOZnpV3uBPRDSESaAXYwzNZJQRJT75
Time for 512×512 on a 3090: 1 minutes 03 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 2048×2048 1 minute 51 seconds
Description: Another “Pixel Art” script. More abstract results than the PixelDraw script above.

'a bronze sculpture of a nightmare creature' Pixel Direct Text-to-Image
a bronze sculpture of a nightmare creature

'a cartoon of Al Pacino' Pixel Direct Text-to-Image
a cartoon of Al Pacino

'a nightclub' Pixel Direct Text-to-Image
a nightclub

'a silk screen of a bouquet of flowers' Pixel Direct Text-to-Image
a silk screen of a bouquet of flowers

'an etching of a worried woman' Pixel Direct Text-to-Image
an etching of a worried woman

'an illustration of of a thunder storm' Pixel Direct Text-to-Image
an illustration of of a thunder storm


Name: FourierVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1nGNBjhbYnDHSumGPjpFHjDOsaZFAqGgF
Time for 512×512 on a 3090: 1 minutes 40 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 1024×1024 4 minutes 07 seconds
Description: Detailed images. The default script generates washed out pastel images, but with some gamma and brightness tweaks they can be improved (still not ideal, but better). Allows very large resolution images.

'a cathedral' FourierVisions Text-to-Image
a cathedral

'a charcoal drawing of zombies' FourierVisions Text-to-Image
a charcoal drawing of zombies

'a detailed painting of a sunset by Thomas Cantrell Dugdale' FourierVisions Text-to-Image
a detailed painting of a sunset by Thomas Cantrell Dugdale

'a ghost made of mist' FourierVisions Text-to-Image
a ghost made of mist

'a kitchen' FourierVisions Text-to-Image
a kitchen

'a movie monster' FourierVisions Text-to-Image
a movie monster

'a pencil sketch of a sad clown' FourierVisions Text-to-Image
a pencil sketch of a sad clown

'a werewolf' FourierVisions Text-to-Image
a werewolf

'an evil clown by Viktor Oliva' FourierVisions Text-to-Image
an evil clown by Viktor Oliva

'an ink drawing of an ugly monster' FourierVisions Text-to-Image
an ink drawing of an ugly monster


Name: PyramidVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1dpAS_wK34y7c6s-CatAFmBtbkjGT_erM
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 1024×1024 10 minutes 48 seconds
Description: Very detailed images. Not the fastest script, but gives some very nice results. Lower VRAM requirements so good for lesser spec GPUs. Definitely one of the better scripts worth exploring.

'a desert oasis' PyramidVisions Text-to-Image
a desert oasis

'a lush rainforest' PyramidVisions Text-to-Image
a lush rainforest

'a marble sculpture of an angry person' PyramidVisions Text-to-Image
a marble sculpture of an angry person

'a minimalist painting of the Amazon Rainforest' PyramidVisions Text-to-Image
a minimalist painting of the Amazon Rainforest

'a nightmare creature' PyramidVisions Text-to-Image
a nightmare creature

'a pastel of a computer made of paper' PyramidVisions Text-to-Image
a pastel of a computer made of paper

'an abstract sculpture of a sad clown' PyramidVisions Text-to-Image
an abstract sculpture of a sad clown

'an acrylic painting of an alien forest | vivid colors' PyramidVisions Text-to-Image
an acrylic painting of an alien forest | vivid colors

'Medusa' PyramidVisions Text-to-Image
Medusa

'vector art of an ugly woman' PyramidVisions Text-to-Image
vector art of an ugly woman


Name: Visions of AI v1
Author: Jason Rampe
Original script: Included with Visions of Chaos. No colab.
Time for 512×512 on a 3090: 1 minutes 32 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 33 seconds
Description: My first attempt at actually creating a Text-to-Image script. Based on the excellent example from Jonathan Whitaker‘s AIAIArt Lesson 3 tutorial. Gives some very nice fine detail in some areas, but suffers the non coherance of other scripts in that it creates multiple copies of the subject throughout the image. After actually trying to write my own script I only have more respect for those who can do this. Hopefully I can improve these results for a version 2. In the meantime, here are some sample from the current Visions of AI script.

'a cartoon of the human condition by Judy Takács' Visions of AI Text-to-Image
a cartoon of the human condition by Judy Takács

'a cubist painting of an evening landscape' Visions of AI Text-to-Image
a cubist painting of an evening landscape

'a digital rendering of frogs' Visions of AI Text-to-Image
a digital rendering of frogs

'a fire breathing dragon' Visions of AI Text-to-Image
a fire breathing dragon

'a hyperrealistic painting of a movie monster' Visions of AI Text-to-Image
a hyperrealistic painting of a movie monster

'a morning landscape' Visions of AI Text-to-Image
a morning landscape

'a shark' Visions of AI Text-to-Image
a shark

'a woodcut of an ugly man' Visions of AI Text-to-Image
a woodcut of an ugly man

'an airbrush painting of C-3PO' Visions of AI Text-to-Image
an airbrush painting of C-3PO

'Frankenstein' Visions of AI Text-to-Image
Frankenstein


Name: Visions of AI v2
Author: Jason Rampe
Original script: Included with Visions of Chaos. No colab.
Time for 512×512 on a 3090: 2 minutes 35 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480.
Maximum resolution on an 8GB 2080: 256×256 2 minutes 36 seconds
Description: An attempt to improve the coherency of the previous script. The first 30 iterations zoom into the image every 10 frames. This results in larger shapes/blobs for the rest of the script to work from. The idea is that it will give larger subjects compared to the v1 script. Kind of works. Gives blurrier results. To be fixed in the next version?

'a morning landscape by William Gear' Visions of AI v2 Text-to-Image
a morning landscape by William Gear

'a raytraced image of a nightclub lens flare' Visions of AI v2 Text-to-Image
a raytraced image of a nightclub lens flare

'a tentacle monster by Carlo Crivelli' Visions of AI v2 Text-to-Image
a tentacle monster by Carlo Crivelli

'a woodcut of a worried woman by Li Keran' Visions of AI v2 Text-to-Image
a woodcut of a worried woman by Li Keran

'an illustration of of a cave made of cheese' Visions of AI v2 Text-to-Image
an illustration of of a cave made of cheese

'Cthulhu' Visions of AI v2 Text-to-Image
Cthulhu

'cyberpunk art of a futuristic city' Visions of AI v2 Text-to-Image
cyberpunk art of a futuristic city

'goldfish' Visions of AI v2 Text-to-Image
goldfish

'reflective spheres' Visions of AI v2 Text-to-Image
reflective spheres

'the Australian outback' Visions of AI v2 Text-to-Image
the Australian outback


Name: Multi-Perceptor CLIP Guided Diffusion
Author: Varkarrus
Original script: https://colab.research.google.com/drive/1y3Vt39A5KSNFRa6Z2bCqDHxteZSVH9NC
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: 896×512 or 1152×384 (dimensions must be divisible by 128).
Maximum resolution on an 8GB 2080: 128×128 1 minute 56 seconds
Description: Builds upon previous CLIP Guided Diffusion scripts. Like the previous script by Dango233 it uses three CLIP models simultaneously to “rate” the generated images, and I have added options to use up to six different CLIP models. The resulting image accuracy compared to the prompt, and the resulting image coherence seem to be much better than previous CLIP Guided Diffusion scripts that could almost have random outputs sometimes. This script is superb and highly recommended. Great lighting, textures and brushstrokes. Normally with these blog posts I do a batch run of random prompts overnight and then pick the best 10 images. In this case I had nearly 50 images in my “good” folder after going through the batch results. So, for this script I am showing 20 sample images.

'a cute creature | TriX 400 TX' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a cute creature | TriX 400 TX

'a digital painting of Frankenstein by Kanzan Shimomura' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a digital painting of Frankenstein by Kanzan Shimomura

'a morning landscape by János SaxonSzász' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a morning landscape by János SaxonSzász

'a nightmare creature' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a nightmare creature

'a photorealistic painting of a teddy bear' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a photorealistic painting of a teddy bear

'a portrait of a young girl' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a portrait of a young girl

'a space nebula | IMAX' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a space nebula | IMAX

'a worried man' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a worried man

'a zombie by Nathaniel Hone' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
a zombie by Nathaniel Hone

'an acrylic painting of a spider by Abram Arkhipov' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
an acrylic painting of a spider by Abram Arkhipov

'an airbrush painting of a monkey by Jeremy Henderson' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
an airbrush painting of a monkey by Jeremy Henderson

'an alien landscape' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
an alien landscape

'an ugly creature made of insects' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
an ugly creature made of insects

'an ultrafine detailed painting of a sad person | ZBrush' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
an ultrafine detailed painting of a sad person | ZBrush

'Arnold Schwarzenegger | trending on ArtStation' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
Arnold Schwarzenegger | trending on ArtStation

'concept art of Robocop' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
concept art of Robocop

'dinosaurs' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
dinosaurs

'Dracula | CGSociety' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
Dracula | CGSociety

'flesh made of insects' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
flesh made of insects

'God by William Simpson' Multi-Perceptor CLIP Guided Diffusion Text-to-Image
God by William Simpson


Name: Pixel MultiColors
Author: Remi Durant
Original script: https://colab.research.google.com/drive/17c-13cl_VQKpHq2rDrnFVi6ZT-CHeZNn
Time for 512×512 on a 3090: 0 minutes 44 seconds
Maximum resolution on a 24 GB 3090: 4096×4096.
Maximum resolution on an 8GB 2080: 2048×2048 7 minutes 45 seconds
Description: Very noisy/pixelated/abstract results. The default script gives dark images which some tweaks to brightness and contrast can help. Maybe a little bit of blur could help too in a future revision. It is fast though, and can support huge image sizes.

'a charcoal drawing of a cute creature made of metal' Pixel MultiColors Text-to-Image
a charcoal drawing of a cute creature made of metal

'a farm' Pixel MultiColors Text-to-Image
a farm

'a forest path by Walter Leighton Clark' Pixel MultiColors Text-to-Image
a forest path by Walter Leighton Clark

'a lighthouse' Pixel MultiColors Text-to-Image
a lighthouse

'a surrealist painting of a beachside resort' Pixel MultiColors Text-to-Image
a surrealist painting of a beachside resort

'a well kept garden' Pixel MultiColors Text-to-Image
a well kept garden

'an abstract sculpture of Pikachu' Pixel MultiColors Text-to-Image
an abstract sculpture of Pikachu

'an art deco painting of a volcano' Pixel MultiColors Text-to-Image
an art deco painting of a volcano

'an ink drawing of tentacles' Pixel MultiColors Text-to-Image
an ink drawing of tentacles

'an octopus Rendered in Cinema4D' Pixel MultiColors Text-to-Image
an octopus Rendered in Cinema4D


Name: Ultraquick CLIP Guided Diffusion
Author: @sadly_existent
Original script: https://colab.research.google.com/github/sadnow/360Diffusion/blob/main/360Diffusion_AlphaTesting.ipynb
Time for 512×512 on a 3090: 1 minute 57 seconds
Maximum resolution on a 24 GB 3090: Locked to either 256×256 or 512×512.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another CLIP Guided Diffusion script. Can give some interesting results.

'a cave' Pixel MultiColors Text-to-Image
a cave

'a color pencil sketch of Cthulhu' Pixel MultiColors Text-to-Image
a color pencil sketch of Cthulhu

'a detailed painting of Shrek' Pixel MultiColors Text-to-Image
a detailed painting of Shrek

'a flemish baroque of the human condition by George Barret Jr' Pixel MultiColors Text-to-Image
a flemish baroque of the human condition by George Barret Jr

'a low poly render of halloween' Pixel MultiColors Text-to-Image
a low poly render of halloween

'a photorealistic painting of a worried woman made of paper by Ann Thetis Blacker' Pixel MultiColors Text-to-Image
a photorealistic painting of a worried woman made of paper by Ann Thetis Blacker

'a surrealist painting of a worried man' Pixel MultiColors Text-to-Image
a surrealist painting of a worried man

'a surrealist sculpture of an angry man 8K 3D' Pixel MultiColors Text-to-Image
a surrealist sculpture of an angry man 8K 3D

'Robocop' Pixel MultiColors Text-to-Image
Robocop

'zombies' Pixel MultiColors Text-to-Image
zombies


Name: ruDALL-E
Author: @sadly_existent
Original script: https://colab.research.google.com/drive/1wGE-046et27oHvNlBNPH07qrEQNE04PQ
Optimized script: https://colab.research.google.com/drive/1euIMG8E6kSFA2nU58LqrVsq6nbXjqELY
Time for 256×256 on a 3090: 1 minute 05 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256.
Maximum resolution on an 8GB 2080: Cannot run on 8GB VRAM
Description: Russian version of DALL-E. Only takes text prompts in Russian, so I do some auto English to Russian translations. Locked to small 256×256 images at this stage, but can create some interesting results.

'a hyperrealistic painting of Chewbacca by Edith Grace Wheatley' ruDALL-E Text-to-Image
a hyperrealistic painting of Chewbacca by Edith Grace Wheatley

'a low poly render of Pikachu' ruDALL-E Text-to-Image
a low poly render of Pikachu

'a man' ruDALL-E Text-to-Image
a man

'a rose' ruDALL-E Text-to-Image
a rose

'a stock photo of puppies' ruDALL-E Text-to-Image
a stock photo of puppies

'egyptian art of a portrait of a woman' ruDALL-E Text-to-Image
egyptian art of a portrait of a woman

'Harry Potter' ruDALL-E Text-to-Image
Harry Potter

'Indiana Jones' ruDALL-E Text-to-Image
Indiana Jones

'Robocop made of gold' ruDALL-E Text-to-Image
Robocop made of gold

'Yoda' ruDALL-E Text-to-Image
Yoda


Name: ruVQGAN+CLIP
Author: nev
Original script: https://colab.research.google.com/drive/1wAnIHocDYFAbWtA7rk8C7cFEUdRyLzwZ
Time for 512×512 on a 3090: 1 minute 28 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 27 seconds
Description: Creates fairly blurry results. Even with post process sharpening. If anyone could get these results crisper it would be really improve the output.

'a 3D render of a wizard by Gertrude Greene' ruVQGAN+CLIP Text-to-Image
a 3D render of a wizard by Gertrude Greene

'a cubist painting of a Pokemon character' ruVQGAN+CLIP Text-to-Image
a cubist painting of a Pokemon character

'a cute creature' ruVQGAN+CLIP Text-to-Image
a cute creature

'a matte painting of halloween by Carlos Trillo Name' ruVQGAN+CLIP Text-to-Image
a matte painting of halloween by Carlos Trillo Name

'a photorealistic painting of an alien landscape by Jacob Ochtervelt' ruVQGAN+CLIP Text-to-Image
a photorealistic painting of an alien landscape by Jacob Ochtervelt

'a rough seascape filmic' ruVQGAN+CLIP Text-to-Image
a rough seascape filmic

'a sea monster' ruVQGAN+CLIP Text-to-Image
a sea monster

'a woodcut of a skull by Gu Hongzhong trending on ArtStation' ruVQGAN+CLIP Text-to-Image
a woodcut of a skull by Gu Hongzhong trending on ArtStation

'Cthulhu' ruVQGAN+CLIP Text-to-Image
Cthulhu

'trypophobia' ruVQGAN+CLIP Text-to-Image
trypophobia


Name: Multi-Perceptor VQGAN+CLIP
Author: Remi Durant
Original script: https://colab.research.google.com/drive/1peZ98vBihDD9A1v7JdH5VvHDUuW5tcRK
Time for 512×512 on a 3090: 2 minute 30 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: As with the previous Multi-Perceptor CLIP Guided Diffusion scripts this one allows two different CLIP models to be used to rate the VQGAN output images. VQGAN is not going to beat diffusion for image coherance, but this script can give some very nice lighting and fine details in images.

'a bronze sculpture of an evil clown made of clay by Dionisio Baixeras Verdaguer' Multi-Perceptor VQGAN+CLIP Text-to-Image
a bronze sculpture of an evil clown made of clay by Dionisio Baixeras Verdaguer

'a fantasy land by Shigeru Aoki' Multi-Perceptor VQGAN+CLIP Text-to-Image
a fantasy land by Shigeru Aoki

'a hyperrealistic painting of puppies' Multi-Perceptor VQGAN+CLIP Text-to-Image
a hyperrealistic painting of puppies

'a midnineteenth century engraving of the Sydney Opera House' Multi-Perceptor VQGAN+CLIP Text-to-Image
a midnineteenth century engraving of the Sydney Opera House

'a statue of reflective spheres' Multi-Perceptor VQGAN+CLIP Text-to-Image
a statue of reflective spheres

'a surrealist painting of a tropical beach' Multi-Perceptor VQGAN+CLIP Text-to-Image
a surrealist painting of a tropical beach

'an alien city CGSociety' Multi-Perceptor VQGAN+CLIP Text-to-Image
an alien city CGSociety

'an oil painting of a fire breathing dragon' Multi-Perceptor VQGAN+CLIP Text-to-Image
an oil painting of a fire breathing dragon

'computer rendering of a well kept garden by Norman Garstin ZBrush' Multi-Perceptor VQGAN+CLIP Text-to-Image
computer rendering of a well kept garden by Norman Garstin ZBrush

'war CryEngine' Multi-Perceptor VQGAN+CLIP Text-to-Image
war CryEngine


Name: Hypertron
Author: Philipuss
Original script: https://colab.research.google.com/drive/10fa8X6EsfZfda1dfhJ_BtfPZ7Te1WGoX
Time for 512×512 on a 3090: 2 minute 00 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 35 seconds
Description: Another VQGAN based script. Has various “flavors” to give different results. Works OK. Can give the “image in a sea of purple/grey” that previous MSE based scripts suffered from. Still worth a try.

'a black and white photo of a fireman' Hypertron Text-to-Image
a black and white photo of a fireman

'a cute monster by Józef Mehoffer' Hypertron Text-to-Image
a cute monster by Józef Mehoffer

'a matte painting of a forest clearing' Hypertron Text-to-Image
a matte painting of a forest clearing

'a pop art painting of a human' Hypertron Text-to-Image
a pop art painting of a human

'a renaissance painting of a ghost by Jan van de Cappelle film' Hypertron Text-to-Image
a renaissance painting of a ghost by Jan van de Cappelle film

'a sea monster made of metal' Hypertron Text-to-Image
a sea monster made of metal

'a tattoo of a zombie' Hypertron Text-to-Image
a tattoo of a zombie

'a watercolor painting of a dragon Flickr' Hypertron Text-to-Image
a watercolor painting of a dragon Flickr

'an art deco painting of a haunted house by Mary Cameron' Hypertron Text-to-Image
an art deco painting of a haunted house by Mary Cameron

'concept art of a mountainscape by Maximilian Cercha' Hypertron Text-to-Image
concept art of a mountainscape by Maximilian Cercha


Name: CLIP Guided Diffusion Secondary Model Method
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1mpkrhOjoyzPeSWy2r7T8EYRaU7amYOOi
Time for 512×512 on a 3090: 2 minute 28 seconds
Maximum resolution on a 24 GB 3090: 1792×768 or 2048×640.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: A new diffusion based script from Katherine Crowson including a new “secondary model” she trained. Capable of some unique results with good textures and lighting.

'a detailed painting of Fozzy Bear by LeConte Stewart' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a detailed painting of Fozzy Bear by LeConte Stewart

'a flemish baroque of a happy person trending on pixiv' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a flemish baroque of a happy person trending on pixiv

'a flock of birds' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a flock of birds

'a Ghostbuster CGSociety' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a Ghostbuster CGSociety

'a kitchen made of cheese' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a kitchen made of cheese

'a nightmare creature' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a nightmare creature

'a photorealistic painting of The Grinch' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a photorealistic painting of The Grinch

'a portrait of a woman' CLIP Guided Diffusion Secondary Model Method Text-to-Image
a portrait of a woman

'an art deco painting of a sad clown' CLIP Guided Diffusion Secondary Model Method Text-to-Image
an art deco painting of a sad clown

'an oil painting of a nightmare' CLIP Guided Diffusion Secondary Model Method Text-to-Image
an oil painting of a nightmare



Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

24 responses to “Text-to-Image Summary – Part 4

    • Looks interesting but I cannot automate a download of the model from dropbox and it seems to need linux only components. I get other strange errors I cannot find a fix for at the moment.
      If they release a more simple to use version with an easily downloadable model I may be able to add a front end to it in Visions of Chaos, but for now, no go.

      • From what I know Pytti is behind a Patreon member’s only paywall at the moment. If I cannot share it with others then I am not going to spend time trying to get it working. If the author ever decides to freely share the script I will look at implementing a front end for it in Visions of Chaos, but in the meantime I am concentrating on the many other freely available and sharable scripts out there.

        Feed Forward VQGAN is on my to look at list. I briefly tried it before but got stuck with dependencies problems and circular references.

  1. Cataloging these scripts is incredibly valuable, thank you! I’m curious as to your methodology for finding all these scripts?

    • Scripts found by web searching, links online, people emailing me, reddit, etc.
      If there was a good way to reliably search all public colabs for certain text then I could easily find many more.

  2. I just updated to be able to run Multi-perceptor and the first results look promising. I’m running an automated batch now to see what comes up.
    I would really like to see an implementation of StyleCLIP (https://github.com/orpatashnik/StyleCLIP) but I don’t know if this version is Linux-only.
    Many thanks for your great work!

    • I had a quick look and I got the basic StyleCLIP “playground” colab notebook working locally. One issue is the model files. Because they are hosted on Google Drive I cannot easily automate the download. So I would need to redirect users to a web page that had instructions for “click here, then save it here” etc so they could manually update the required model files on their local PC.

      When I tried to get the “StyleCLIP_global” notebook running locally it needs the linux ninja build. StyleCLIP_global is the notebook needed to modify any image/photo so at this time it won’t work. Maybe if someone can get all this working locally on Windows then I can add it to Visions of Chaos, but not yet.

  3. I’ve been having a hard time getting the Multi-Perceptor Clip-Guided Diffusion script to generate outputs anywhere near as good as yours… my own system hit a VRAM bottleneck, so I’m running it directly from the linked Google Colab. What settings did you use to create those outputs? And did you tweak anything while implementing it into VoC?

    • Not every image is going to be a keeper. For my images I tend to run a batch process overnight with random prompts. Then the next day I pick the best 10 from the hundreds of results.
      But, having a quick look at the colab, the main changes I made are;
      timestep_respacing = ‘200’
      cutn_batches = 1
      Not using ddim seems to improve the results the most.
      You can always see my version of the script as I include it with Visions of Chaos under
      C:\Users\YourUserName\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\Text To Image\CLIP Guided Diffusion\multi_perceptor_clip_guided_diffusion.py

  4. The new ruDALL-E script looks amazing, but when I try to run it, I get a missing module error for ‘youtokentome’. When I execute the required pip-instruction I get an error saying this module requires Python 3.5 whereas I use 3.9.5 as per your installation instructions. Any idea how I can get this interesting script to run?

    • Try running these commands. That ensures all the latest libraries are installed and then checks PyTorch just in case.

      pip install –no-cache-dir –ignore-installed –upgrade –force-reinstall deep-translator==1.5.4
      pip install –no-cache-dir –ignore-installed –upgrade –force-reinstall cython==0.29.24
      pip install –no-cache-dir –ignore-installed –upgrade –force-reinstall youtokentome==1.0.6
      pip install –no-cache-dir –ignore-installed –upgrade –force-reinstall more-itertools==8.10.0
      pip uninstall -y torch
      pip uninstall -y torch
      pip install –no-cache-dir –ignore-installed –upgrade –force-reinstall torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
      pip list

      • Thanks for your swift reply! Using these commands, I got an incomprehensible error message about a non-existing option (-n or -m). I therefore did a complete reinstall of all packages and that did the trick. It is now running my standard list of test prompts, but what I’ve seen thus far, the results are disappointing. Even on simple instructions like ‘a dog’, I get imagery that’s nowhere near.
        I’m doing most of my testing on Multi-perceptor, which works great at just 100 iterations. This reduces runtime to 1m43s on my system. I’ve also reworked and added to your prompt files and I now get interesting imagery most of the time. If you like, I can mail the altered txt-files for you to try.
        Two final suggestions: the random seed option for running batches is useful, but would it be a lot of work to add a sequential option starting at seed X with increments of Y? Another useful feature would be the ability to use only a subset of styles, artists, etcetera using checkboxes before starting a random batch.
        Have a nice day!

  5. I’ve noticed an odd bug in Multi-perceptor that – in retrospect – must have been there for a while: it renders 6 iterations less than what’s asked for. I haven’t seen this behavior with other AI.

    • That was due to a hard coded skip iteration value. I have fixed it now for the next release so you get exactly the right number of iterations shown rather than 6 less than intended.

      • This bug was gone but in 88.1 it appears to be back. It’s not a big deal because I simply add 6. Just wanted to let you know.
        I also noticed I now have more control over random batches, which makes me a happy camper!
        When I experimented with this feature, I noticed I can’t seem to lock more than one line, though. After locking the subject, I also tried locking the medium, but that kept being randomized.

      • Yes, it is back to what it was.
        I had to revert to the old code, because automatically adding 6 to the iterations caused problems when the ddim checkbox was checked. Triggered an error deep within some other python code that I couldn’t fix.

    • Hi Mark,

      The “VQGAN+CLIP Animations” does only generate movies as you said (via checking the create frames from updates). This is because the zooming, panning and rotation is done within the script code rather than by Visions of Chaos between frames.

      For more complete control over frames etc you want to use the script functionality. See here for how the other movie options work.

      Regards,
      Jason.

    • Not really. That is just the way they are. I haven’t seen any VQGAN based script that does not suffer from coherence issues. You are better off using one of the newer Multi-Perceptor CLIP Guided Diffusion scripts (if you have a GPU with lots of VRAM) as they give superior results compared to VQGAN.

      Jason.

  6. all your work here is so fascinating Jason, thank you for Visions of Chaos.

    i was wondering if you would ever consider making your enhanced version of the Multi-Perceptor CLIP Guided Diffusion available on Colab for those of us without a high-end GPU.

    while i have had lots of enjoyment and artistic fulfillment with other Colab notebooks, the examples you posted show that your enhancements to that script somehow encourage CLIP to wrangle faces into the proper number of eyes, nose and mouth, and located in the right place! that’s something i haven’t been able to do with any other method. faces are apparently very difficult for AIs!

    • I never use colab beyond downloading scripts to edit locally. Anyone can merge my edits with a new colab though. My versions of all the Text-to-Image scripts ship with Visions of Chaos.

      Note that the example images in these blog posts are cherry picked best results from a lot of random prompts. No Text-to-Image system is going to give good images every time. If you want really good results you need to run multiple prompts overnight and then pick the best ones. Faces especially can take hundreds of outputs before you get that reasonable looking result. The basic rule for all Text-to-Image scripts is to have a lot of patience if you want the best results. My edits do not make the script suddenly able to make perfect faces every time.

      Jason.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s