Text-to-Image Summary – Part 7

This is Part 7. There is also Part 1, Part 2, Part 3, Part 4, Part 5, Part 6 and Part 8.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.

---

Name: Multi-Perceptor VQGAN+CLIP v4
Author: Remi Durant
Original script: https://colab.research.google.com/drive/1peZ98vBihDD9A1v7JdH5VvHDUuW5tcRK
Time for 512×512 on a 3090: 2 minutes 36 seconds
Maximum resolution on a 24 GB 3090: 1120×480
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Version 4 of Remi’s Multi-Perceptor VQGAN+CLIP script.

a bronze sculpture of a garden

a church by Tadeusz Kantor

a color pencil sketch of a monkey hyperdetailed

a comic book panel of a lush rainforest

a matte painting of a witch by William Geissler

a peninsula by Ei-Q CGSociety

a surrealist sculpture of hell

an eyeball made of flowers

cyberpunk art of a canyon

lineart of dense woodland

---

Name: V-Majesty Diffusion v1.2
Authors: Original script by Dango233 and multimodalart
Original script: https://colab.research.google.com/github/multimodalart/MajestyDiffusion/blob/main/v.ipynb
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: 1664×704.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: A new diffusion based script.

a black and white photo of war

a brownstone by Oliver Sin super detailed

a cute creature

a doctor

a drawing of a babbling brook photorealistic

a hill

a ninja by Michael Ford psychedelic

a photo of a beautiful young girl in a summer garden at dusk

a storybook illustration of a cozy den

New York City

---

Name: Latent Majesty Diffusion v1.3
Authors: Original script by Dango233 and multimodalart
Original script: https://colab.research.google.com/github/multimodalart/MajestyDiffusion/blob/main/latent.ipynb
Time for 512×512 on a 3090: 2 minutes 24 seconds
Maximum resolution on a 24 GB 3090: 512×512 (when using GFPGAN upscaling)
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Starts with a smaller resolution image (usually 256×256 pixels), upscales it with GFPGAN, and then does a few more diffusion passes. GFPGAN can really help get better coherency in faces.

a hyperrealistic painting of a cute creature

a hyperrealistic painting of an evil clown

a picture of a tree

a surrealist painting of kittens

an engraving of an angry woman made of voxels

an oil painting of an attractive woman by Eileen Aldridge

an ultrafine detailed painting of Bruce Willis 4K HD realism

Robert DeNiro ZBrush

Tweety Pie

Yoda

---

Name: Huemin JAX Diffusion v2.7
Author: Huemin
Original script: https://colab.research.google.com/github/huemin-art/jax-guided-diffusion/blob/v2.7/Huemin_Jax_Diffusion_2_7.ipynb
Time for 512×512 on a 3090: 3 minutes 55 seconds
Maximum resolution on a 24 GB 3090: 2496×1088
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Starts with a smaller resolution image (usually 256×256 pixels), upscales it with GFPGAN, and then does a few more diffusion passes. GFPGAN can really help get better coherency in faces.

a babbling brook

a mountain path Flickr

a river CGSociety

a spooky forest by John F. Peto

a storybook illustration of a mansion by Donald Roller Wilson

a surrealist sculpture of an alien forest

a watercolor painting of fear made of bones ZBrush

a wetland by Alexander Robertson super detailed

New York City

vector art of a bouquet of flowers ZBrush

---

Name: Disco Diffusion v5.2
Authors: Original script by @somnai, @gandamu and @zippy731
Original script: https://colab.research.google.com/github/zippy731/disco-diffusion-turbo/blob/skunk/Disco_Diffusion.ipynb
Time for 512×512 on a 3090: 2 minutes 2 seconds
Maximum resolution on a 24 GB 3090: 2496×1088
Maximum resolution on an 8GB 2080: 768×768
Description: Latest version of Disco Diffusion.

a bay by Louis Valtat Tri-X 400 TX

a hyperrealistic painting of a mansion

a hyperrealistic painting of a nightmare

a pond

a tributary by Jacob Marrel

an alien landscape

an island by Andrew Robertson IMAX

Frankenstein made of liquid metal CryEngine

medusa by Gai Qi

the Amazon Rainforest

---

Name: DALL-E Mini
Author: Original script by Boris Dayma
Original script: https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb
Time for 512×512 on a 3090: Locked to 256×256 – 1 minute 13 seconds
Maximum resolution on a 24 GB 3090: 256×256
Maximum resolution on an 8GB 2080: 256×256
Description: Capable of rendering multiple images in one pass. Very nice results. Limited to 256×256 at this time. These examples show a 4×4 grid of 16 images for each prompt.

a fine art painting of a fire breathing dragon

a hyperrealistic painting of an ugly monster

a planet

a river

a rose

a surrealist painting of a king

an airbrush painting of satan

an engraving of a frog

an ultrafine detailed painting of fear

Darth Vader trending on pixiv

---

Name: Latent Majesty Diffusion v1.6
Authors: Original script by Dango233 and multimodalart
Original script: https://colab.research.google.com/github/multimodalart/MajestyDiffusion/blob/main/latent.ipynb
Time for 512×512 on a 3090: 2 minutes 07 seconds
Maximum resolution on a 24 GB 3090: 512×512 (when using GFPGAN upscaling)
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: The latest amazing update to Latent Diffusion. Awesome colors, textures, lighting, details, coherency. Highly recommended.

a colorful parrot

a detailed matte painting of puppies

a gallery

a mountain cabin by Tom Palin 4K HD realism

a renaissance painting of a spooky forest

a school of tropical fish by Jane Carpanini

an ugly creature

the Amazon Rainforest photorealistic

The Grinch

Yoda trending on Flickr

---

Name: Disco Diffusion v5.4
Authors: Original script by @somnai, @gandamu, @zippy731 and @devdef
Original script: https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb
Time for 512×512 on a 3090: 2 minutes 20 seconds
Maximum resolution on a 24 GB 3090: 2496×1088
Maximum resolution on an 8GB 2080: 768×768
Description: Latest version of Disco Diffusion.

a 3D render of the Grand Canyon by Dóra Keresztes

a hyperrealistic painting of a zombie made of cheese and feathers CryEngine and rendered in Cinema4D

a macro photograph of an ugly creature

a matte painting of a monument

a picture of a vast city

a skyscraper

a thunder storm

Jason Vorhees by Chen Chi

reflective spheres hyperrealistic

the country by Robert Thomas and Chen Jiru rendered in unreal engine and 4K photo

---

Name: Pixel Art Diffusion v3
Authors: Original script by @somnai, @gandamu, @zippy731 and @KaliYuga_ai
Original script: https://colab.research.google.com/github/KaliYuga-ai/Pixel-Art-Diffusion/blob/main/Pixel_Art_Diffusion_v3_0_(With_Disco_Symmetry).ipynb
Time for 512×512 on a 3090: 3 minutes 14 seconds
Maximum resolution on a 24 GB 3090: 1920×1088
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Uses a fine tuned model to generate pixel art like imagery.

a bedroom hyperrealistic and hyperdetailed #pixelart

a cephalopod IMAX and lens flare #pixelart

a cinematic painting of a cottage #pixelart

a cross stitch of a townhouse made of voxels and timber #pixelart

a pastel of a cute girl #pixelart

a surrealist sculpture of Charmander #pixelart

an alien city by Anders Zorn and Laura Muntz Lyall photorealistic and CGSociety #pixelart

concept art of a wetland ZBrush and CryEngine #pixelart

Frankenstein made of string and vines trending on pixiv and hyperrealistic #pixelart

pixel art of a cloudy sunset #pixelart

---

Name: Disco Diffusion v5.6
Authors: Original script by @somnai and @gandamu
Original script: https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb
Time for 512×512 on a 3090: 3 minutes 34 seconds
Maximum resolution on a 24 GB 3090: 2496×1088
Maximum resolution on an 8GB 2080: 768×768
Description: Latest version of Disco Diffusion.

a collage painting of a lush rainforest by Doc Hammer and Alexander Ivanov hyperrealistic and CryEngine

a cubist painting of a lion and a sunset CryEngine and trending on pixiv

a fine art painting of a zombie

a gulf by I Ketut Soki and Alfons von Czibulka

a monastery trending on Flickr and #film

a morning landscape

a prairie CGSociety and CryEngine

a werewolf

ballpoint pen art of a monument

cyberpunk art of heaven filmic and CryEngine

---

Name: CLIP Guided k-diffusion
Author: Original script by Katherine Crowson
Original script: https://colab.research.google.com/drive/1w0HQqxOKCk37orHATPxV8qb0wb4v-qa0
Time for 512×512 on a 3090: 6 minutes 56 seconds
Maximum resolution on a 24 GB 3090: Fixed to 512×512 resolution.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: A new script by Katherine. Seems to generate more abstract results and these example images needed a long run of random prompts to select from.

a jigsaw puzzle of paranoia by Petr Brandl and Sasha Putrya

a landscape vivid colors

a pastel of Cookie Monster by Ren Bonian and Ángel Botello for sale on Facebook Marketplace and CryEngine

a reef

a renaissance painting of Al Pacino

a statue of a submarine made of metal and crystals by James Sessions American painter and Elfriede Lohse-Wächtler

an airbrush painting of a nightmare creature vivid colors and rendered in Cinema4D

an oil painting of a cephalopod made of paper and mist

an ugly person and an area 4K HD realism and trending on pixiv

conceptual art of an ugly monster

---

Name: CLIP Prior + VQGAN (MSE method)
Author: Original script by Katherine Crowson
Original script: https://colab.research.google.com/drive/1yOpCY9eXvzELHppvh-o0DevhxVYOGr5i
Time for 512×512 on a 3090: 3 minutes 31 seconds
Maximum resolution on a 24 GB 3090: 832×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: A new script by Katherine. Can give some interesting details but coherence may suffer at larger resolutions.

a collage painting of a tiger vivid colors and photorealistic

a cove 4K photo and CryEngine

a cute creature

a glacier

a space nebula

a townhouse

a valley

an oil painting of a peacock by Wu Hong and Eve Ryder

Cthulhu

digital art of a wetland made of cheese and timber by Jacob Duck and Jacob Gerritsz Cuyp

---

Name: Latent Diffusion LAION_400M v2
Author: Original script by pesser
Original script: https://github.com/pesser/stable-diffusion
Time for 16 256×256 images on a 3090: 49 seconds
Maximum resolution on a 24 GB 3090: 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Renders multiple images quickly. Coherency is best at 256×256 so these example images are 2×2 tiled results. Each took 35 seconds on a 3090.

a babbling brook

a colorful parrot

a fine art painting of a castle

a matte painting of a rose

a pencil sketch of a cave 4K photo and hyperrealistic

a photorealistic painting of Cthulhu for sale on Facebook Marketplace and Flickr

a surrealist painting of a cloudy sunset

a surrealist painting of a monkey

an illustration of of a tiger by Stanley Twardowicz and Antoni Pitxot

an impressionist painting of a cottage

---

Name: Stable Diffusion
Author: Original script by pesser
Original script: https://github.com/CompVis/stable-diffusion
Time for 512×512 on a 3090: 34 seconds
Maximum resolution on a 24 GB 3090: 1280×640
Maximum resolution on an 8GB 2080: 640×576
Description: Incredible. Latest and greatest. Beats all previous Text-to-Image systems. If you only use one, use this one.

a black and white photo of puppies

a cathedral rendered in unreal engine and super detailed

a city made of mist trending on ArtStation and trending on Flickr

a detailed matte painting of a lush rainforest made of crystals and feathers

a king

a polaroid photo of a clown vivid colors and 8K 3D

an airbrush painting of the Terminator CryEngine and for sale on Facebook Marketplace

an ambient occlusion render of a wetland by William Forsyth and Victorine Foot trending on pixiv and CryEngine

poster art of a farm by Frederic Leighton and Yang Borun rendered in unreal engine and 8K 3D

the Australian outback

---

Name: Deforum Stable Diffusion v0.3
Author: Original script by Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer
Original script: https://colab.research.google.com/github/deforum/stable-diffusion/blob/main/Deforum_Stable_Diffusion.ipynb
Time for 512×512 on a 3090: 34 seconds
Maximum resolution on a 24 GB 3090: 1280×640
Maximum resolution on an 8GB 2080: 640×576
Description: Incredible. Latest and greatest. Beats all previous Text-to-Image systems. If you only use one, use this one. Deforum builds upon Stable Diffusion with animation support.

a babbling brook

a forest path

a photo of a lake

a ranch by Nikolai Alekseyevich Kasatkin and Harriet Zeitlin

a watercolor painting of a rectory hyperdetailed and trending on ArtStation

Al Pacino by Lam Qua and George Frederick Harris

an angry person by Kazys Varnelis and Dóra Keresztes

an astronaut

puppies

war

giger xenomorphs, airbrush, HD, 4K, 8K, hyperrealistic, highly detailed, highly textured

---

Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

Text-to-Image Summary – Part 4

This is Part 4. There is also Part 1, Part 2, Part 3, Part 5, Part 6, Part 7 and Part 8.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.

---

Name: PixelDraw
Author: dribnet
Original script: https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/PixelDrawer.ipynb
Time for 512×512 on a 3090: 1 minutes 59 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Generates “pixel art” images. I had a lot of requests to add support for this one.

a cartoon of a peacock

a cloudy sunset

a gorilla

a morning landscape

a watercolor painting of a castle

an art deco painting of Al Pacino

Hell

Shrek

---

Name: DirectVisions
Author: Jens Goldberg
Original script: https://colab.research.google.com/drive/127lKSsQjx-UDDUSvIkLL6mREfZ0KQu5D
Time for 512×512 on a 3090: 2 minutes 39 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 4096×4096
Description: Interesting detailed images. Can create huge resolution results.

a color pencil sketch of a western town

a detailed painting of a cephalopod

a digital rendering of an ugly face

a pencil sketch of Buzz Lightyear

a rough seascape by Pinchus Kremegne

a stock photo of a president

a sunset

an alien city

an alien forest by Helen Berman

an evening landscape

---

Name: Pixel Direct
Author: Unknown
Original script: https://colab.research.google.com/drive/1F9ZOZnpV3uBPRDSESaAXYwzNZJQRJT75
Time for 512×512 on a 3090: 1 minutes 03 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 2048×2048 1 minute 51 seconds
Description: Another “Pixel Art” script. More abstract results than the PixelDraw script above.

a bronze sculpture of a nightmare creature

a cartoon of Al Pacino

a nightclub

a silk screen of a bouquet of flowers

an etching of a worried woman

an illustration of of a thunder storm

---

Name: FourierVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1nGNBjhbYnDHSumGPjpFHjDOsaZFAqGgF
Time for 512×512 on a 3090: 1 minutes 40 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 1024×1024 4 minutes 07 seconds
Description: Detailed images. The default script generates washed out pastel images, but with some gamma and brightness tweaks they can be improved (still not ideal, but better). Allows very large resolution images.

a cathedral

a charcoal drawing of zombies

a detailed painting of a sunset by Thomas Cantrell Dugdale

a ghost made of mist

a kitchen

a movie monster

a pencil sketch of a sad clown

a werewolf

an evil clown by Viktor Oliva

an ink drawing of an ugly monster

---

Name: PyramidVisions
Author: Unknown
Original script: https://colab.research.google.com/drive/1dpAS_wK34y7c6s-CatAFmBtbkjGT_erM
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 1024×1024 10 minutes 48 seconds
Description: Very detailed images. Not the fastest script, but gives some very nice results. Lower VRAM requirements so good for lesser spec GPUs. Definitely one of the better scripts worth exploring.

a desert oasis

a lush rainforest

a marble sculpture of an angry person

a minimalist painting of the Amazon Rainforest

a nightmare creature

a pastel of a computer made of paper

an abstract sculpture of a sad clown

an acrylic painting of an alien forest | vivid colors

Medusa

vector art of an ugly woman

---

Name: Visions of AI v1
Author: Jason Rampe
Original script: Included with Visions of Chaos. No colab.
Time for 512×512 on a 3090: 1 minutes 32 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 33 seconds
Description: My first attempt at actually creating a Text-to-Image script. Based on the excellent example from Jonathan Whitaker‘s AIAIArt Lesson 3 tutorial. Gives some very nice fine detail in some areas, but suffers the non coherance of other scripts in that it creates multiple copies of the subject throughout the image. After actually trying to write my own script I only have more respect for those who can do this. Hopefully I can improve these results for a version 2. In the meantime, here are some sample from the current Visions of AI script.

a cartoon of the human condition by Judy Takács

a cubist painting of an evening landscape

a digital rendering of frogs

a fire breathing dragon

a hyperrealistic painting of a movie monster

a morning landscape

a shark

a woodcut of an ugly man

an airbrush painting of C-3PO

Frankenstein

---

Name: Visions of AI v2
Author: Jason Rampe
Original script: Included with Visions of Chaos. No colab.
Time for 512×512 on a 3090: 2 minutes 35 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480.
Maximum resolution on an 8GB 2080: 256×256 2 minutes 36 seconds
Description: An attempt to improve the coherency of the previous script. The first 30 iterations zoom into the image every 10 frames. This results in larger shapes/blobs for the rest of the script to work from. The idea is that it will give larger subjects compared to the v1 script. Kind of works. Gives blurrier results. To be fixed in the next version?

a morning landscape by William Gear

a raytraced image of a nightclub lens flare

a tentacle monster by Carlo Crivelli

a woodcut of a worried woman by Li Keran

an illustration of of a cave made of cheese

Cthulhu

cyberpunk art of a futuristic city

goldfish

reflective spheres

the Australian outback

---

Name: Multi-Perceptor CLIP Guided Diffusion
Author: Varkarrus
Original script: https://colab.research.google.com/drive/1y3Vt39A5KSNFRa6Z2bCqDHxteZSVH9NC
Time for 512×512 on a 3090: 3 minutes 08 seconds
Maximum resolution on a 24 GB 3090: 896×512 or 1152×384 (dimensions must be divisible by 128).
Maximum resolution on an 8GB 2080: 128×128 1 minute 56 seconds
Description: Builds upon previous CLIP Guided Diffusion scripts. Like the previous script by Dango233 it uses three CLIP models simultaneously to “rate” the generated images, and I have added options to use up to six different CLIP models. The resulting image accuracy compared to the prompt, and the resulting image coherence seem to be much better than previous CLIP Guided Diffusion scripts that could almost have random outputs sometimes. This script is superb and highly recommended. Great lighting, textures and brushstrokes. Normally with these blog posts I do a batch run of random prompts overnight and then pick the best 10 images. In this case I had nearly 50 images in my “good” folder after going through the batch results. So, for this script I am showing 20 sample images.

a cute creature | TriX 400 TX

a digital painting of Frankenstein by Kanzan Shimomura

a morning landscape by János SaxonSzász

a nightmare creature

a photorealistic painting of a teddy bear

a portrait of a young girl

a space nebula | IMAX

a worried man

a zombie by Nathaniel Hone

an acrylic painting of a spider by Abram Arkhipov

an airbrush painting of a monkey by Jeremy Henderson

an alien landscape

an ugly creature made of insects

an ultrafine detailed painting of a sad person | ZBrush

Arnold Schwarzenegger | trending on ArtStation

concept art of Robocop

dinosaurs

Dracula | CGSociety

flesh made of insects

God by William Simpson

---

Name: Pixel MultiColors
Author: Remi Durant
Original script: https://colab.research.google.com/drive/17c-13cl_VQKpHq2rDrnFVi6ZT-CHeZNn
Time for 512×512 on a 3090: 0 minutes 44 seconds
Maximum resolution on a 24 GB 3090: 4096×4096.
Maximum resolution on an 8GB 2080: 2048×2048 7 minutes 45 seconds
Description: Very noisy/pixelated/abstract results. The default script gives dark images which some tweaks to brightness and contrast can help. Maybe a little bit of blur could help too in a future revision. It is fast though, and can support huge image sizes.

a charcoal drawing of a cute creature made of metal

a farm

a forest path by Walter Leighton Clark

a lighthouse

a surrealist painting of a beachside resort

a well kept garden

an abstract sculpture of Pikachu

an art deco painting of a volcano

an ink drawing of tentacles

an octopus Rendered in Cinema4D

---

Name: Ultraquick CLIP Guided Diffusion
Author: @sadly_existent
Original script: https://colab.research.google.com/github/sadnow/360Diffusion/blob/main/360Diffusion_AlphaTesting.ipynb
Time for 512×512 on a 3090: 1 minute 57 seconds
Maximum resolution on a 24 GB 3090: Locked to either 256×256 or 512×512.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another CLIP Guided Diffusion script. Can give some interesting results.

a cave

a color pencil sketch of Cthulhu

a detailed painting of Shrek

a flemish baroque of the human condition by George Barret Jr

a low poly render of halloween

a photorealistic painting of a worried woman made of paper by Ann Thetis Blacker

a surrealist painting of a worried man

a surrealist sculpture of an angry man 8K 3D

Robocop

zombies

---

Name: ruDALL-E
Author: @sadly_existent
Original script: https://colab.research.google.com/drive/1wGE-046et27oHvNlBNPH07qrEQNE04PQ
Optimized script: https://colab.research.google.com/drive/1euIMG8E6kSFA2nU58LqrVsq6nbXjqELY
Time for 256×256 on a 3090: 1 minute 05 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256.
Maximum resolution on an 8GB 2080: Cannot run on 8GB VRAM
Description: Russian version of DALL-E. Only takes text prompts in Russian, so I do some auto English to Russian translations. Locked to small 256×256 images at this stage, but can create some interesting results.

a hyperrealistic painting of Chewbacca by Edith Grace Wheatley

a low poly render of Pikachu

a man

a rose

a stock photo of puppies

egyptian art of a portrait of a woman

Harry Potter

Indiana Jones

Robocop made of gold

Yoda

---

Name: ruVQGAN+CLIP
Author: nev
Original script: https://colab.research.google.com/drive/1wAnIHocDYFAbWtA7rk8C7cFEUdRyLzwZ
Time for 512×512 on a 3090: 1 minute 28 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 27 seconds
Description: Creates fairly blurry results. Even with post process sharpening. If anyone could get these results crisper it would be really improve the output.

a 3D render of a wizard by Gertrude Greene

a cubist painting of a Pokemon character

a cute creature

a matte painting of halloween by Carlos Trillo Name

a photorealistic painting of an alien landscape by Jacob Ochtervelt

a rough seascape filmic

a sea monster

a woodcut of a skull by Gu Hongzhong trending on ArtStation

Cthulhu

trypophobia

---

Name: Multi-Perceptor VQGAN+CLIP
Author: Remi Durant
Original script: https://colab.research.google.com/drive/1peZ98vBihDD9A1v7JdH5VvHDUuW5tcRK
Time for 512×512 on a 3090: 2 minute 30 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: As with the previous Multi-Perceptor CLIP Guided Diffusion scripts this one allows two different CLIP models to be used to rate the VQGAN output images. VQGAN is not going to beat diffusion for image coherance, but this script can give some very nice lighting and fine details in images.

a bronze sculpture of an evil clown made of clay by Dionisio Baixeras Verdaguer

a fantasy land by Shigeru Aoki

a hyperrealistic painting of puppies

a midnineteenth century engraving of the Sydney Opera House

a statue of reflective spheres

a surrealist painting of a tropical beach

an alien city CGSociety

an oil painting of a fire breathing dragon

computer rendering of a well kept garden by Norman Garstin ZBrush

war CryEngine

---

Name: Hypertron
Author: Philipuss
Original script: https://colab.research.google.com/drive/10fa8X6EsfZfda1dfhJ_BtfPZ7Te1WGoX
Time for 512×512 on a 3090: 2 minute 00 seconds
Maximum resolution on a 24 GB 3090: 1120×480.
Maximum resolution on an 8GB 2080: 256×256 1 minute 35 seconds
Description: Another VQGAN based script. Has various “flavors” to give different results. Works OK. Can give the “image in a sea of purple/grey” that previous MSE based scripts suffered from. Still worth a try.

a black and white photo of a fireman

a cute monster by Józef Mehoffer

a matte painting of a forest clearing

a pop art painting of a human

a renaissance painting of a ghost by Jan van de Cappelle film

a sea monster made of metal

a tattoo of a zombie

a watercolor painting of a dragon Flickr

an art deco painting of a haunted house by Mary Cameron

concept art of a mountainscape by Maximilian Cercha

---

Name: CLIP Guided Diffusion Secondary Model Method
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1mpkrhOjoyzPeSWy2r7T8EYRaU7amYOOi
Time for 512×512 on a 3090: 2 minute 28 seconds
Maximum resolution on a 24 GB 3090: 1792×768 or 2048×640.
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: A new diffusion based script from Katherine Crowson including a new “secondary model” she trained. Capable of some unique results with good textures and lighting.

a detailed painting of Fozzy Bear by LeConte Stewart

a flemish baroque of a happy person trending on pixiv

a flock of birds

a Ghostbuster CGSociety

a kitchen made of cheese

a nightmare creature

a photorealistic painting of The Grinch

a portrait of a woman

an art deco painting of a sad clown

an oil painting of a nightmare

---

Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

Text-to-Image Summary – Part 3

This is Part 3. There is also Part 1, Part 2, Part 4, Part 5, Part 6, Part 7 and Part 8.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.

---

Name: CLIP Guided Diffusion v4
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1V66mUeJbXrTuQITvJunvnWVn96FEbSI3
Time for 512×512 on a 3090: 3 minutes 05 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another CLIP Guided Diffusion script. Locked to 512×512 resolution. Like the other CLIP Diffusion scripts, some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. When it gets a “hit” it can create very detailed impressive results, but the amount of “misses” stops it from getting a great rating. Still worth a try if you have the patience to run a large batch of images waiting for the best results. The following samples came hand picked from a large batch run of random prompt phrases.

a forest clearing

a storybook illustration of a nightmare

an impressionist painting of a cemetery

Harry Potter in the style of Rembrandt

a detailed painting of a witch

a babbling brook

a desert oasis

a hyperrealistic painting of an android

eyeballs

a cross stitch of Buzz Lightyear

---

Name: CLIP Guided Decision Transformer
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1V66mUeJbXrTuQITvJunvnWVn96FEbSI3
Time for 512×512 on a 3090: 1 minutes 13 seconds
Maximum resolution on a 24 GB 3090: Locked to 384×384
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another one from Katherine Crowson. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. When it gets a “hit” it can create very detailed impressive results, but the amount of “misses” stops it from getting a great rating. The following samples came hand picked from a large batch run of random prompt phrases.
Another good point for CLIP Decsision Transformer is that it will generate a batch of images from each run. So rather than a single image for the prompt text you can specify (for example) 8 images to be generated from the prompt. This allows a much larger set of images to be quickly generated to find those great outputs in.
For these images I have enhanced the resolution 4x using Real-ESRGAN (the thumnails are the original output images and the clicked images are resized x4).

a detailed painting of a palace by Thomas Kinkade

a drawing of Chewbacca

a forest path

a renaissance painting of a mountain range

a rough seascape

a rough seascape

a spooky forest

an oil on canvas painting of a western town

Frankenstein

The Grand Canyon

---

Name: CLIPIT
Author: dribnet
Original script: https://github.com/dribnet/clipit
Time for 512×512 on a 3090: 2 minutes 38 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another GAN+CLIP script. Gives nice results that tend to match the prompt text more closely. This one is heavy on VAM usage.

a happy family by Piet Mondiran

a landscape

a peacock

a tropical beach by Thomas Kinkade

a woodcut of Dracula

an ambient occlusion render of a zombie

eyeballs in the style of Claude Monet

---

Name: Art Machine
Author: Hillel Wayne
Original script: https://colab.research.google.com/drive/1n_xrgKDlGQcCF6O-eL3NOd_x4NSqAUjK
Time for 512×512 on a 3090: 4 minutes 04 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 1 minute 50 seconds
Description: Another VQGAN+CLIP scipt.

a charcoal drawing of a kitchen

a mosaic of a mountain path | CryEngine

a silk screen of a tropical beach in the style of Kandinsky

a woodcut of a nightmare creature

an illustration of of a mountainscape

an ultrafine detailed painting of a green tree frog as created by Craig Mullins

Dracula

Planets

---

Name: VQGAN+CLIP v5
Author: Max Woolf
Original script: https://colab.research.google.com/drive/1wkF67ThUz37T2_oPIuSwuO4e_-0vjaLs
Time for 512×512 on a 3090: 2 minutes 13 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 2 minutes 02 seconds
Description: Another VQGAN+CLIP scipt. More abstract results from this one.

a desert oasis in the style of Salvador Dali

a hyperrealistic painting of a dragon

Big Bird

Cthulhu

Robert DeNiro

Yoda “hmmm, abstract I am”

---

Name: Zoetrope 5.5
Author: Bearsharktopusdev
Original script: https://colab.research.google.com/drive/1LpEbICv1mmta7Qqic1IcRTsRsq7UKRHM
Time for 512×512 on a 3090: 3 minutes 27 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: 256×256 3 minutes 23 seconds
Description: Updated version of Zoetrope 5. Supports more VQGAN models, CLIP models and optimizers compared to Zoetrope 5.

a cephalopod

a flemish baroque of a demon

a photo of a submarine in the style of Vincent van Gogh

a snail

Cthulhu

flesh

---

Name: Zeta Quantize
Author: afiaka87
Original script: https://colab.research.google.com/gist/afiaka87/a97cca3b54c02209b94ff805224f9eb5/zeta_quantize.ipynb
Time for 512×512 on a 3090: 4 minutes 18 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: 256×256 5 minutes 01 seconds
Description: Another VQGAN+CLIP scipt.

a cute creature made of silver

a detailed painting of a cephalopod

a detailed painting of a ghost

a forest fire made of copper

a peacock

a sketch of a Pokemon character in the style of Odilon Redon

a watercolor painting of dense woodland

---

Name: Experimental VQGAN
Author: Various
Original script: https://colab.research.google.com/drive/1jx3klUxlGbYUwvtqzC9SYl4XZKHL3R81
Time for 512×512 on a 3090: 1 minutes 12 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: 256×256 0 minutes 52 seconds
Description: Very nice smooth results from this one.

a desert oasis in the style of Craig Mullins

a dragon

a manga drawing of a happy alien

a nightmare

a surrealist painting of love

a watercolor painting of a lighthouse

an airbrush painting of a well kept garden by Piet Mondiran

Cookie Monster

---

Name: SlideShowVisions
Author: Active Galaxy
Original script: https://colab.research.google.com/drive/1IihC4ZJvCh_tOgBVd900BzHX-ulPEFsa
Time for 512×512 on a 3090: 2 minutes 25 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: 128×128 1 minute 56 seconds
Description: Tends to give more abstract paper cutout looks.

a happy child

a house vivid colors

a sea monster

a thunder storm

a tree

a woodcut of war

an engraving of zombies

Han Solo

---

Name: Quick CLIP Guided Diffusion
Author: Daniel Russell
Original script: https://colab.research.google.com/drive/1FuOobQOmDJuG7rGsMWfQa883A9r4HxEO
Time for 512×512 on a 3090: 43 seconds
Maximum resolution on a 24 GB 3090: 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Modified version of CLIP Guided Diffusion that gets results quicker. Option for 256×256 or 512×512 sized images. Still very hit and miss when getting images that resemble the input prompt. The following samples came from a large overnight batch run of random prompts.

a cathedral

a digital painting of a space nebula

a lounge room

a monkey | lens flare

a nightmare creature

a rough seascape

a landscape

an android

an attractive woman

an oil on canvas painting of a cloudy sunset

---

Name: CLIP Guided Diffusion v5
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/1QBsaDAZv8np29FPbvjffbE1eytoJcsgA
Time for 512×512 on a 3090: 3 minutes 48 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Another CLIP Guided Diffusion script. Locked to 512×512 resolution. Needs less VRAM than the previous versions. The following samples came hand picked from a large batch run of random prompt phrases.

a cityscape

a gorilla

Cthulhu by Craig Mullins

computer rendering of Emporer Palpatine made of cheese by Evan Charlton

digital art of a mountainscape as created by Persis Goodale Thurston Taylor

a digital rendering of Chewbacca

an ugly person

See this tweet for an example of using CLIP Guided Diffusion to stylize a portrait.

---

Name: MSE Regulized Modified
Author: jbusted
Original script: https://colab.research.google.com/drive/1gFn9u3oPOgsNzJWEFmdK-N9h_y65b8fj
Time for 512×512 on a 3090: 3 minutes 02 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: 256×256 2 minutes 45 seconds
Description: Modified and updated version of the previous “MSE Regulized VQGAN+CLIP” script. Less likely to suffer the previous script’s issue of subjects floating in a purple void.

a bronze sculpture of a planet

a cave by Asher Brown Durand

a charcoal drawing of Emporer Palpatine

a cozy den

a detailed drawing of a heart made of string by William MacTaggart

a digital rendering of Arnold Schwarzenegger made of metal by Muriel Brandt

a lounge room

a palace by Jules Joseph Lefebvre

an oil on canvas painting of a lush rainforest

an oil on canvas painting of Cookie Monster

---

Name: Pixray
Author: dribnet
Original script: https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/Start_Here.ipynb
Time for 512×512 on a 3090: 1 minutes 44 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×720
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Updated version of the previous “CLIPIT” script.

a bronze sculpture of a nightmare creature

a fire breathing dragon by Jan Baptist Weenix

a morning landscape

a surrealist sculpture of an elephant

a watercolor painting of an astronaut

an oil painting of a worried woman | Rendered in Cinema4D

an ugly creature

Dracula

Frankenstein

vector art of a forest clearing

---

Name: CLIP Guided Diffusion v6
Author: Dango233
Original script: https://colab.research.google.com/drive/14xBm1aSxQLbq26-jmDJi8I1HJ4ti5ybt
Time for 512×512 on a 3090: 3 minutes 10 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Latest CLIP Guided Diffusion script. The best one yet. Capable of some very nice results.

a hyperrealistic painting of a human

a sketch of planets

a storybook illustration of a cloudy sunset

a wizard | vivid colors

an art deco sculpture of a planet

an attractive man by John Linnell

an oil on canvas painting of satan

an oil painting of a clown

digital art of an ugly person by Avigdor Arikha

princess in sanctuary trending on artstation photorealistic portrait of a young princess

---

Name: CLIPDraw
Author: Kevin Frans
Original script: https://colab.research.google.com/github/kvfrans/clipdraw/blob/main/clipdraw.ipynb
Time for 512×512 on a 3090: 7 minutes 10 seconds
Maximum resolution on a 24 GB 3090: Huge. 4096×4096 and beyond.
Maximum resolution on an 8GB 2080: 1024×1024
Description: Generates images by a series of lines. Very abstract results.

a cloudy sunset

a digital painting of a rose

a sad clown

an abstract painting of Yoda

an etching of a library

The Sydney Harbour Bridge

---

Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

Text-to-Image Summary – Part 2

This is Part 2. There is also Part 1, Part 3, Part 4, Part 5, Part 6, Part 7 and Part 8.

This post continues listing the Text-to-Image scripts included with Visions of Chaos and some example outputs from each script.

---

Name: VQGAN Gumbel
Author: Eleiber
Original script: https://colab.research.google.com/drive/1tim3xTsZXafK-A2rOUsevckdl4OitIiw
Time for 512×512 on a 3090: 3 minutes 27 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 4 minutes 05 seconds
Description: Variation using the gumbel-8192 model. Results are a bit rougher than others.

a childs drawing of a space nebula

a movie monster in the style of Edvard Munch

a raytraced image of the Amazon Rainforest

a tropical beach in the style of Polock

digital art of a rose

---

Name: OpenAI DVAE+CLIP
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/10DzGECHlEnL4oeqsN-FWCkIe_sq3wVqt
Time for 512×512 on a 3090: 3 minutes 07 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 2 minutes 20 seconds
Description: Results are very colorful and more abstract. By default it gives more noisy output images but this can be disabled if you prefer.

a dragon

a hyperrealistic painting of planets

a mountain cabin

a woodcut of a mountain range in the style of Marvel Comics

an angry person

---

Name: Aphantasia
Author: Vadim Epstein
Original script: https://github.com/eps696/aphantasia
Time for 512×512 on a 3090: 1 minute 5 seconds
Maximum resolution on a 24 GB 3090: 4096×4096 or 2520×1080
Maximum resolution on an 8GB 2080: 4096×4096 7 minutes 48 seconds
Description: Different and more messy pastel abstract Turneresque output. I spent a few hours trying many different combinations of settings trying to get the output more coherent and deeper colors. The following samples are as good as I could push it. I give up for now. If you can do better let me know. It does support creating larger 1280×720 resolution images on a 3090 GPU.

a marble sculpture of a computer

an eyeball

an octopus

digital art of frogs in the style of Dr Seuss

medusa

---

Name: Text2Image VQGAN
Author: Vadim Epstein
Original script: https://colab.research.google.com/github/eps696/aphantasia/blob/master/CLIP_VQGAN.ipynb
Time for 512×512 on a 3090: 2 minutes 8 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 2 minutes 15 seconds
Description: Allows larger sized 480p images (854×480) on a 3090 GPU.

a digital painting of the Las Vegas strip

a midnineteenth century engraving of a cute monster

a skeleton

an ultrafine detailed painting of a crying person

puppies

---

Name: MSE VQGAN+CLIP z+quantize
Author: jbusted
Original script: https://colab.research.google.com/drive/1gFn9u3oPOgsNzJWEFmdK-N9h_y65b8fj
Time for 512×512 on a 3090: 6 minutes 19 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 3 minutes 36 seconds
Description: Awesome crisp results. Allows larger sized 480p images (854×480) on a 3090 GPU. One of the best scripts in this list worth exploring.

a charcoal drawing of a country town

a hyperrealistic painting of an ugly creature

a landscape made of mist

a mosaic of christmas

an octopus in the style of Vincent van Gogh

MSE VQGAN+CLIP z+quantize allows specifying an image as the input starting point. If you take the output and repeatedly use it as the input with some minor image stretching each frame you can get a movie zooming into the Text-to-Image output. No blending of frames or optical flow for this one, just straight combining of the 854×480 resolution frames into a movie. The VQGAN model was “vqgan_imagenet_f16_16384” and the CLIP model was “ViT-B/32”. The prompts for this movie were “hyperrealistic homer simpson”, “hyperrealistic marge simpson”, “hyperrealistic bart simpson”, “hyperrealistic lisa simpson” and “hyperrealistic maggie simpson”. The original 480p upload was badly compressed and looked terrible after YouTube compressed it, so I upscaled the 480p to 2160p (4K) in DaVinci Resolve and reuploaded to YouTube. This caused their compression to do a better encoding job so the movie is now watchable.

This next example is how MSE VQGAN+CLIP z+quantize interprets various common human phobias. Text prompts were “a hyperrealistic painting depicting acrophobia” etc. To try and smooth out the “flickering” when zooming I started using ImageMagick for zooming. ImageMagick allows sub pixel image resizing options. This movie was also originally 480p and upsized to 4K in Davinci Resolve before uploading.

I have also added some basic scripting (as in automating a series of steps rather than a Python py script) support to Visions of Chaos. Scripting allows the prompt, zoom speed, rotation and panning to be changed during the movie with smooth interpolations between them each frame.

The following video is a test of the scripting. This video is a Powers of Ten homage with zooming in from the largest scales to the smallest scales.

Another recent addition is the ability to use a series of images as “seed images” that are processed one at a time and then combined into a movie. The following GIF of the Alien chestburster scene is an example of this. The Text-to-Image prompt was “impasto oil painting”.

This next example movie is showing a “Self-Driven” zoom movie. As in a regular zoom movie the output frames are slightly stretched and fed back into the system each frame. The self-driven difference with this movie is that the Text-to-Image prompt text is automatically changed every 2 seconds by CLIP detecting what it “sees” in the current frame. This way the movie subjects are automatically changed and steered in new directions in a totally automated way. There is no human control except me setting the initial “Rainbow colored blobs” prompt. After that it was fully automated.

By default the CLIP Image Captioning script is very good at detecting what is in an image. Using the default accuracy resulted in a zoom movie that got stuck with a single topic or subject. One got stuck on a slight variation of a prompt dealing with kites, so as the zoom movie went deeper it only showed kites. Luckily after tweaking and decreasing the accuracy of the CLIP captioning the predicitons allow the resulting subjects to drift to new topics during the movie.

---

Name: Monster Maker
Author: P_Hoep
Original script: https://colab.research.google.com/drive/1ZbLnt5fLS_BDfpQY-9Dh_T40pLjfqSAC
Time for 512×512 on a 3090: 2 minutes 01 seconds
Description: No longer available. I was contacted by the author who does not want it shared publicly. The colab link no longer works.

a black and white photo of a library in the style of Rembrandt

a forest fire

a forest path

a heart made of feathers

a surrealist painting of the Las Vegas strip

---

Name: CLIP Guided Diffusion
Author: Katherine Crowson
Original script: https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj
Time for 256×256 on a 3090: 1 minutes 35 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: This one gives very unique results compared to the other scripts. Locked to 256×256 resolution. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

a clown

a hyperrealistic painting of a witch

a sea monster

a surrealist sculpture of an android

Brad Pitt

New York City

---

Name: CLIP Guided Diffusion v2
Author: afiaka87
Original script: https://colab.research.google.com/github/afiaka87/clip-guided-diffusion/blob/main/colab_clip_guided_diff_hq.ipynb
Time for 256×256 on a 3090: 2 minutes 38 seconds
Maximum resolution on a 24 GB 3090: Locked to 256×256
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
escription: Modified CLIP Guided Diffusion with more options. This one gives very unique results compared to the other scripts. Locked to 256×256 resolution. Hopefully larger resolution versions of this script will appear in the future. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

a digital painting of a crying person

a fine art painting of heaven in the style of Edvard Munch

a flemish baroque of an angry person

a flemish baroque of hell

a surrealist painting of a witch

the australian outback

---

Name: CLIPRGB
Author: Jonathan Whitaker
Original script: https://colab.research.google.com/drive/1MiKaFFgau6V5QhIed5tpNdLUiSbof4nI
Time for 512×512 on a 3090: 4 minutes 51 seconds
Maximum resolution on a 24 GB 3090: 4096×4096
Maximum resolution on an 8GB 2080: 4096×4096
Description: Very early 0.1 version shows a lot of potential. Can render huge resolution images up to 4096×4096 on a 3090 so I am really looking forward to future versions of this code with sharper details.

a digital painting of a wizard

a forest path

a tattoo of planets

a vampire

---

Name: CLIP Guided Diffusion v3
Author: Michael Friesen
Original script: https://colab.research.google.com/drive/1Fl2SZvLv23MVSAHxkoiNdxPeAZwibvu1
Time for 512×512 on a 3090: 2 minutes 23 seconds
Maximum resolution on a 24 GB 3090: Locked to 512×512
Maximum resolution on an 8GB 2080: Unable to run on 8GB VRAM
Description: Modified CLIP Guided Diffusion that generates larger 512×512 images. Some of the results can be very detailed and interesting, but a lot of time it is hit and miss to get a result that reliably matches the input phrase. The following samples came hand picked from a large batch run of random phrases.

a cubist painting of a castle

a human made of vines

a rough seascape

frogs

h r giger

a matte painting of a landscape

---

Name: Zoetrope 5
Author: Bearsharktopusdev
Original script: https://colab.research.google.com/drive/1LpEbICv1mmta7Qqic1IcRTsRsq7UKRHM
Time for 512×512 on a 3090: 2 minutes 36 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1280×720
Maximum resolution on an 8GB 2080: 256×256 2 minutes 09 seconds
Description: Nice crisp results. Can generates up to 720p (1280×720) resolution images on a 3090. Includes a lot of new ideas from multiple people to help improve the outputs.

a detailed painting of a Pixar character

a futuristic city

a planet

a surrealist sculpture of a sea monster

an art deco scultpture of a policeman

cyberpunk art of a forest fire in the style of Edvard Munch

---

Name: CLIP RGB Optimization
Author: hotgrits
Original script: https://cdn.discordapp.com/attachments/730484623028519072/871624258260987934/CLIP__RGB_Optimization_v0_3.ipynb
Time for 512×512 on a 3090: 2 minutes 50 seconds
Maximum resolution on a 24 GB 3090: 4096×4096
Maximum resolution on an 8GB 2080: 4096×4096
Description: Another CLIP RGB based script without the pixelated artefacts of the CLIPRGB script. Can render huge resolution images up to 4096×4096 on a 3090. This script gives more impressionistic textures. By default the output was a bit too dark for my liking so I have added options to tweak the gamma and contrast of the output images in the script. The gamma and contrast tweaks are only at the display stage and do not change the internal image being generated.

a babbling brook

a movie monster

an amusement park

Chewbacca

Freddy Kruger in the style of Rembrandt

---

Name: MSE Regulized VQGAN+CLIP
Author: jbusted
Original script: https://colab.research.google.com/drive/1hf1seGOZctOJUznkhJNblLluXHbWLKZh
Time for 512×512 on a 3090: 3 minutes 16 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 128×128 2 minutes 30 seconds
Description: Generates good images but they tend to be inside a grey/purple border void.

a bronze sculpture of a heart

a cubist painting of Buzz Lightyear

a house made of string

an art deco sculpture of a vampire

chalk art of C-3PO

---

Name: Sequential VQGAN+CLIP
Author: Jakeukalane and Avengium
Original script: https://colab.research.google.com/drive/1CcibxlLDng2yzcjLwwwSADRcisc1qVCs
Time for 512×512 on a 3090: 1 minutes 41 seconds
Maximum resolution on a 24 GB 3090: 768×768 or 1120×480
Maximum resolution on an 8GB 2080: 256×256 2 minutes 11 seconds
Description: Really nice results and fast.

a campfire in the style of Vincent van Gogh

a colorful parrot

a hyperrealistic painting of C-3PO

an impressionist painting of Buzz Lightyear made of paper

New York City

---

Name: CLIPRGB ImStack
Author: Jonathan Whitaker
Original script: https://colab.research.google.com/drive/1MCC2IwAaRNCTBUzghuG41ypAkxjJvGtq
Time for 512×512 on a 3090: 2 minutes 07 seconds
Maximum resolution on a 24 GB 3090: 2048×2048
Maximum resolution on an 8GB 2080: 512×512 6 minutes 21 seconds
Description: Another CLIP RGB variation. Nice results after some brightness, contrast and sharpness tweaks to the generated images. Could still be a bit sharper.

a fine art painting of an angry person

a fireplace in the style of Claude Monet

a frog in the style of Beksinski

a nightmare creature in the style of H R Giger

a pointalism painting of a vampire made of copper

---

Any Others I Missed?

Do you know of any other colabs and/or github Text-to-Image systems I have missed? Let me know and I will see if I can convert them to work with Visions of Chaos for a future release. If you know of any public Discords with other colabs being shared let me know too.

Jason.

NVIDIA OptiX Denoiser now included with Visions of Chaos

When you render Flame Fractals the pixels accumulate over time. As this process happens different areas accumulate at different rates. Areas that have high hit counts render smoothly while other areas that have lower hit counts render more noisy/speckled.

While I was recently revisiting the Fractal Flame mode in Visions of Chaos I experimented with some ideas I had to make a “smart blur” function. This was going to supposedly blur pixels by different amounts depending on their neighbors. So areas with more noise would be blurred more than areas with little difference between pixels. After some experimenting with various convolution kernels etc I gave up. My results were either too blurry, too dark or other issues.

Searching for other methods got me to “denoising”.

Note: all of the example images in this post need to be seen full 4K size to notice the differences.

NVIDIA OptiX

NVIDIA OptiX is a raytracing engine. Part of what it can do is denoising.

Denoising is a big deal in ray tracing areas. When raytracing images (depending on the method used) you need to make many passes of the image before the details are smoothed out.

Here is an example from Visions of Chaos (the “Pathtracing Global Illumination” example shader) after running for a few seconds.

You can see a lot of noise visible as the rendering was stopped way before the pathtracing converged to a smooth image.

Running the image through the OptiX denoiser gives the following result.

A much smoother result.

The alternative is to allow the pathtracer to run for much much longer until it smooths itself out. The next image took around 10 minutes to accumulate the smoothness that OptiX took a second to smooth. If you look closely the details are better in this image compared to the denoised image.

In some cases the noise may be desirable. If like me you prefer the film grain of a good 70’s movie to the smooth digital look of a modern film then a bit of noise may be more aesthetically pleasing to you, but in general less noise is desirable in rendered images.

Accumulation Fractals

For Visions of Chaos my interest in denoising was for modes like Buddhabrots, Flame Fractals, Iterated Function Systems, and Strange Attractor images. All the modes that have images that are built up by the accumulation of points. A side effect of this accumulation is that areas of the image less hit show up noisier than areas hit more frequently.

Denoising to the rescue!

When It Works

Here are some examples of how the denoiser can help. It seems to work best (at least from my initial experiments) with images that have whispy details.

Iterated Function System with noisy areas.

The same image processed with OptiX.

Flame Fractal with noisy areas.

The same image processed with OptiX.

Strange Attractor with noisy areas.

The same image processed with OptiX.

When It Doesn’t Work

Denoising is not a magic bullet to fix any pixelated image. It was specifically trained to denoise noisy raytraced images.

Here are a few example images that it did not help. When the denoising “fails” it tends to smear areas of the image rather than enhancing them.

Buddhabrot Fractal. Surprisingly Buddhabrots do not seem to be a good candidate for OptiX denoising. The noisy areas do get denoised, but areas with details tend to get smeared more.

Buddhabrot before.

Buddhabrot after OptiX denoising.

Multi-Scale Turing Pattern before.

Multi-Scale Turing Pattern after.

Magnetic Pendulum before. This is a good example of how denoising needs noise to work. This image has a lot of fine detail, but nothing that could be classified as noise.

Magnetic Pendulum after.

Availability

Denoising is now included with Visions of Chaos. I am including the excellent command line version from Declan Russell. Denoising any image created in Visions of Chaos is now just a click away. If you want to denoise other images outside Visions of Chaos I highly recommend Declan’s implementation.

Jason.

Dendritic Crystal Growth

Dendrites are the multi-branched fractal like patterns that can grow in crystals or metals. Snowflakes are also a good example dendritic growth.

After seeing this post on r/Simulations that linked to the paper Numerical Simulation of Dendritic crystal growth using phase field method and investigating the effects of different physical parameter on the growth of the dendrite (great name) which includes some Matlab code I was able to add Dendritic Crystal Growth as a new mode in Visions of Chaos.

Also see the original paper referenced which is Ryo Kobayashi’s “Modeling and numerical simulations of dendritic crystal growth“.

Results

Here are some results from the new mode. I have also added an option to give the growths a 3D embossed look.

Here is a sample movie

Bug fix for code

At first my translation of the Matlab code in the paper did not result in anything but boring smooth edged blobs. I don’t own a copy of Matlab to verify the code, but I assumed (as any programmer does when their code doesn’t work) that it must be something I am doing wrong. But after some time checking and double-checking I did find some problems with the code in the paper.

Firstly this line near the end

tnew(i,j) =t(i,j) + lap _t(i,j)*dt + k*(phi(i,j) -phiold);

needs to be changed to

tnew(i,j) =t(i,j) + lap _t(i,j)*dt + k*(phinew(i,j) -phiold);

otherwise the phi-phiold cancels out and k is multiplied by zero.

Secondly the calculations for grad_epsilon_x and grad_epsilon_y

grad_epsilon2_x = (epsilon(ip,j)^2 -epsilon(im,j)^2)/dx;
grad_epsilon2_y = (epsilon(i,jp)^2 -epsilon(i,jm)^2)/dy;

should be moved to the second loop before the term1 calculation.

With those two simple fixes my code was generating the correct dendritic structures and created the images in this post.

Availability

Dendritic Crystal Growth and two Snowflake modes are now included under the new Dendritic Growth mode menu in Visions of Chaos.

Jason.

Combinations Cellular Automata

This is a new idea for a 1D cellular automata that came from Asher (blog YouTube).

Trying to explain this clearly is harder work than programming it.

Rule string

Combinations CAs can have between 2 and 10 maximum states per rule. 2 states means the cells can be either on or off. 3 states mean cells have 1 dead state and 2 living states.

The CAs are governed by a rule string.

The rule string needs to be states^2 characters in length. For 2 state rules the rule string will need 4 characters, for 4 states the rule string needs 16 characters, for 10 states the rule string needs 100 characters.

For 2 states the rule string characters are in base 2, so binary values of either 1 or 0. For 3 states the rule string is base 3, so the characters are all between 0 and 2. For 10 states the rule string is base 10, so the characters are all between 0 and 9.

An example 2 state rule is 0110 which creates the following;

An example 3 state rule could be 012120200 that creates the following result;

An example 10 state rule could be

50391696401156117866581414266160495600057647563383
52633979845117861158665940485681011955680007489199

which gives this result;

Number of possible rules

When you have a maximum of 2 states, the rule string has 4 digits of either 0 or 1. The maximum rule string is 1111, which converts to 15 in binary. That gives you 16 possible rules (0000 is also counted) for state 2.

Increasing to 3 states has a 9 digit rule string with base 3 digits. 222222222 is the maximum base 3 number which translates to 19682, so there are 19683 possible 3 state CA rules.

4 states is 16 characters. 3333333333333333 is 4,294,967,295 in decimal, so 4,294,967,296 possible 4 state rules.

10 states is 100 characters. No need for decimal conversion as base 10 is decimal. So 1 with 100 zeros after it possible rules. That may as well be an infinite space to look for rules within.

How the cells are updated

Each cell in the CA is updated by taking into account 2 cells from the previous step. These 2 cells are converted into a decimal value that is used as an index in a rule string to give the new cell state. I bet that makes loads of sense! Let me try and explain.

An example with a 2 state CA using rule 0110, so cells can only be either state 0 or state 1.

If the first step is a single center cell, then the values would look like;

00000100000
?

Cell at ? looks at the cell above it and the cell next to it. These are 00. 00 converted to decimal is 0. So we look at the 0th entry in the rule string, so the cell value becomes 0.

00000100000
0

Same for the next 3 cells

00000100000
0000

Then we get to the next ? cell

00000100000
0000?

This cell has the cell above it and next to it as 01. 01 converts to decimal 1, so we look at the 1th (second) digit in the rule string. The second digit in the rule is a 1, so the new cell becomes a 1.

00000100000
00001?

The next ? cell now has 10 above it. This converts to a 2 in decimal, so we look at the third rule digit. Again it is a 1.

00000100000
000011

The rest of the row has 00 above them, so they get set to 0.

00000100000
00001100000

If you continue this process, you get the following result;

Brick wall neighborhood

This CA also uses the brick wall neighborhood idea for selecting neighborhoods. See here for an animation of brick wall. Basically you shift the neighbor locations being checked every other line.

For the Combinations Cellular Automata in this post this means the location of the 2 neighbor cells changes every line.

For even lines you use the cell directly above and to the right. For example, in this next diagram the ? cell would use the 3 and 4 locations.

12345
  ?

For odd lines you use the cell to the left and the cell above. This would be the 2 and 3 neighbor cells.

Interesting Rules

Rule 002200121 behaves similar to Wolfram’s Rule 30. During bulk runs of rules I have seen other results that also look like this one.

Another result that has been observed to occur are these “counting triangles” rules. These show similar behavior to Wolfram’s “Rule 225“. The splitting of bifurcation like structure on the right hand side is like a binary counter. This is rule 010221200.

Rule 200122011 also gives a binary counter structure.

More example images

See my flickr gallery for more example results.

Availability

Combinations Cellular Automata are now included in Visions of Chaos.

Jason.

Visions of Chaos movie tutorials

For years now I have been asked for some more in depth tutorials on Visions of Chaos. One recent comment on my YouTube said a movie watching me use it may help them.

I have wanted to do movie tutorials but I was never happy with my voice. After doing these first 3 parts I have nothing but the utmost respect for anyone that can do this easily. Even with a full script pre-written of what I want to say it is not easy.

I used the free OBS Studio to capture my screen and then the Davinci Resolve free version for editing. Davinci has been great for chopping out all my seemingly endless umms, errs, pauses, stutters, flubbed lines, etc.

For the first 3 tutorials I used a super cheap headset with a built in microphone so the sound is very rough, but hopefully they can help people and get some feedback on other areas of the program people want tutorials on.

After buying a better microphone I recorded the following TensorFlow tutorials.

There are occasional very low end thumping noises. They were a mystery at first (sounds almost like construction noise, but there was none that I could hear at the time of recording) but turned out to be the lift in my building. Otherwise you may hear cicadas and other Australian wildlife in the background.

Another tutorial about the Physarum Simulations in Visions of Chaos.

Number 6 is a more relaxed non-scripted tutorial on Video Feedback simulation. This one was a bugger to edit.

The 7th tutorial covers how to print Mandelbrot Fractals across multiple pages to make large wall sized posters.

The 8th tutorial covers DeepDream functionality in more depth.

The 9th is all about Style Transfer

I wouldn’t say I am getting good at this yet, but the recording and editing process is getting easier. If you have any specific requests for tutorial topics let me know.

Jason.

Automatic Color Palette Creation

Fractint MAP format palette files

Going back 30 years, Fractint was a fractal generation program for DOS based systems. For its time it was the fractal program of choice for enthusiasts.

Fractint used a simple text format for its color palettes. These *.MAP files were text files with each color’s RGB values separated by spaces each on a new line. So, for example if you wanted the first color in your palette to be blue the first line would be “0 0 255”.

When I first started creating Visions of Chaos I adopted the format. The most common map files had 256 colors (you could have palettes with other color counts but I only use 256 color palettes).

The rest of this post covers the palette creation methods that have been included with Visions of Chaos. Although I use these methods specifically to create 256 color MAP files the principles could be applied to any number of colors for different sized palettes.

If you are just looking for a Fractint color palette collection, scroll down to the end of this post and grab the archive provided.

Smoothly blending colors

This is probably the first and most obvious method to use. Take a small number of base colors (I allow up to 16) and blend them into a palette.

How you get the colors to blend can be;

1. User selects them from the standard color picker dialog.
2. User can use eye dropper functionality to pick them out of a photo.
3. Set them at random.
4. Use the color wheel. Allows selection of complmentary colors, tetrads, and other color theory based colors.

5. Extract colors from an image. See this previous blog post explaining how that works.

Once you have the colors there are numerous ways you can blend them;

1. Smooth blend. Smoothly interpolate the colors.

2. Fade out blend. Fade each of the colors to black.

3. Fade in blend. Fade each of the colors from black.

4. Neon blend. Fade from black to the colors then back to black.

5. Stripe blend. Alternate each color for the duration of the palette.

Using curves to create palettes

The idea here is to use various mathematical functions to generate curves for the RGB components of the palette. The following is a list of the various methods I use so far.

Sine. Each RGB color component is its own sine wave. Randomize the wave amplitude, frequency and period.

Multiple Sine. Add multiple sine waves together for each RGB component and then scale down to between 0 and 255.

IQ. Idea from Inigo Quilez.

Perlin. Use repeating noise loops as in this coding train video. Map the resulting noise values to each RGB channel. Using a looping noise function is best because it means the palette wraps around smoothly and using it for fractal zooms does not show a sharp break when the palette ends and restarts. I have only implemented this method over the last few days (at the time of writing this post), but so far it gives some really unique color palettes.

Here are some examples palettes created using Perlin noise. Click to see the full sized image.

Simplex. Same as Perlin, but uses Simplex noise.

Simplex + Perlin. Create each RGB value by adding Simplex noise to Perlin noise.

Here are some examples of Simplex and Simplex + Perlin palettes. Click for full size.

Multiple Perlin – Add/subtract multiple Perlin Noise curves into RGB amounts.

Random Walk. Random curve for each RGB component between index 0 and 127. Reverse for the rest of the palette. Each step the RGB is changed by +random(5)-2 to randomly go up and/or down.

Terrain Fault. Take 2 random points between 0 and 255. Between the points randomly raise or lower by a small amount. Repeat this a number of times.

HSL to RGB. Random HSL curves converted to RGB.

RGB. Random curves for each RGB component. Use various easing functions to tween curve control points.

YUV to RGB. Random YUV curves converted to RGB.

Combine palettes. Take 2 previously created palettes and combine their RGB components by addition, subtraction or multiplication.

Multiple RGB. Combine multiple RGB curves.

Multiple YUV to RGB. Combine multiple YUV to RGB curves.

Modify an existing palette

Once you have palette files, you can also use various techniques to modify them;

1. Increase or decrease the individual RGB channel amounts
2. Brightness
3. Contrast
4. Increase or decrease the individual YUV channel amounts
5. Wrap. Take the existing palette, halve it, then add the flipped half to itself. This is useful when you want a non repeating palette to wrap around.

6. Double. If you have a palette that is too smooth/sparse for the current fractal image, doubling can add more lines/gradients to the palette.

7. Blur. Just like a blur function in image processing. Averages out the palette values with neighbor colors.
8. Sharpen. Just like a sharpen function in image processing.
9. Shift RGB. R->G,G->B,B->R.

10. Invert. R=255-R, G=255-G, B=255-B.
11. Reverse. Flip the order of the palette colors.
12. Histogram equalize palette. Like the auto-levels in Photoshop. My method tends to make the results slightly too bright. Needs fixing when I get a chance.

13. Matrix multiplication. Take a 3×3 matrix and multiply the 1×3 RGB components by the matrix to get new RGB amounts.

Any other ideas?

If you know of any other ways to generate palettes, or have an idea for ways to create new unique color palettes, let me know.

Availability

The color palette editor shown in this post is included with Visions of Chaos.

Just give me the palettes!

If you are using another program that uses Fractint palette files you can download the 3371 color palettes I include with Visions of Chaos here. Some created by me, others found on various Internet sites over the years, some converted from gradient packs. No copyright on them so do with them as you wish.

If you do have any other sets of MAP palettes you would like to share, send me an email. You can never have enough colors when creating fractal images.

Jason.

Visions of Chaos now supports Pixar RenderMan

Visions of Chaos can now use Pixar‘s Free Non-Commercial RenderMan to render 3D scenes.

This is the same RenderMan engine used in movies like Finding Dory

and Rogue One

I am only using a tiny fraction of RenderMan’s features. Rendering millions of spheres or cubes with nice shading and lighting covers my requirements, but it is good to know that I have the extra power of RenderMan to expand if needed in the future.

All the end user needs to do is download the stand-alone Pixar RenderMan and point Visions of Chaos to the main command line RenderMan prman.exe renderer file. After that Visions of Chaos constructs the RIB format files RenderMan understands and gets prman to render the images.

Currently RenderMan support is added to the following Visions of Chaos modes;

3D Ant Automata

3D Cyclic Cellular Automata

3D Cellular Automata

3D Hodgepodge Machine

3D Voxel Automata Terrain

3D Diffusion-Limited Aggregation

3D Cube Divider

RenderMan seems to be able to handle a massive number of objects so far. This next test 4K image was over 33 million little cubes. RenderMan churned away for over 2 hours on this one image. The main slowdown was my lack of memory and constant hard drive paging. 32 GB of RAM just does not cut it when rendering this sort of data. The RIB scene file alone was 13 GB.

RenderMan is slower than Mitsuba at this stage. But that could be down to any number of reasons including my lack of RenderMan knowledge.

Jason.