Last Week in AI

Share this post
Generating AI Art from Text with Google Colab
lastweekin.ai
Editorials

Generating AI Art from Text with Google Colab

A brief overview and guide to playing with cutting edge AI easily with Google Colab

Feb 11
Comment
Share

Generating AI Art from Text with Google Colab

With Dream, using AI tools to generate art has officially gone mainstream in a big way, powered by the viral popularity it gained on TikTok. Amazingly, the technique that powers this app was introduced less than a year before the app itself was released, when OpenAI announced CLIP and DALL-E – a model to score whether some text describes the contents of an image and another model to generate images from text, respectively. 

The period between the announcement of DALL-E and the release of Dream is in itself an interesting one, during which many AI researchers and hackers got hooked to playing around with these techniques and posting their weird creations on Twitter. As DALL-E was not open-sourced but CLIP was, these same researchers and hackers found ways to cobble together their own approximations of DALL-E by combining the image-generating powers of VQ-GAN with CLIP, as covered well in the article Alien Dreams: An Emerging Art Scene some six months before Dream came to be. 

Twitter avatar for @advadnounAdverb @advadnoun
The Big Sleep Here's the notebook for generating images by using CLIP to guide BigGAN. It's very much unstable and a prototype, but it's also a fair place to start. I'll likely update it as time goes on.
colab.research.google.com/drive/1NCceX2m…
Image

January 18th 2021

112 Retweets625 Likes

The speed with which AI researchers and hackers started playing with and refining these techniques was in large part powered by one tool: Google Colab. A descendant of the IPython and Jupyter notebook interfaces already commonly used within the AI community, Colab is basically a Google Doc in which you can run code. Importantly, the compute backing running this code is free and moreover comes with a GPU, making it very appealing for AI applications. This enabled people to not only share images from their own CLIP-like implementations, but also to directly share the code necessary to use and build upon these implementations – with no annoying setup steps necessary. Starting with The Big Sleep, new Colab notebooks were regularly developed and released to the community, continually accelerating the process of innovation for such applications. Here’s just a sample of such notebooks that have come out in the past year:

  • VQGAN+CLIP (codebook sampling method)

  • AI Art Machine

  • Create realistic AI-Generated Images with VQGAN+CLIP

  • VQGAN+CLIP (with pooling and quantize method)

  • VQGAN+CLIP (z+quantize method with augmentations)

  • CLIP Guided Diffusion HQ 512x512

Twitter avatar for @arankomatsuzakiAran Komatsuzaki @arankomatsuzaki
When you generate images with VQGAN + CLIP, the image quality dramatically improves if you add "unreal engine" to your prompt. People are now calling this "unreal engine trick" lol e.g. "the angel of air. unreal engine"
Image

May 31st 2021

402 Retweets2,643 Likes

Colab was not the only thing accelerating progress. Perhaps as important was the sheer number of people playing around with these algorithms, and in the process discovering fun tricks for what could be included in the text inputs to yield different results. Surprisingly, just telling the models to generate something “high resolution” or “rendered by Unity” could often lead to much nicer results, not to mention qualitatively different. These tricks were shared around Twitter, but also on other community spaces such as EleutherAI’s Discord. 

Twitter avatar for @RiversHaveWingsRivers Have Wings @RiversHaveWings
By a user in EleutherAI Discord, using the VQGAN+CLIP bot there: "recursive recursion of the recursive imagination of a landscape by james gurney" and "landscape of recursion by james gurney". CLIP understands that "recursion" means you put the thing inside itself etc?
Image
Image

July 30th 2021

34 Retweets242 Likes

So, Google Colab is special in that it is a breeding ground for innovation that enables the whole AI community to play around with new ideas and release their findings into the world – which turned out to be especially true for text-to-image AI art creation. Moreover, Colab allows anyone to play around with cutting edge AI, with the only requirements being a Google Drive account and the time to figure out how a given notebook works. This is especially true for greetings AI images from text, with there being handy tutorials and newer Colab notebooks with user-friendly interfaces that make it easier than ever. In fact, there is a nice compilation of notebooks for all sorts of applications of AI, not just generating images. So, if you haven’t done so already, give it a try! 

An image of ‘friendly terminator’ I just made in Google Colab
CommentComment
ShareShare

Create your profile

0 subscriptions will be displayed on your profile (edit)

Skip for now

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.

TopNewCommunity

No posts

Ready for more?

© 2022 Skynet Today
Privacy ∙ Terms ∙ Collection notice
Publish on Substack Get the app
Substack is the home for great writing