Last Week in AI

Share this post

Generating AI Art from Text with Google Colab

lastweekin.ai
Editorials

Generating AI Art from Text with Google Colab

A brief overview and guide to playing with cutting edge AI easily with Google Colab

Andrey Kurenkov
Feb 11, 2022
Share this post

Generating AI Art from Text with Google Colab

lastweekin.ai

Generating AI Art from Text with Google Colab

With Dream, using AI tools to generate art has officially gone mainstream in a big way, powered by the viral popularity it gained on TikTok. Amazingly, the technique that powers this app was introduced less than a year before the app itself was released, when OpenAI announced CLIP and DALL-E – a model to score whether some text describes the contents of an image and another model to generate images from text, respectively. 

The period between the announcement of DALL-E and the release of Dream is in itself an interesting one, during which many AI researchers and hackers got hooked to playing around with these techniques and posting their weird creations on Twitter. As DALL-E was not open-sourced but CLIP was, these same researchers and hackers found ways to cobble together their own approximations of DALL-E by combining the image-generating powers of VQ-GAN with CLIP, as covered well in the article Alien Dreams: An Emerging Art Scene some six months before Dream came to be. 

Twitter avatar for @advadnoun
Adverb @advadnoun
The Big Sleep Here's the notebook for generating images by using CLIP to guide BigGAN. It's very much unstable and a prototype, but it's also a fair place to start. I'll likely update it as time goes on. colab.research.google.com/drive/1NCceX2m…
Image
5:25 AM ∙ Jan 18, 2021
625Likes112Retweets

The speed with which AI researchers and hackers started playing with and refining these techniques was in large part powered by one tool: Google Colab. A descendant of the IPython and Jupyter notebook interfaces already commonly used within the AI community, Colab is basically a Google Doc in which you can run code. Importantly, the compute backing running this code is free and moreover comes with a GPU, making it very appealing for AI applications. This enabled people to not only share images from their own CLIP-like implementations, but also to directly share the code necessary to use and build upon these implementations – with no annoying setup steps necessary. Starting with The Big Sleep, new Colab notebooks were regularly developed and released to the community, continually accelerating the process of innovation for such applications. Here’s just a sample of such notebooks that have come out in the past year:

  • VQGAN+CLIP (codebook sampling method)

  • AI Art Machine

  • Create realistic AI-Generated Images with VQGAN+CLIP

  • VQGAN+CLIP (with pooling and quantize method)

  • VQGAN+CLIP (z+quantize method with augmentations)

  • CLIP Guided Diffusion HQ 512x512

Twitter avatar for @arankomatsuzaki
Aran Komatsuzaki @arankomatsuzaki
When you generate images with VQGAN + CLIP, the image quality dramatically improves if you add "unreal engine" to your prompt. People are now calling this "unreal engine trick" lol e.g. "the angel of air. unreal engine"
Image
9:02 PM ∙ May 31, 2021
2,643Likes402Retweets

Colab was not the only thing accelerating progress. Perhaps as important was the sheer number of people playing around with these algorithms, and in the process discovering fun tricks for what could be included in the text inputs to yield different results. Surprisingly, just telling the models to generate something “high resolution” or “rendered by Unity” could often lead to much nicer results, not to mention qualitatively different. These tricks were shared around Twitter, but also on other community spaces such as EleutherAI’s Discord. 

Twitter avatar for @RiversHaveWings
Rivers Have Wings @RiversHaveWings
By a user in EleutherAI Discord, using the VQGAN+CLIP bot there: "recursive recursion of the recursive imagination of a landscape by james gurney" and "landscape of recursion by james gurney". CLIP understands that "recursion" means you put the thing inside itself etc?
Image
Image
10:31 PM ∙ Jul 30, 2021
242Likes34Retweets

So, Google Colab is special in that it is a breeding ground for innovation that enables the whole AI community to play around with new ideas and release their findings into the world – which turned out to be especially true for text-to-image AI art creation. Moreover, Colab allows anyone to play around with cutting edge AI, with the only requirements being a Google Drive account and the time to figure out how a given notebook works. This is especially true for greetings AI images from text, with there being handy tutorials and newer Colab notebooks with user-friendly interfaces that make it easier than ever. In fact, there is a nice compilation of notebooks for all sorts of applications of AI, not just generating images. So, if you haven’t done so already, give it a try! 

An image of ‘friendly terminator’ I just made in Google Colab
Share this post

Generating AI Art from Text with Google Colab

lastweekin.ai
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 Skynet Today
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing