Last Week in AI #198: How GPT learns to reason, Waymo expands to all of SF, Artists can opt-out from Stable Diffusion 3, and more!

Language models learn to reason from code, Waymo significantly expands SF and Pheonix operations, artists can opt-out their work from future stable diffusion datasets

Dec 19, 2022

How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources

In this detailed blog post, researchers from the Allen Institute for AI break down the evolution of OpenAI’s GPT models and offer theories on where their impressive capabilities come from. Two important capabilities of large language models (LLMs) are 1) the ability to perform “in-context learning” (teaching the LLM new tasks with prompt engineering) and 2) how aligned they are with human preferences (ChatGPT is arguably the most aligned LLM today - it follows human instructions very well). The post argues that there is a trade-off between these two - well-aligned models tend to perform worse at in-context learning, and vice-versa. Further, the post presents convincing evidence that complex reasoning capabilities come from training LLMs on code. Even for natural language tasks that have nothing to do with code, including code in the training set proved pivotal to task performance.

Our take: This is one of the most accessible and insightful sources I’ve read that demystifies LLMs and explains OpenAI’s models in particular. What’s exciting is that we’re seeing a better and better understanding of LLMs from the scientific community in just recent months, and no doubt this understanding will enable us to train better LLMs more efficiently in the near future. If you’re interested in how GPT and other LLMs work, I recommend checking out this post in detail.

Wheels up for Waymo as we expand our 24/7 rider-only territories

Waymo recently expanded its robo-taxi service areas. In Pheonix Arizona, Waymo now allows rides between the airport and the downtown Pheonix area (before it was just downtown). More impressively, Waymo can now serve all of SF, with some areas currently serving only employees and guests. The Waymo robo-taxi could be hailed 24/7 in SF, and it is the only company that can do so for multiple metro areas at this time.

Our take: This is yet another milestone for commercial self-driving. While this year has been tough for the sector (Argo’s shutdown, shutdowns of multiple sidewalk delivery robots), Waymo and Cruise have been making steady progress toward the vision of robo-taxis. It would not be a surprise if after these companies’ operations mature in SF, expansions to other US cities happen much faster. I think this year we can finally say with confidence that robo-taxis are “almost here” for most urban dwellers in America.

Stability AI plans to let artists opt out of Stable Diffusion 3 image training

An AI-generated image of someone leaving a building.

One of the many debates storming in the AI sphere recently surrounds the claim that "AI art is theft." Indeed, DALL-E, Stable Diffusion, and their ilk are trained on data that includes the works of many artists. You can, therefore, prompt them to deliver you visuals in the style of a particular artist. In light of this, Stability AI has announced that it would allow artists to remove their work from Stable Diffusion 3.0's training set. Ars Technica, who investigated the opt-out system, reports that artists must register and manually flag images in the LAION database--they cannot flag groups of pictures at a time and must manually search for and flag duplicated images. Stability AI is operating within US/European law to train Stable Diffusion on images scraped without permission, but they and their CEO acknowledge the ethical debate and appear open to suggestions.

Our Take: Artists are, understandably, not too happy that their work might suffer as a result of AI art--potential buyers may opt to generate AI art in an artist's style, and the shareability of AI art may flood the internet to such a point that it will be difficult to find an artist's real work online. I sympathize with this point of view. It is already tough to make a living as an artist, and I wonder what the flood of AI-generated art will do for our appreciation of art, its style, and its contents. We would do well to remember that "progress" is a complicated, hairy thing. AI art is one of many examples we will see in the coming years that demonstrate just how complicated progress can be.

Other News

Research

$71 Coat Makes Wearers Invisible to AI Security Cameras - "Students in China have invented a $71 “invisibility cloak” that hides wearers from artificial intelligence (AI) security cameras."

Using machine learning to improve the toxicity assessment of chemicals - "machine learning to assess the toxicity of chemicals."

Model shows how intelligent-like behavior can emerge from non-living agents - "A new model describes how biological or technical systems form complex structures equipped with signal-processing capabilities that allow the systems to respond to stimulus and perform functional tasks without external guidance."

Waymo's Collision Avoidance Testing: Evaluating our Driver’s Ability to Avoid Crashes Compared to Humans - "Fully autonomous systems need to handle the entire driving task with no human in the driver’s seat, so they go through much more in-depth testing than driver assist systems."

Black hole growth unveiled by Machine Learning - "In a new study, researchers have used Machine Learning to deduce the rules governing the relationship between the growth of a galaxy and the growth of the supermassive black hole at its centre."

Machine learning massively speeds up scouring of periodic table for stable structures - "An algorithm has been created that can rapidly scan hypothetical crystal structures containing any natural element and find those likely to be stable."

New machine learning model could encode words to kill cancer cells - "US researchers, using new machine learning techniques have developed a virtual molecular library of “words” that encode commands to kill cancer cells."

Researchers to Develop Machine-Learning System to Automate Insulin Delivery - "A $1.2 million NIH grant is supporting the development of a machine-learning tool to enhance the accuracy of an artificial pancreas system for type 1 diabetes patients."

Geoffrey Hinton’s Forward-Forward Algorithm Charts a New Path for Neural Networks - "In his NeurIPS keynote speech last week, Hinton offered his thoughts on the future of machine learning — focusing on what he has dubbed the “Forward-Forward” (FF) algorithm."

Novel machine learning method detects animal coronaviruses that might infect humans - "In a recent study posted to the bioRxiv* preprint server, researchers used machine learning (ML) tools to discover animal coronaviruses (CoVs), both alpha and beta CoVs, previously unknown to infect humans."

Using machine learning to better understand how water behaves - "Molecular simulation results showing how water molecules move and structure around one another in the high density liquid phase."

Applications

Machine learning and the arts: A creative continuum - "CAST Visiting Artist Andreas Refsgaard engages the MIT community in the ethics and play of creative coding."

DoNotPay is launching an AI chatbot that can negotiate your bills - "DoNotPay, the company that bills itself as “the world’s first robot lawyer,” is launching a new AI-powered chatbot that can help you negotiate bills and cancel subscriptions without having to deal with customer service."

YouTube moderation bots will crack down on spam in comments, live chats - "YouTube has announced a plan to crack down on spam and abusive content in comments and livestream chats. "

The AI that might take your job, make your medical decisions and decide your bail - "From ChatGPT to DALL-E, Dave Maclean reports on what’s possible now — and what AI is poised to do for us in the future"

Try ‘Riffusion,’ an AI model that composes music by visualizing it - "AI-generated music is already an innovative enough concept, but Riffusion takes it to another level with a clever, weird approach that produces weird and compelling music using not audio but images of audio. Sounds strange, is strange. But if it works, it works. And it does work! Kind of."

Stumbling with their words, some people let AI do the talking - "A machine that talks like a person has long been a science fiction fantasy. Now it’s closer to reality than ever before."

Lore Machine Meets Unlimited Dream Co. - "From Eyeball Dog Pagodas to Brain-Scanning GANs"

Business

Machine Learning Market is Projected to Reach US$ 276.58 Billion in 2028 - "Machine Learning Market is expected to reach US$ 276.58 Billion in 2028, growing at a CAGR of 44% during 2022-2028, reports Stratview Research. Raipur, Dec."

Picsart’s AI Image Generator Can Now Replace Objects and Create Backgrounds - "Picsart is updating its generative artificial intelligence (AI) system to allow it to replace objects in photos as well as create entirely new backgrounds."

Shield AI raises another $60M at a $2.3B valuation for its military autonomous flying tech - "Defense technology continues to get a lot of attention from investors, and today, one of the bigger startups in the space is announcing more funding. Shield AI — which develops platforms and planes for autonomous flying systems, targeting the U.S."

NTT Data Innovation Index: 93% of Execs Think AI Will Be in Common Use Within 2 Years - "NTT DATA, a $30 billion global innovator of digital business and IT services, has released its latest annual research report that shows business leaders are shifting priorities from survival to growth."

Alphabet’s Intrinsic Acquires Majority of Open Robotics - "Today, Open Robotics, which is the organization that includes the nonprofit Open Source Robotics Foundation (OSRF) as well as the for-profit Open Source Robotics Corporation (OSRC), is announcing that OSRC is being acquired by Intrinsic, a standalone company within Alphabet that’s developing softw"

Google execs warn company's reputation could suffer if it moves too fast on AI-chat technology - "CEO Sundar Pichai suggested that the company has chat products underway for 2023."

A.I. startup Cohere lands a top YouTube exec in another sign of NLP’s acceleration - "One overwhelming trend that was clear from the conference was the way in which advances in natural language processing are rapidly transforming business. The technology is enabling unstructured data to be pulled directly from written documents and analyzed and summarized."

Exclusive: ChatGPT owner OpenAI projects $1 billion in revenue by 2024 - sources - "Dec 15 (Reuters) - ChatGPT, the new chatbot that is the talk of Silicon Valley, can spit out haikus, crack jokes in Italian and may soon be the scourge of teachers everywhere facing fake essays generated by the AI-powered technology."

Rick And Morty Creator Used AI Art, Voice Acting In Recent Game - "As the debate around AI “art” continues to rage on sites like ArtStation, Twitter, and Reddit (and likely in the comments below this very article) it’s already being used in commercial projects."

Amazon Devices Chief Pledges Big Alexa Bets Despite Job Cuts - "Amazon.com Inc.’s devices chief said the company remains committed to building out the Alexa ecosystem despite job cuts in the once fast-growing division. "

Wayve puts forward its end-to-end Deep Learning Model MILE for Self-Driving Cars - "Wayve, a London-based startup, rethinks how to solve the problem of autonomous automobiles and build new solutions deriving AI’s power."

Concerns

AI unmasks anonymous chess players, posing privacy risks - "Software that identifies unique playing styles could lead to better tutorials and game play"

AI composer bias: Listeners like music less when they think it was composed by an AI - "The use of artificial intelligence (AI) to compose music is becoming mainstream. Yet, there is a concern that listeners may have biases against AIs. Here, we test the hypothesis that listeners will like music less if they think it was composed by an AI."

AI art apps are cluttering the App Store’s Top Charts following Lensa AI’s success - "Lensa’s AI popularity has had a notable impact on the App Store’s Top Charts."

Tesla Full Self-Driving data looks awful: We challenge Elon Musk to prove otherwise - "The little data that is coming out of Tesla’s Full Self-Driving Beta looks awful, and that’s putting it mildly. We challenge Elon Musk to prove otherwise by releasing disengagement and driver intervention data."

A Tech Worker Used AI To Make A Children’s Book And Professional Illustrators Are Pissed - "Ammaar Reshi told BuzzFeed News that he has received death threats and messages encouraging self-harm on social media."

Image-generating AI can copy and paste from training data, raising IP concerns - "Image-generating AI models like DALL-E 2 and Stable Diffusion can — and do — replicate aspects of images from their training data, researchers show in a new study, raising concerns as these services enter wide commercial use."

The College Essay Is Dead - "Nobody is prepared for how AI will transform academia. Suppose you are a professor of pedagogy, and you assign an essay on learning styles. A student hands in an essay with the following opening paragraph:"

The viral AI avatar app Lensa undressed me—without my consent - "My avatars were cartoonishly pornified, while my male colleagues got to be astronauts, explorers, and inventors."

How it feels to be sexually objectified by an AI - "This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. My social media feeds this week have been dominated by two hot topics: OpenAI’s latest chatbot, ChatGPT, and the viral AI avatar app Lensa."

‘Magic’ AI Avatars Are Already Losing Their Charm - "When Lensa, a four-year-old photo-editing app, introduced a new AI-portrait generator in November, the internet was enthralled. But concerns have mounted, particularly among women"

Is Lensa AI Stealing From Human Art? An Expert Explains The Controversy - "The Lensa photo and video editing app has shot into social media prominence in recent weeks, after adding a feature that lets you generate stunning digital portraits of yourself in contemporary art styles."

Artists Are Revolting Against AI Art on ArtStation - "Starting Tuesday, artists began protesting against the platform by uploading the image onto their portfolios after some users pointed out that AI-generated art was being featured on the site’s main Explore page."

AI-generated fake faces have become a hallmark of online influence operations - "Fake accounts on social media are increasingly likely to sport fake faces. Facebook parent company Meta says more than two-thirds of the influence operations it found and took down this year used profile pictures that were generated by a computer."

Tor Tried to Hide AI Art on a Book Cover, and It Is a Mess - "The globally acclaimed science fiction and fantasy publisher delivers a weak apology for using AI art for a not-yet-published book."

Analysis

Is AI Going To Kill the Student Essay? College Professors Aren’t Convinced - "At UCLA, Professors See 'Exciting Opportunities' in AI Writing Tools"

Voice assistants failed because they serve their makers more than they help users - "By 2022 we'd have self-driving cars, robo-maids, even voice-activated "friends" – digital companions to keep us well-informed. What went wrong?"

What Would Plato Say About ChatGPT? - "Plato mourned the invention of the alphabet, worried that the use of text would threaten traditional memory-based arts of rhetoric."

The state of AI in 2022—and a half decade in review - "The results of this year’s McKinsey Global Survey on AI show the expansion of the technology’s use since we began tracking it five years ago, but with a nuanced picture underneath."

Generative AI is changing everything. But what’s left when the hype is gone? - "No one knew how popular OpenAI’s DALL-E would be in 2022, and no one knows where its rise will leave us."

ChatGPT hype in the Washington Post - "The Washington Post generally has better than average tech coverage, I’ve found (less filled with hype, more likely to focus on the way in which tech companies wield power), but their December 10th piece on ChatGPT is a study in AI hype."

Policy

For AI bias law coming January 1, unanswered questions remain - "The Department of Consumer and Worker Protection (DCWP) announced it is postponing enforcement until April 15, 2023"

China Bans AI-Generated Images Which Don’t Have Watermarks - "Authorities in China have made a series of policy announcements regarding artificial intelligence (AI), including banning AI images that are not labeled as such."

U.S. safety regulators investigating GM's Cruise self-driving car system - "U.S. automotive safety officials have opened a formal probe into the autonomous driving system used by General Motors' self-driving vehicle unit Cruise."

China wants legal sector to be AI-powered by 2025 - "China wants its judicial sector to be supported by an artificial intelligence (AI) infrastructure that must be in place by 2025. The directive aims to drive integration of AI with judicial work and enhance legal services."

Expert Opinions

AI timelines: What do experts in artificial intelligence expect for the future? - "Many AI experts believe there is a real chance that human-level artificial intelligence will be developed within the next decades, and some believe that it will exist much sooner."

Fun

Meet Ghostwriter, a haunted AI-powered typewriter that talks to you - "On Wednesday, a designer and engineer named Arvind Sanjeev revealed his process for creating Ghostwriter, a one-of-a-kind repurposed Brother typewriter that uses AI to chat with a person typing on the keyboard."

Last Week in AI