Last Week in AI #174: Cerebras sets record for largest AI model on one device, open source large language model, robotaxis paralyzed, and more!

Cerebras trains 20 billion parameter AI model on single chip, open-source language model BLOOM just finished training, fleet of autonomous robotaxis stuck in SF

Top News

Cerebras Systems sets record for largest AI models ever trained on one device

Cerebras Systems, with its latest WSE-2 chip, has set the record for the largest AI model ever trained on a single device. The chip, which has 850k cores and 2.6 trillion transistors, is much larger than the largest GPUs. It has 123x more cores, 1k times more memory, and 12k times more bandwidth than the largest GPU. This allowed Cerebras to train a 20 billion parameter neural network model on a single chip. Doing so with GPUs would require complex compute cluster engineering and management, which could be much more expensive and only doable at large tech companies. In addition to training speed, Cerebras touts this ease of use as a major advantage for its large chips, and with them the company hopes to democratize large AI model training.

Our take: This is certainly an impressive milestone as it demonstrates the maturity of this large-chip approach for commercial use. 20 billion parameters is a lot and many fields that do not have access to GPU cluster experts will surely benefit. It is worth noting that this chip does not completely remove the need for compute clusters. OpenAI’s GPT-3 has 175 billion parameters, and Google’s latest language model, PaLM, has 540 billion. Of course, managing tens of Cerebras chips is probably easier to do than hundreds of GPUs. This is a promising direction and we will surely see more successes of AI-specialized chips in the near future.

Open-source language AI challenges big tech’s models

BLOOM Is the Most Important AI Model of the Decade

An international team of around 1,000 largely academic volunteers has tried to break big tech’s stranglehold on natural-language processing and reduce its harms. Trained with US$7-million-worth of publicly funded computing time, the BLOOM language model will rival in scale those made by firms Google and OpenAI, but will be open-source. BLOOM will also be the first model of its scale to be multilingual.

Our take: This is incredibly exciting! These sorts of models are increasingly becoming pivotal to AI progress, and have largely been exclusively being trained by large for profit companies. Having a collective of academics succeed in this initiative makes it far easier for universities and other non for-profit entities to still participate in this space.

Driverless Robotaxi Fleet Paralyzed for Hours in San Francisco

San Francisco has borne witness to the first system breakdown of a driverless taxi fleet. For a few hours, at least a dozen autonomous Chevrolet Bolts blocked the intersection of Gough Street and Fulton Street.

Our take: There's not too much to say here, but this is pretty funny. As driverless vehicles become more and more common, we may see more of this in the future.

Other News

Research

AI May Have Just Invented The Ultimate Battery - "Fluoride batteries are coming."

Meta’s AI Takes an Unsupervised Step Forward - "In the quest for human-level intelligent AI, Meta is betting on self-supervised learning"

Microsoft AI Researchers Open-Source ‘GODEL’: A Large Scale Pre-Trained Language Model For Dialog - "Recent developments in machine learning have made virtual assistants reliable for various activities, including restaurant recommendations, bill-paying assistance, and appointment reminders. A novel work by the Microsoft research team now presents GODEL, a Grounded Open Dialogue Language Model."

Computer scientists' interactive program aids motion planning for environments with obstacles - "Just like us, robots can't see through walls. Sometimes they need a little help to get where they're going. Engineers at Rice University have developed a method that allows humans to help robots "see" their environments and carry out tasks."

Researchers release open-source photorealistic simulator for autonomous driving - "Hyper-realistic virtual worlds have been heralded as the best driving schools for autonomous vehicles (AVs), since they’ve proven fruitful test beds for safely trying out dangerous driving scenarios."

Robot overcomes uncertainty to retrieve buried objects - "For humans, finding a lost wallet buried under a pile of items is pretty straightforward -- we simply remove things from the pile until we find the wallet. But for a robot, this task involves complex reasoning about the pile and objects in it, which presents a steep challenge."

A way to expand training data sets for manipulation tasks improves the performance of robots by 40% or more - "In a step toward robots that can learn on the fly like humans do, a new approach expands training data sets for robots that work with soft objects like ropes and fabrics, or in cluttered environments."

Google researchers develop AI that can answer math questions - "Google today detailed Minerva, an internally-developed neural network that can answer mathematical questions and tackle other complex topics such as physics. Minerva is a natural language processing model."

Applications

Photographer Successfully Uses Dall-E 2 AI to Edit his Photos - "DALL-E 2, an artificial intelligence system that can create photo-realistic images based only on a brief description, has been used by a photographer to fix an out-of-focus photograph."

Google engineer identifies anonymous faces in WWII photos with AI facial recognition - "While visiting the POLIN museum in 2016, Daniel Patt realized he could be walking by his own relatives without knowing it, so he designed a platform ‘From Numbers to Names’"

Amazon’s Alexa could turn dead loved ones into digital assistant - "Amazon plans to let people turn their dead loved ones into digital servants, with the company promising the ability to “make the memories last”."

5 Surprising Cyberattacks AI Stopped This Year - "We're now halfway through 2022, and already we have seen a range of cyberattacks, familiar and unfamiliar, disrupting organizations. However, we have also seen uplifting stories of successful threat detection efforts, as well."

Business

Amazon Shows Off New Warehouse Robots That Actually Seem Useful - "It’s a lot of progress over just one year"

10 new AI unicorns flying high in 2022 - "Here is a look at ten AI companies from the newest unicorn cohort that are soaring in 2022"

Tech Giants Pour Billions Into AI, but Hype Doesn’t Always Match Reality - "Google, Meta and OpenAI are investing heavily in the technology, which is increasingly capturing the public imagination"

Tesla lets go of hundreds of Autopilot data labelers as it closes San Mateo office - "Tesla has laid off about 200 people working at its San Mateo offices as it closes the location. Most of the employees were working as Autopilot data labelers – a critical project for Tesla to achieve self-driving capability."

‘Accent Translation’ Startup Sanas Raises USD 32m in Series A - "Less than a year after its August 2021 founding, accent-translation technology startup Sanas raised USD 32m in a series A funding round, led by Insight Partners. TechCrunch reported post-money valuation to be USD 150m."

Concerns

Companies viewed as less legally liable for discrimination from algorithms, study finds - "People are less morally outraged when gender discrimination occurs because of an algorithm rather than direct human involvement, according to research published by the American Psychological Association."

AI’s Next Big Thing Is ‘Fake’ Data - "Last week Microsoft Corp. said it would stop selling software that guesses a person’s mood by looking at their face. The reason: It could be discriminatory."

‘It’s not going to work’: Keeping race out of machine learning isn’t enough to avoid bias - "As more machine learning tools reach patients, developers are starting to get smart about the potential for bias to seep in. But a growing body of research aims to emphasize that even carefully trained models — ones built to ignore race — can breed inequity in care."

After Roe, concerns mount over A.I.-enabled surveillance - "Digital privacy groups fear the technology could make it easier to identify women who have sought abortions."

AI made these stunning images. Here's why experts are worried - "The bias in these AI systems presents a serious issue, experts told CNN Business. The technology can perpetuate hurtful biases and stereotypes."

Analysis

Is AI good or bad for the climate? It’s complicated - "A new framework for understanding and shaping the impacts of Artificial Intelligence on greenhouse gas emissions"

AI’s progress isn’t the same as creating human intelligence in machines - "The term “artificial intelligence” really has two meanings. AI refers both to the fundamental scientific quest to build human intelligence into computers and to the work of modeling massive amounts of data."

Policy

Waymo, UPS, others pressure Gov. Newsom to allow autonomous trucking in California - "Waymo, Aurora, UPS and Luminar are among a group of 34 autonomous vehicle developers, California business organizations, and automotive and logistics companies that signed an open letter to Governor Gavin Newsom on Monday asking him to revisit the California Department of Motor Vehicles’s 2015 prohibition on the operation of autonomous trucks in the state."

The UK wants to boost AI development by removing data mining hurdles - "The U.K. is planning to tweak an existing law to allow text and data mining “for any purpose,” in a move that’s designed to boost artificial intelligence (AI) development across the country. "

---

Copyright © 2022 Skynet Today, All rights reserved.