Last Week in AI #153: OpenAI's InstructGPT is less-toxic than GPT-3, Meta to build fastest AI supercomputer, Deepfake regulations across the world, and more!
Last Week in AI #153: OpenAI's InstructGPT is less-toxic than GPT-3, Meta to build fastest AI supercomputer, Deepfake regulations across the world, and more!
OpenAI's InstructGPT uses RL to improve responses, Meta to build the fastest AI supercomputer with 16k GPUs, new Deepfake regulations in the EU, UK, US, and China
Top News
OpenAI rolls out new text-generating models that it claims are less toxic
OpenAI’s GPT-3 has marked a major turning point in AI research since its announcement in 2020, with its impressive performance on a multitude of tasks it was not trained for being a huge revelation. While significant, this advance did come with a caveat; GPT-3 and similar models trained on massive amounts of uncurated data from the internet can exhibit misinformation, sexism, ageism, racism, and conspiracies. OpenAI’s new paper Training language models to follow instructions with human feedback presents an approach for avoiding these issues, by taking a model such as GPT-3 and further training it with humans providing feedback for both desirable and undesirable outputs for any given task. The resulting model, InstructGPT, is more truthful and less prone to creating toxic outputs while having essentially comparable performance to GPT-3 on standard benchmarks. InstructGPT models are now deployed as the default in OpenAI’s API.
Our take: This is very cool! While not especially novel in terms of its technical approach, the demonstration of this technique working on a model as large as GPT-3 is quite significant and likely to influence similar future work. A lot of work has demonstrated the potential negative ramifications of using models such as GPT-3, so it’s also significant that OpenAI itself undertook this research and has even replaced GPT-3 with InstructGPT in its commercial offerings.
Meta Aims to Build the World's Fastest AI Supercomputer
On Monday, January 24, Meta introduced the AI Research SuperCluster (RSC). The result of nearly two years of work, the supercomputer has 6,080 Nvidia GPUs. Now ranked fifth, Meta claims RSC will become the fastest supercomputer in the world once it is fully built—at that stage, it will house around 16,000 GPUs. According to Meta, the supercomputer will primarily be used for research and products will not come from it for years. Its primary purpose is to use Meta’s vast troves of data to develop more capable AI systems that can learn from a variety of modalities and understand situations contextually.
Our take: Recent work from Meta like data2vec hints at the possible directions their research might go using the RSC. With this level of computing power, we can expect to see a lot of interesting research from Meta in the coming years. At the same time, this development does follow the trend of imbalance between the amount of compute (and data) available to researchers in industry vs elsewhere, such as academia.
Deepfake Regulations in EU, UK, and US and China
Last week, the European Parliament ratified new regulations on Deepfakes that were “surprisingly limited” - it just requires large online platforms (e.g. Facebook, YouTube) to label known deepfake videos as such. It does not ban any type of deepfakes or require platforms to actively identify them. UK’s new draft bill seems also similarly lax, and critics complain of “unclear and impractical” consequences for harmful deepfake applications. The US has more stringent regulations, but they happen at the state level, not the federal. Texas banned political deepfakes in 2019, without mentioning anything about deepfake pornography. Virginia and New York, banned unlawful dissemination or sale of deepfakes, while California and Maryland banned pornographic deepfakes.
China’s recent deepfake regulation proposal is much more detailed, and it covers all forms of AI-assisted generative content: text, images, audio, faces, voice, video, and even virtual scenes. Interestingly, China seems mostly focused on audio deepfakes in the role of spreading fake news. The regulation requires deepfake providers to register their applications with the government, clear labels of deepfakes and portals for user complaints, and platforms that host deepfakes to bear liability for algorithm and content management. See more details on this Twitter thread. The proposal is a draft and not yet made into law.
Our take: The comparison among these countries’ approaches to deepfake regulation reflects their different political systems and attitude toward regulating emerging technologies. The western nations are more on the “wait and see” side, avoiding overregulation and probably waiting for a “DeepfakeGate” to summon the necessary political will for more comprehensive legislation. China is a lot more proactive and maybe responding to what it sees as a threat to the government’s control of online information.
Check out our latest editorials!
Other News
Research
Researchers Build AI That Builds AI - "By using hypernetworks, researchers can now preemptively fine-tune artificial neural networks, saving some of the time and expense of training. Artificial intelligence is largely a numbers game."
Here's how scientists are using machine learning to listen to fish - "In addition to the purrs and clicks that Munger likes, Bell said in her research she's heard fish -- she's still not sure which species -- make mooing sounds, and other mysterious sounds that remind her of tubas and Jet Skis."
Legged Robots Learn to Hike Harsh Terrain - "ANYmal demonstrates locomotion performance that’s slightly superhuman"
The rise and fall (and rise) of datasets - "Growing criticisms of datasets that were built from user-generated data scraped from the web have led to the retirement or redaction of many popular benchmarks. Their afterlife, as copies or subsets that continue to be used, is a cause for concern."
AI2 shows off an open, Q&A-focused rival to GPT3 - "The Allen Institute for AI (AI2) has demonstrated a model that performs as well or better than GPT-3 on answering questions, but is a tenth the size."
Robot performs surgery without help from humans - "The machine successfully performed keyhole surgery on a pig, attaching organs in a range of different animals. And it did so without the help of a human, for the first time."
DARPA’s RACER Program Sends High-Speed, Autonomous Vehicles Off-Road - "For the next three years, robotic vehicles will be pushing the limits of all-terrain racing"
Intel, Dell Launch Initiative to Build AI Labs in Community Colleges - "Intel, Dell Technologies and the American Association of Community Colleges (AACC) have announced the launch of the new AI Incubator Network, which is an initiative aimed at designing and building artificial intelligence (AI) labs throughout the U.S."
AI Chatbots Pose Ethical Risks. Here’s How One University Is Handling Those. - "Visit the University of California at Irvine’s admissions page, and a box pops up in the bottom right corner. Click on it, and there’s Peter the Anteater, a chatbot affectionately named after the university’s mascot, clad in a varsity jacket and grinning."
Google AI Propose An Machine Learning (ML) Based Audio Separation Approach That Can Identify Birdsongs For Better Species Classification - "Google AI presented a novel unsupervised method termed mixture invariant training (MixIT) to address the fundamental difficulty of training ML models to automatically separate audio recordings without access to examples of separated sounds. "
Machine-learning model shows diamond melting at high pressure - "A Sandia National Laboratories supercomputer simulation model called SNAP that rapidly predicts the behavior of billions of interacting atoms has captured the melting of diamond when compressed by extreme pressures and temperatures"
Applications
Google AI tools bring back women in science to the fore - "However, they have often not received proper credit or acknowledgement for their essential work."
How AI, machine learning are being increasingly used to drive social good - "Data and technology solutions are key to reaching the most vulnerable children at speed"
Deploying machine learning to improve mental health - "MIT scientist Rosalind Picard collaborates with clinicians to develop tools for mental health care delivery."
Using machine learning to find space rocks in Antarctica - "To apply an AI system to the task, the researchers trained it with satellite data covering the entire continent, along with data that showed where blue ice fields containing meteorites have been found. "
Watch an AI Play the Best Game of Tetris You've Ever Seen - "The AI easily shatters previous world records by prioritizing clearing four-line tetrises. Then the game completely melts down."
Using AI, ML Will Help the Government Tackle Climate Change, Experts Say - "the use of AI has been rather limited until now, focusing most often on some single dimension of climate risk."
Business
Boston Dynamics’ warehouse robot gets a $15M gig working for DHL - "Today the Hyundai-owned firm announced its first commercial customer — and it’s a big one."
Fathom raises $3.7M for its AI notetaker - "Fathom, a startup that is building an AI notetaker for Zoom, today announced that it has raised a $3.7 million seed round from a range of early-stage investors, including Zoom’s own Apps Fund."
SenseTime launches powerful AI computing center in Shanghai - "The first mega artificial intelligence (AI) computing center in East China operated by Chinese emerging AI firm SenseTime officially started operation in Shanghai on Monday, which is set to become one of the largest AI computing centers in Asia."
Metaphysic, AI startup behind Tom Cruise deepfakes, raises $7.5M - "The funding will help expand Metaphysic’s work on synthetic content creation tools for emergent metaverse worlds that are being built by Facebook and other networks."
The self-driving car industry is abandoning the term ‘self-driving’ and leaving it to Tesla - "Meet the Autonomous Vehicle Industry Association"
Fidelity Has Big Ambitions for Its Machine-Learning Compliance Offering - "Fidelity Investments has unveiled a new technology that aims to automate much of the compliance work that goes into producing marketing materials, a major pain point for many advisory firms that operate in a highly regulated industry."
Shield AI wins Air Force contract intended to help bridge 'valley of death' - "Shield AI, the defense-focused autonomous drone startup, won a possibly $60 million contact from the Air Force designed to keep cash flowing to companies while they await opportunities to get on major contracts."
Concerns
IRS Will Require Facial Recognition Scans to Access Your Taxes - "You will have to submit sensitive government documents, your Social Security number, credit history, and a face scan to ID.me, a third-party company."
No, large language models aren’t like people with disabilities (and it’s problematic to argue that… - "There’s a tendency I’ve observed where people trying to argue that language models “understand” language to draw analogies to the experience of disabled people (especially blind and Deafblind people). These analogies are both false and dehumanizing."
A social media app just for 'females' intentionally excludes trans women — and some say its face-recognition AI discriminates against women of color, too - "Trans women are banned, and some reviews say the app misidentifies women of color as men."
Analysis
How Claude Shannon Helped Kick-start Machine Learning - "Conventional cameras in stereo mode can indeed detect objects, gauge their distance, and estimate their speeds, but they don’t have the accuracy required for fully autonomous driving."
Conversational AI Systems for Social Good: Opportunities and Challenges - "Overview: Conversational artificial intelligence, or ConvAI, has the potential to advance the United Nations’ Sustainable Development Goals (SDGs) for social good due to the technology’s current deployment in various industries and populations."
Our children are growing up with AI. Here's what you need to know - "Decision-makers and tech innovators must prioritize children's rights and wellbeing when designing and developing AI systems. Efforts should be undertaken to create AI curricula, and build AI literacy and skills of future generations."
Policy
New DOD Chief Digital and AI Office to start work by Feb. 1 - "The Department of Defense’s new top artificial intelligence office will reach initial operational capability February 1, the Navy’s chief data officer said Wednesday."
Expert Opinions
Will Robots Really Destroy the Future of Work? - "Some people on the political right love robots and hate labor unions. Some people on the political left are the opposite: They hate robots and love labor unions. Then there’s David Autor, a labor economist at the Massachusetts Institute of Technology. He loves both."
Copyright © 2022 Skynet Today, All rights reserved.