Last Week in AI #211: GPT-4 is here, more LLMs released, Microsoft and Google integrate AI in products

OpenAI announces GPT-4 which is much better than ChatGPT, Stanford trains small but powerful LLM based on data generated by GPT3.5, AI product announcements by Microsoft and Google

Top News

GPT-4

Language models have leveled up in many recently. OpenAI released the much-anticipated GPT-4 on Pi Day. The new system not only demonstrates impressive text-generation capabilities and a version with a 32,000-token context window but is also multimodal. In a demo on the release day, OpenAI President Greg Brockman asked GPT-4 to write code for a simple webpage design drawn on a napkin—the model responded with working code for that webpage. GPT-4 also vastly outperforms ChatGPT on various standardized tests, achieving 90th percentile on the Bar exam. GPT-4 is available today via OpenAI API, and it’s also confirmed that it’s the model behind the new Bing Chat.

Our Take: We're seeing what happens as we go further along the scaling curve—it's impressive. After GPT-3's release, our Daniel Bashir and Sathvik Nair wrote that it was not coming for your job; I'm willing to say the same of GPT-4, but it’ll certainly make a lot of jobs a lot easier. While GPT-4 still has some of the limitations fundamental to LLMs, it's not the holes we can poke in GPT-4's capabilities that matter. It's what it can do and what it portends. As we adopt these new technologies and figure out how to integrate them into our work, we may be able to achieve a lot more. The future is much less clear.

Alpaca: A Strong, Replicable Instruction-Following Model

As OpenAI scaled up GPT, Stanford sought to make instruction-following research accessible to the academic community with the 7-billion parameter model Alpaca, based on Meta's recently-released LLaMA. Training Alpaca is very affordable - the model was fine-tuned for under $600 and can run on a single GPU or CPU. The key that made Alpaca work was training it on a dataset of generated tasks that were by produced GPT-3.5, effectively making the model a “distilled” version of GPT-3.5.

Our take: I'm equally excited about Alpaca. Research on large language models is not accessible to most, but powerful language models that can run on consumer hardware seem to be in the near future. The approach of using larger and more powerful language models to generate training data for smaller ones seems bound to accelerate progress in these systems significantly. This also points to an interesting phenomenon where it may really be difficult to build a moat around a language model if sufficient API access is granted - others can just query said language model and train another model to imitate it.

Microsoft and Google Unveil A.I. Tools for Businesses

Both Google and Microsoft announced product releases this week that directly integrate LLM technology into their existing products, such as Microsoft Teams and Office, Gmail and Google Workspace. These moves make sense for two reasons: 1) generative language AI is truly useful in these productivity apps (e.g. summarizing/writing emails/briefs, taking notes during video meetings, generating slideshows, etc) and 2) it’s a lot easier to make money from apps that businesses are willing to pay (and are already paying) for instead of open-ended chatbots in search engines.

Our take: First, this should really highlight how easy it is for tech incumbents, who have in-house AI teams, to integrate AI features into their existing products. In other words, the AI capabilities found in productivity software are features, not companies, and startups that were thin wrappers on top of existing language models will probably lose out on both unit economics as well as software integration. The second point is Google probably could’ve released something like this a long time ago, given how quickly these features were deployed after ChatGPT went viral last December (tech features at large companies are notoriously slow to ship - it’s hard to believe they can pull this off in 3 months if they’re starting from scratch). They’re only doing so now with urgency because of Microsoft. Big tech’s race to produce AI products at scale is now on. I wonder who else will join the race in the coming months (maybe even weeks!)

Other News

Research

GPT-4 is bigger and better than ChatGPT—but OpenAI won’t say why - "We got a first look at the much-anticipated big new language model from OpenAI. But this time how it works is even more deeply under wraps."

MIT researchers uncover the structural properties and dynamics of deep classifiers, offering novel explanations for optimization, generalization, and approximation in deep networks - "Researchers from MIT and Brown University have conducted a groundbreaking study on the dynamics of training deep classifiers, a widespread neural network used for tasks like image classification, speech recognition, and natural language processing."

Highly dynamic bistable soft actuators allow for varied locomotion - "Most animals can quickly transition from walking to jumping to crawling to swimming if needed without reconfiguring or making major adjustments. Most robots cannot."

Machine learning takes starring role in exploring the universe - "Penn State astronomers -- and Institute for Computational and Data Sciences co-hires -- are using machine learning algorithms to examine the treasure trove of data from the James Webb Space Telescope. They say that this information is already changing our understanding of the universe."

Microsoft Open-Sources Weather Forecasting Deep Learning Model ClimaX - "Researchers from Microsoft's Autonomous Systems and Robotics Research group have open-sourced ClimaX, a deep learning foundation model for weather and climate modeling."

A new method to boost the speed of online databases - "Hashing is a core operation in most online databases, like a library catalogue or an e-commerce website. A hash function generates codes that replace data inputs."

Applications

How Computer Vision Is Changing Manufacturing in 2023 - "Industrial components with bounding boxes. Image from MVTech Industrial 3D Object Detection Dataset, courtesy of MVTech. Welcome to the second installment of Voxel51’s computer vision industry spotlight blog series."

Using AI and 3D stereo cameras to support fisheries - "The Alaska Fisheries Science Center is using drones, remotely operated vehicles and 3D stereo cameras with Artificial Intelligence to collect information that helps to support sustainable fisheries."

ChatGPT and Generative AI are booming, but at a very expensive price - "Before OpenAI's ChatGPT emerged and captured the world's attention for its ability to create compelling sentences, a small startup called Latitude was wowing consumers with its AI Dungeon game that let them create fantastical tales based on their prompts."

Microsoft’s new share button makes it easy to show people what Bing AI is saying - "Microsoft is trying to make it easier to share your experiences with its GPT-4-powered Bing Chat by adding a button that lets you post the AI’s response to Facebook, Twitter, or Pinterest."

GM plans to let you talk to your car with ChatGPT, Knight Rider-style - "In the 1982 TV series Knight Rider, the main character can have a full conversation with his futuristic car."

GPT-4’s new capabilities power a ‘virtual volunteer’ for the visually impaired - "OpenAI has introduced the world to its latest powerful AI model, GPT-4, and refreshingly the first thing they partnered up on with its new capabilities is helping people with visual impairments."

PyTorch 2.0 brings new fire to open-source machine learning - "After months in preview, PyTorch 2.0 has been made generally available by the PyTorch Foundation. "

Google flexes its health care AI muscle - "A Google chatbot now consistently passes medical exam questions with a score of 85%."

UPMC Explores Machine Learning’s Potential to Personalize HIV Testing - "A UPMC expert outlines how machine-learning software may help improve laboratory workflows and enhance care for those undergoing HIV testing."

Can AI and Machine Learning Help Park Rangers Prevent Poaching? - "Globally there are too few park rangers to prevent the illegal trade of wildlife across borders, or poaching."

Generative AI: Unlocking the future of fashion - "As this season’s fashion weeks wrap up in London, Milan, New York, and Paris, brands are working to produce and sell the designs they’ve just showcased on runways—and they’re starting next season’s collections."

Business

funding plummets as investors favor generative AI - "AI has replaced metaverse as media and tech's new buzzword."

Introducing Claude - “Claude is a next-generation AI assistant based on Anthropic’s research into training helpful, honest, and harmless AI systems.”

Autonomous Trucking Upstart Embark Goes From $5B Valuation To Kaput In 16 Months - "Autonomous trucking startup Embark Trucks capped off one of the faster riches-to-rags stories of the SPAC era, announcing that it is laying off most employees and winding down operations."

Microsoft Strung Together Tens of Thousands of Chips in a Pricey Supercomputer for OpenAI - "When Microsoft Corp. invested $1 billion in OpenAI in 2019, it agreed to build a massive, cutting-edge supercomputer for the artificial intelligence research startup."

NextGen Food Robotics Announces Development of AI-Powered App - "Lilly’s core feature will be an AI-powered natural language processing chat interface that will suggest food suggestions based on the user’s preferences communicated by voice."

Lowe's is testing 400-pound, egg-shaped autonomous robots to patrol parking lots at some of its stores - "Lowe's is testing autonomous, 400-pound security robots from robotics company Knightscope to patrol parking lots at some of its stores to enhance safety, the company told Insider Monday."

Morgan Stanley is testing an OpenAI-powered chatbot for its 16,000 financial advisors - "Morgan Stanley is rolling out an advanced chatbot powered by OpenAI's latest technology to help the bank's army of financial advisors, CNBC has learned."

Adept Raises $350 Million To Build AI That Learns How To Use Software For You - "With new funding that values it at $1 billion, the year-old company already counts Microsoft, Nvidia and Workday as strategic partners."

Bing said to remove waitlist for its GPT-4 powered chat - "Microsoft’s Bing is enjoying the spotlight for the first time in a decade after it released a GPT-powered interface last month. But the tech giant has so far been cautious about the pace at which it is making the new Bing offering — powered by  OpenAI’s GPT-4 tech — available to users."

Microsoft’s Bing reaches 100 million daily active users - "Microsoft Corp. has disclosed that its Bing search engine is now used by 100 million consumers every day. Yusuf Mehdi, the corporate vice president of Microsoft’s modern life, search and devices group, announced the milestone in a Wednesday blog post."

‘Let 1,000 Flowers Bloom’: A.I. Funding Frenzy Escalates - "When four leading artificial intelligence researchers left Google this year to create a start-up called Mobius AI, they weren’t sure what their product might be — just that it would involve A.I. technology that could generate its own photos and videos."

Concerns

I’m an ER doctor: Here’s what I found when I asked ChatGPT to diagnose my patients - "ChatGPT recently passed the U.S. Medical Licensing Exam, but using it for a real-world medical diagnosis would quickly turn deadly. "

Hyper-realistic beauty filters are here to stay - "A new filter on TikTok has the internet up in arms. It's an important debate for anyone who cares about the future of social media."

Microsoft just laid off one of its responsible AI teams - "As the company accelerates its push into AI products, the ethics and society team is gone"

An A.I. Start-Up Boomed, but Now It Faces a Slowing Economy and New Rules - "Using A.I. tools in hiring is one of the most promising and contentious uses of the technology. The experience of a company called Eightfold AI shows the challenges."

Illustration competitions grapple with generative AI - "AI's unique legal and ethical considerations have prompted some organizations to take a definitive public stance."

Online storm erupts over AI work in Dutch museum’s ‘Girl with a Pearl Earring’ display - "The Mauritshuis museum in The Hague, Netherlands, is facing criticism for showing an image made using artificial intelligence (AI) which is inspired by Vermeer’s famous Girl with a Pearl Earring."

A Bunch of Top Music Advocates Want to Ensure AI Doesn’t Replace Your Favorite Artist - "Artificial intelligence is shaping up to be the largest disruptor to the music industry since digital downloads, and it poses one of the most important existential questions to music creation the art form itself has ever faced."

The World Must Reconcile AI Skepticism and AI Optimism - "Controversial topics are weird. People spontaneously gather around black vs white dichotomies: if you’re not with me, you’re against me. That's happening in AI. You may have noticed that, as its popularity grows, the the public opinion has become extremely polarized."

AI Injected Misinformation Into Article Claiming Misinformation in 'Navalny' Doc - "Investigative news outlet The Grayzone recently published an article that included AI-generated text as a source for its information."

The stupidity of AI - "Artificial intelligence in its current form is based on the wholesale appropriation of existing culture, and the notion that it is actually intelligent could be actively dangerous"

Not magic: Opaque AI tool may flag parents with disabilities - "For the two weeks that the Hackneys’ baby girl lay in a Pittsburgh hospital bed weak from dehydration, her parents rarely left her side, sometimes sleeping on the fold-out sofa in the room."

AI Scams and Influencers - "As some of you may know, GPT 4 has been released. This will no doubt add a lot of noise and hype to the field, with more AI and Productivity Gurus crawling out of the woodwork to sell their courses, groups, and products."

OpenAI Seems Like a Very Sleazy Company to Be Creating World-Changing AGI - "Let's just say, as a hypothetical, that someone does build Artificial General Intelligence (AGI) — human-level AI that, if realized, would undoubtedly change everything."

Analysis

How Siri, Alexa and Google Assistant Lost the A.I. Race - "On a rainy Tuesday in San Francisco, Apple executives took the stage in a crowded auditorium to unveil the fifth-generation iPhone. The phone, which looked identical to the previous version, had a new feature that the audience was soon buzzing about: Siri, a virtual assistant."

Policy

AI and Copyright: Human Artistry Campaign Launches to Support Songwriters and Musicians’ Rights - "The fast rise of AI technology has opened up a world of brain-busting questions about copyright and creators’ rights"

AI Generated Image from Text is not Human Authorship says US Copyright Office - "The U.S. Copyright Office has warned that an image generated solely from a text prompt does not qualify for human authorship in fresh guidance released yesterday. The Copyright Office likened text prompts to “instructions to a commissioned artist."

---

Copyright © 2023 Skynet Today, All rights reserved.