Last Week in AI #250: Microsoft's Phi-2, Mistral's Mixtral 8x7b, efficient sequence with Mamba, $24 Deepfakes disrupting Bangladesh's election, and more!
The small 2.7B Phi-2 model is surprisingly strong, Mistral releases MoE model, Mamba may replace transformers for better performance, cheap deepfakes ran amok in election campaigns
Top News
Phi-2: The surprising power of small language models
Microsoft's Machine Learning Foundations team has developed a new small language model (SLM) named Phi-2, which has demonstrated impressive reasoning and language understanding capabilities. Despite its smaller size of 2.7 billion parameters, Phi-2 has shown performance equivalent to or better than larger models on various benchmarks, even outperforming models up to 25 times larger. The Phi-2 model's success is attributed to the quality of training data and innovative scaling techniques. The model was trained using high-quality "textbook" data and web data selected for its educational value. Furthermore, knowledge from the previous 1.3 billion parameter model, Phi-1.5, was embedded into Phi-2, accelerating training and boosting performance. Phi-2 is now available in the Azure AI Studio model catalog, providing a platform for researchers to explore and improve language models.
Mixtral of experts
Mixtral 8x7b is a new LLM released by Mistral AI that surpasses GPT-3.5 in open-access AI. It utilizes a sparse Mixture of Expert (MoE) layer, replacing feed-forward layers, and has eight expert models within a single framework. The MoE allows for exceptional performance and enables models to be pretrained with less computational power. The model has a context length capacity of 32,000 tokens and performs well in various languages and coding tasks. Mixtral Instruct, a variant of the model, excels in industry standards and demonstrates flexibility in prompt formats. Mixtral is available for use on Mistral AI's platform and can be deployed using an open-source stack.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
The paper presents a new model called Mamba, which improves upon the current deep learning models, most of which are based on the Transformer architecture. The Transformer models, while powerful, have a computational inefficiency when dealing with long sequences. The Mamba model addresses this by improving the ability to perform content-based reasoning and selectively propagating or forgetting information depending on the current token. This allows for faster processing and better scaling with sequence length, even up to sequences of a million in length. The Mamba model also performs better across various applications such as language, audio, and genomics. In language modeling, the Mamba model even outperforms Transformer models of the same size and matches those twice its size. This paper has the potential to upend the foundation models and make them much faster and more efficient in the future.
Deepfakes for $24 a month: how AI is disrupting Bangladesh’s election
Ahead of Bangladesh's elections in January, AI-generated disinformation has become a significant issue. Pro-government news outlets and influencers have been promoting disinformation created with affordable tools provided by AI start-ups. Examples include a fake news clip criticizing the US and a deepfake video showing an opposition leader wavering on support for Gazans. This has led to increased pressure on tech platforms to regulate misleading AI content, especially ahead of major elections in 2024. However, controlling the use of these tools in smaller markets that may be overlooked by American tech companies presents a challenge. The situation in Bangladesh highlights the potential for AI tools to be exploited in elections and the difficulty in regulating their use.
Other News
Tools
Google’s most capable AI, Gemini, is now available for enterprise development - Google has announced that its powerful generative AI model, Gemini, is now available to enterprises for app development, with the Pro version accessible via API and free to use for now, but with certain usage limitations.
Introducing DeciLM-7B: The Fastest and Most Accurate 7 Billion-Parameter LLM to Date - Deci introduces DeciLM-7B, a fast and accurate 7 billion-parameter language model that surpasses its competitors in accuracy and throughput, making it ideal for applications such as customer service bots and data analysis.
Microsoft drastically expands Azure AI Studio to include Llama 2 Model-as-a-Service, GPT-4 Turbo with Vision - Microsoft has expanded Azure AI Studio to include the open-sourced AI model Llama 2 as a "model-as-a-service," offering customers more choices and a lower-cost option compared to OpenAI's GPT-3.5 and 4 models, while also making OpenAI's GPT-4 Turbo with Vision available to Azure customers.
Google unveils MedLM, a family of healthcare-focused generative AI models - Google has unveiled MedLM, a family of healthcare-focused generative AI models that can aid healthcare workers in completing their tasks, with two models available for complex tasks and scaling across tasks.
Duet AI for Developers, Google’s GitHub Copilot competitor, is now generally available and will soon use the Gemini model - Google has announced that Duet AI for Developers, its suite of AI-powered assistance tools for code completion and generation, is now generally available and will soon incorporate the more powerful Gemini model, while also partnering with 25 companies to provide datasets and documentation to assist developers in building and troubleshooting their applications.
Meta’s AI for Ray-Ban smart glasses can identify objects and translate languages - Meta is introducing multimodal AI features for its Ray-Ban smart glasses, allowing users to receive suggestions on clothing matches, translations, and image captions through the glasses' camera and microphones.
Google debuts Imagen 2 with text and logo generation - Google has released Imagen 2, an AI model that can generate and edit images based on text prompts, with improved image quality and the ability to render text and logos, but the company has not disclosed the data used to train the model or provided a way for creators to opt out or receive compensation for inadvertently contributing to the dataset.
WALT is a new AI video tool that creates photorealistic clips from a single image — you have to see it to believe it - A new AI model called WALT can convert a single image or text input into a photorealistic video with fluid 3D motion, although the quality of the output is not as high as other video models like Runway or Pika Labs.
Output’s AI-powered software automatically generates music sample packs from text prompts - Output has launched an AI tool called Pack Generator that automatically generates music sample packs based on text prompts, using pre-existing samples from the company's in-house library.
ANYmal’s Wheel-Hand-Leg-Arms Open Doors Playfully - The ANYmal quadruped, customized by Swiss-Mile, has been upgraded with powered wheels to make it faster and more efficient, while still maintaining its ability to handle curbs and stairs.
Snapchat now lets subscribers share AI-generated snaps - Snapchat now allows subscribers to create AI-generated images based on text prompts and send them to friends, along with other AI-powered features like adjusting the background of photos.
Meta unveils Audiobox, an AI that clones voices and generates ambient sounds - Meta has unveiled Audiobox, a voice cloning program that uses generative AI to replicate a person's vocal stylings, allowing users to create custom audio for a wide range of use cases, although the technology is currently restricted from commercial use and use in certain US states.
Google Pixel 8’s AI wallpapers appear to be coming to Samsung Galaxy in One UI 6.1 - Samsung's upcoming Android update, One UI 6.1, will feature generative AI wallpapers similar to those found on Google's Pixel 8 series, as leaked images suggest.
Spotify confirms test of prompt-based AI playlists feature - Spotify is testing a new feature that allows users to create playlists using AI technology and prompts, although the company has not provided further details or a launch timeframe.
Deadmau5-founded startup Korus taps into AI for music creation - Pixelynx, a metaverse company co-founded by Deadmau5, has announced new features for its AI-powered music creation platform Korus, including interactive visuals, a layering tool, video recording, and a rewards program to incentivize artistic contributions.
AI can now turn a rough sketch of a skyscraper into a detailed rendering in a matter of minutes. A leading architect demonstrates how - AI can turn a rough sketch of a skyscraper into a detailed rendering in minutes, allowing architects to quickly generate multiple options for clients.
Midjourney Alpha is here with AI image generations on the web - Midjourney, a popular image-generating AI model, has launched an alpha version of its website that allows users to generate imagery directly on the site instead of using Discord, with plans to make it available to more users in the future.
Open-Source LLM360 Unveiled by Cerebras Systems, Petuum and MBZUAI - Cerebras Systems, Petuum, and MBZUAI have unveiled LLM360, an open-source framework for creating large language models, with the release of two models, Amber and CrystalCoder, and plans for a third model, Diamond, in an effort to promote AI research and development in the UAE.
Instagram introduces gen-AI powered background editing tool - Instagram has introduced a generative AI-powered background editing tool that allows users to change the background of their images through prompts for Stories.
Salesforce strengthens AI play with vector database support, enhanced Einstein Copilot - Salesforce is strengthening its AI offering by adding vector database support and enhancing its Einstein Copilot generative assistant with AI search capabilities, making it easier for teams to take advantage of AI in their workflows.
H&R Block launches AI tax filing assistant - H&R Block has launched AI Tax Assist, a conversational AI chatbot that answers taxpayer questions and provides information on tax rules, exemptions, and other tax-related issues, with the option to consult a human tax expert for personalized advice.
Agility is using large language models to communicate with its humanoid robots - Agility is using large language models to revolutionize the way its humanoid robot, Digit, communicates, learns, looks, and is programmed, showcasing the potential of natural language commands and the future of robotics.
AI-generated news anchors to be part of new national news channel premiering next year - An upcoming news station plans to use AI-generated news anchors alongside human anchors, aiming to provide a more personalized news experience for viewers.
Mozilla Planning for MemoryCache Local AI Bot in Firefox - Mozilla is developing MemoryCache, an experimental project that integrates a conversational AI system directly into the Firefox browser, providing users with a personalized and offline-accessible AI companion that adapts to their interests and needs.
Business
Partnership with Axel Springer to deepen beneficial use of AI in journalism - Axel Springer partners with OpenAI to integrate journalism into AI technologies, enriching users' experience with ChatGPT by providing authoritative news content and supporting a sustainable future for journalism.
Tesla unveils its latest humanoid robot, Optimus Gen 2, in demo video - Tesla has released a demo video showcasing its latest humanoid robot, Optimus Gen 2, which features improved hardware and capabilities such as walking, manipulating objects, and delicate object manipulation.
Essential AI emerges from stealth with backing from Google, Nvidia and AMD - San Francisco-based startup Essential AI has emerged from stealth mode with $56.5 million in funding from investors including Google, Nvidia, and AMD, and plans to develop and launch large language model-driven AI products that automate time-consuming workflows and increase productivity.
Intel unveils new AI chip to compete with Nvidia and AMD - Intel has unveiled Gaudi3, an AI chip that will compete with Nvidia and AMD chips in powering big and power-hungry AI models, aiming to attract AI companies away from Nvidia's dominant position in the market.
Cruise slashes 24% of self-driving car workforce in sweeping layoffs - Cruise, the self-driving car subsidiary of GM, is laying off 900 employees, or about 24% of its workforce, in an effort to cut costs and revamp the company following a recent incident involving one of its robotaxis.
Sports Illustrated fires its CEO, who becomes the fourth executive to leave publisher amid fallout from AI-generated articles - Sports Illustrated fires its CEO and three other executives amid fallout from AI-generated articles, with accusations that the company had been publishing stories written by AI.
CitrusX raises $4.5 million Seed for AI explainability collaboration platform - Israeli startup CitrusX has raised $4.5 million in Seed funding for its AI validation and explainability platform, which aims to address the challenges of model development, validation, explainability, risk assessment, and legal approval in the adoption of AI.
Research
OpenAI Demos a Control Method for Superintelligent AI - OpenAI demonstrates a control method for superintelligent AI, raising the possibility of humans creating AI systems that surpass us intellectually.
DeepMind AI outdoes human mathematicians on unsolved problem - An AI system called FunSearch, based on large language models, has shown that it can help mathematicians generate new solutions to problems inspired by the card game Set, going beyond what was previously known by mathematicians and computer scientists.
Photorealistic Video Generation with Diffusion Models - The article discusses the generation of photorealistic videos using diffusion models.
DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing - DiffMorpher is a new approach that enables smooth and natural image interpolation using diffusion models, addressing the limitation of these models in smoothly interpolating between two image samples and achieving better image morphing effects than previous methods.
VILA: On Pre-training for Visual Language Models - The article discusses the design options for pre-training visual language models (VLMs) and introduces three main findings, including the benefits of freezing and unfreezing language models during pre-training, the importance of interleaved pre-training data, and the advantages of re-blending text-only instruction data with image-text data.
HoneyBee: Intel Labs and Mila Collaborate on State-of-the-Art Language Model for Materials Science - Intel Labs and Mila have collaborated on HoneyBee, a state-of-the-art language model specialized for materials science, achieving state-of-the-art performance on the MatSci-NLP benchmark.
PathFinder: Guided Search over Multi-Step Reasoning Paths - A new AI system called PathFinder allows for guided search over multi-step reasoning paths, providing a more efficient and effective way to find information.
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention - Accelerating Transformers with Mixture-of-Experts Attention explores the use of SwitchHead to enhance the performance of AI models by incorporating a mixture-of-experts attention mechanism.
Foundation Models in Robotics: Applications, Challenges, and the Future - Foundation models pretrained on internet-scale data have the potential to enhance various components of the robot autonomy stack, but there are still challenges to overcome in terms of scarcity of training data, safety guarantees, and real-time execution.
GenTron: Delving Deep into Diffusion Transformers for Image and Video Generation - GenTron explores the use of diffusion transformers for generating images and videos, allowing users to upload media through various methods.
Introducing Stable Zero123: Quality 3D Object Generation from Single Images - Stable Zero123, a new in-house trained model for view-conditioned image generation, is released for research purposes and not intended for commercial use.
HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image - HyperDreamer is a tool that generates and edits hyper-realistic 3D content from a single image.
StemGen: A music generation model that listens - StemGen is a music generation model that can listen and create music based on the input it receives.
CogAgent: A Visual Language Model for GUI Agents - A visual language model called CogAgent is introduced for GUI agents, allowing for easy upload of images, audio, and videos.
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis - The article discusses the goal of building general-purpose robots and explores how existing foundation models from NLP and CV can be applied to the field of robotics, as well as what a robotics-specific foundation model would look like.
Deep neural networks show promise as models of human hearing - Deep neural networks that mimic the human auditory system have the potential to improve hearing aids, cochlear implants, and brain-machine interfaces, according to a study from MIT that found these models generate internal representations similar to those seen in the human brain when listening to sounds.
Alignment for Honesty - The article discusses the importance of alignment for honesty in large language models (LLMs) and proposes solutions for measuring and improving the honesty of LLMs through metric development, benchmark creation, and training methodologies.
Cheating Fears Over Chatbots Were Overblown, New Research Suggests - Research suggests that fears of mass cheating among high school and college students using AI chatbots were overblown, with a survey finding that 12 to 28 percent of high school students had used an AI tool as an unauthorized aid during tests or assignments, prompting researchers to shift the focus to helping students understand and critically engage with AI tools.
Advancements in machine learning for machine learning - Advancements in machine learning for machine learning include using ML to improve the efficiency of ML workloads, releasing a dataset called TpuGraphs for learning cost models for programs running on TPUs, and introducing a method called Graph Segment Training for scaling GNN training to handle large graphs.
New Mind-Reading "BrainGPT" Turns Thoughts Into Text On Screen - Researchers at the University of Technology Sydney have developed a breakthrough technology called BrainGPT that can translate thoughts into words on a screen using only brainwaves as input, without the need for brain implants or an MRI machine.
Concerns
News publisher files class action antitrust suit against Google, citing AI’s harms to their bottom line - A class action lawsuit has been filed against Google and parent company Alphabet, accusing them of anticompetitive behavior and harming news publishers' bottom line through the use of AI technologies like Google's Search Generative Experience and Bard AI chatbot.
Tesla recalls 2 million cars with ‘insufficient’ Autopilot safety controls - Tesla is recalling over 2 million vehicles equipped with Autopilot systems due to "insufficient" safeguards against driver misuse, following an investigation that identified several fatal or serious crashes involving Tesla drivers using Autopilot on roads where the software was not intended to be used.
Meta used copyrighted books for AI training despite its own lawyers' warnings, authors allege - Meta Platforms allegedly used copyrighted books to train its AI models despite warnings from its own lawyers, according to a new filing in a copyright infringement lawsuit.
OpenAI suspends ByteDance’s account after it used GPT to train its own AI model. - OpenAI suspends ByteDance's account for violating developer license by using GPT-generated data to train its own AI model in China.
Big Tech's LLM evals are just marketing - Big Tech companies like Microsoft and Google are engaging in misleading marketing tactics by comparing the evaluation scores of their AI models without the ability to evaluate their competitors, leading to inflated claims and a lack of transparency in the field of AI.
A financial news site uses AI to copy competitors — wholesale - Investing.com is using AI to rewrite articles from competitors, causing concern among competitors about the threat to journalism and original content creation.
AI-generated Nazi memes thrive on Musk’s X despite claims of crackdown - AI-generated hate memes, including antisemitic and racist content, are thriving on Elon Musk's social media platform X, despite claims of a crackdown on such material.
Hackers behind recent ChatGPT outage say they'll target the AI bot until it stops 'dehumanizing' Palestinians - Anonymous Sudan claims responsibility for recent ChatGPT outages and says it will continue until the AI bot stops "dehumanizing" Palestinians.
ChatGPT users complain the AI is getting lazy and sassy - Users of OpenAI's ChatGPT, built on the GPT-4 model, have complained that the chatbot has become "lazy" and unhelpful, prompting OpenAI to investigate the issue.
Civitai and OctoML Introduce Radical New Measures to Stop Abuse After 404 Media Investigation - Civitai, a text-to-image AI model sharing platform, is seeking a new cloud computing provider and instructing its users to complain to its current provider, OctoML, after OctoML introduced a content filter that is stopping Civitai users from generating sexually explicit images.
Adobe Signals That AI Boost Will Take Longer Than Expected - Adobe acknowledges that the progress of AI will be slower than anticipated.
Policy
San Francisco Expands Curbs on Robotaxi Deliveries - San Francisco has passed a new law that prohibits the use of charging stations for electric vehicle fleets for package deliveries, reflecting concerns over traffic safety, congestion, and job loss associated with the expansion of autonomous vehicles.
The US has a new plan for wielding AI to fight climate change - The US Department of Energy is creating an office to coordinate the use of AI in fighting climate change, with priorities including developing nuclear fusion power, increasing energy efficiency of supercomputers, testing AI models for vulnerabilities, and making data more accessible.
Judges Given the OK to Use ChatGPT in Legal Rulings - Judges in the UK have been given permission to use ChatGPT and other AI tools to write legal rulings, despite acknowledging the potential pitfalls and limitations of the technology.
Analysis
Sam Altman on OpenAI, Future Risks and Rewards, and Artificial General Intelligence - Sam Altman, CEO of OpenAI, discusses his ousting and reinstatement at OpenAI, the potential risks and rewards of AI, and the democratization of artificial general intelligence.
Explainers
Explaining ChatGPT to Anyone in <20 Minutes - This article provides an overview of the key components of generative large language models (LLMs), including the transformer architecture, language model pretraining, and the alignment process, and emphasizes the importance of effectively communicating about AI technology.
Copyright © 2023 Skynet Today, All rights reserved.