Last Week in AI #91

AI's replication crisis, government-sponsored medical AI, and more!

Nov 16, 2020

mri scan of brain — Image credit: Tom Simonite / Getty Images via Wired

Mini Briefs

AI is wrestling with a replication crisis

It's well-known that AI research is plagued by problems of transparency and reproducibility, but the issue has received particular attention recently. For example, a Google Health paper in Nature involving breast cancer detection drew ire because it included so little information about code and testing that "the study amounted to nothing more than a promotion of proprietary tech." While reproducibility is considered a bedrock of scientific research, few AI studies are replicated because of the field's ambition to move forward and produce new studies. In addition to academic incentives and a lack of transparency, differential access to resources and money required to conduct experiments put the ability to produce state-of-the-art research in the hands of large companies.

But there's hope for mitigating some of these issues: reproducibility challenges, in which participants reproduce AI papers, have drawn many participants, while more and more researchers submit code with their papers at top conferences. While publication norms are being fought for by many researchers, the long-term impact of such practices depends on the direction AI takes--if large models like GPT-3 stay trendy, many will be unable to participate in cutting-edge research. There is no question that industry research labs will continue to dominate AI research, but concerned researchers maintain that democratizing participation will be key.

The US Government Will Pay Doctors to Use These AI Algorithms

Medical AI research is continuing to take off, and beyond convincing US government actors like the FDA of its effectiveness in improving outcomes, we're seeing the government begin to use taxpayer dollars to fund the adoption of medical AI:

The US Centers for Medicare & Medicaid Services (CMS) recently said it would pay for use of two AI systems: one that can diagnose a complication of diabetes that causes blindness, and another that alerts a specialist when a brain scan suggests a patient has suffered a stroke.

But the implications are wider than for just Medicare and Medicaid patients: such investment could drive increased adoption of AI in healthcare. This new offer isn't the first: CMS began paying for the use of San Francisco startup Viz.ai's AI software, ContaCT, which alerts a neurosurgeon when algorithms see evidence in a CT scan that a patient has a blood clot in their brain. Speeding up diagnosis and treatment can reduce a patient's disabilities and recovery time. This government willingness to pay for the use of AI tools may not only increase adoption, but also encourage other companies working on medical AI products. While some technologies might not yet be economically feasible, payments like the ones CMS has begun to make could help such technologies propagate much faster.

Podcast

Check out our weekly podcast covering these stories! Website | RSS | iTunes | Spotify | YouTube

News

Advances & Business

Two Students Develop Software to Combat CO2 Caused by AI - "Experts agree that the current path of artificial intelligence (AI) development will cause it to become one of the leading contributors of CO2, despite its use to combat that very problem."
Blizzard says Overwatch and World of Warcraft toxicity has dropped thanks to machine learning - "Blizzard's giving fans an update on its year so far, and one of the company's proudest achievements is using new tech to cut down on in-game toxicity. Blizzard president J."
Amazon's new Alexa feature uses AI to infer what users really want - "Amazon today announced a new Alexa feature for U.S.-based English-language users that enables devices powered by the assistant to infer latent goals, or goals implicit in requests but not directly expressed."
MIT, UChicago, Harvard, Diffeo Researchers Use Bayesian Inference to Cook Up Multi-Agent Collaboration - "'Figuring out how to flexibly coordinate a collaborative endeavour is a fundamental challenge for any agent in a multi-agent world,' explains a team of researchers from MIT, UChicago, Harvard and Diffeo."
Researchers claim AI algorithm uses heart rate and motion data to predict age, sex, and more - "How much information about a person's behavior can you ascertain from a wearable device? Researchers at the University of Cambridge and Alan Turing Institute sought to investigate this in a study involving over 280,000 hours of wrist accelerometer and wearable electrocardiogram (ECG) data."
Perceptions of AI in the creative community - "Ask an Uber driver what he thinks about self-driving cars and you'll probably get a mix of denial, dismissal, and dread. That should not surprise, as the profession is projected to become obsolete when self-driving cars finally hit the road."
Google wants you to help train its AI by labeling images in Google Photos - "Google has updated its Google Photos app on Android with a new option that lets users tell the search giant about the contents of their pictures. By labeling these images, Google can improve its object recognition algorithms, which in turn make Photos more useful."
It's the screams of the damned!' The eerie AI world of deepfake music - "'It's Christmas time! It's hot tub time!'' sings Frank Sinatra. At least, it sounds like him. With an easy swing, cheery bonhomie, and understated brass and string flourishes, this could just about pass as some long lost Sinatra demo."
The key to smarter robot collaborators may be more simplicity - "Think of all the subconscious processes you perform while you're driving. As you take in information about the surrounding vehicles, you're anticipating how they might move and thinking on the fly about how you'd respond to those maneuvers."
How the pandemic readied Alibaba's AI for the world's biggest shopping day - "The erratic consumer behavior during the pandemic helped Chinese e-commerce giants prepare for Single's Day. The news: While the US has been hooked on its election, China has been shopping."

Concerns & Hype

Frankenstein's Monster:' Images of Sexual Abuse Are Fueling Algorithmic Porn - "Content warning: This article includes firsthand accounts of sexual abuse."
How artificial intelligence may be making you buy things - "The shopping lists we used to scribble on the back of an envelope are increasingly already known by the supermarkets we frequent. Firstly via the loyalty cards we scan at checkouts, and more and more so from our online baskets, our shopping habits are no longer a secret."
Ethical Frameworks for AI Aren't Enough - "Environmental well-being. Human agency. Transparency. These are just a few of the ill-defined principles commonly listed in ethical frameworks for artificial intelligence (AI), hundreds of which have now been released by organizations ranging from Google to the government of Canada to BMW."
Challenges of Responsible AI Development - "Modern customers prefer businesses that offer customized solutions for simple convenience. At the same time, they expect companies to be fair and transparent about how they’re using their personal information."
'Cool Projects' or 'Expanding the Efficiency of the Murderous American War Machine?'' - "In a world where computing technologies such as artificial intelligence are increasingly critical for government operations and national security, the nature of the relationship between the U.S. tech industry and the Department of Defense is important."

Analysis & Policy

People Think Robots Are Pretty Incompetent And Not Funny, New Study Says - "Dang robots are crummy at so many jobs, and they tell lousy jokes to boot. In two new studies, these were common biases human participants held toward robots."
Microsoft and OpenAI propose automating U.S. tech export controls - "Microsoft and OpenAI, the AI research lab in which Microsoft has invested over $1 billion, today submitted a document to the U.S. government describing how a 'digitally transformed' export controls system might work and the benefits it could provide."

Expert Opinions & Discussion within the field

What's next for AI: Gary Marcus talks about the journey toward robust artificial intelligence - "He was not particularly talented at writing, yet became a best-selling author. He is an academic who founded two startups -- one acquired by Uber, another just scored $15 million to make building smarter robots easier."
China Has Caught Up To U.S. In AI, Says AI Expert Kai-Fu Lee - "The possessor of some AI technology is not nearly as valuable as the possessor of data."

That’s all for this week! If you are not subscribed and liked this, feel free to subscribe below!

Last Week in AI