A caution about rising hype for artificial intelligence in medicine

hal9000-300x225In recent years, doctors, hospitals, and popular media have promoted emerging treatments to the public with enthusiasm that in each case would turn out to be overblown. Just consider the red-hot chatter that once surrounded regenerative medicine, precision medicine, gene therapy, or immunotherapy. And now, it may be the turn of artificial intelligence to be hyped hard in health care.

Caveat emptor, as Liz Szabo reported for the Kaiser Health News Service. She sets the stage, thusly, about developments in a field that might worry some who remember Hal 9000 from “2001: a Space Odyssey”:

“Health products powered by artificial intelligence, or AI, are streaming into our lives, from virtual doctor apps to wearable sensors and drugstore chatbots. IBM boasted that its AI could ‘outthink cancer.’ Others say computer systems that read X-rays will make radiologists obsolete. ‘There’s nothing that I’ve seen in my 30-plus years studying medicine that could be as impactful and transformative’ as AI, said Dr. Eric Topol, a cardiologist and executive vice president of Scripps Research in La Jolla, Calif. AI can help doctors interpret MRIs of the heartCT scans of the head and photographs of the back of the eye, and could potentially take over many mundane medical chores, freeing doctors to spend more time talking to patients, Topol said. Even the Food and Drug Administration ― which has approved more than 40 AI products in the past five years ― says ‘the potential of digital health is nothing short of revolutionary.’”

But Szabo warns of the problems that loom with AI, particularly due to its roots in high tech — a sector that believes, as a Facebook founder once argued, that tech innovators should “move fast and break things.” This can contradict medicine’s tenets and practices, which might include “go slow and protect the patient,” she reported:

“Early experiments in AI provide a reason for caution, said Mildred Cho, a professor of pediatrics at Stanford’s Center for Biomedical Ethics. Systems developed in one hospital often flop when deployed in a different facility, Cho said. Software used in the care of millions of Americans has been shown to discriminate against minorities. And AI systems sometimes learn to make predictions based on factors that have less to do with disease than the brand of MRI machine used, the time a blood test is taken or whether a patient was visited by a chaplain. In one case, AI software incorrectly concluded that people with pneumonia were less likely to die if they had asthma ― an error that could have led doctors to deprive asthma patients of the extra care they need. ‘It’s only a matter of time before something like this leads to a serious health problem,’ said Dr. Steven Nissen, chairman of cardiology at the Cleveland Clinic.”

Such cautions, of course, haven’t dissuaded tech-happy investors from pouring $1.6 billion into AI medical products in just the third quarter of 2019, Szabo found. And, while hope and optimism may abound now, the practical and regulatory obstacles in this area can’t be ignored. She reported that:

“None of the AI products sold in the U.S. have been tested in randomized clinical trials, the strongest source of medical evidence, Topol said. The first and only randomized trial of an AI system ― which found that colonoscopy with computer-aided diagnosis found more small polyps than standard colonoscopy ― was published online in October. Few tech startups publish their research in peer-reviewed journals, which allow other scientists to scrutinize their work, according to a January article in the European Journal of Clinical Investigation … And although software developers may boast about the accuracy of their AI devices, experts note that AI models are mostly tested on computers, not in hospitals or other medical facilities. Using unproven software ‘may make patients into unwitting guinea pigs,’ said Dr. Ron Li, medical informatics director for AI clinical integration at Stanford Health Care.”

Google, the tech behemoth, recently funded and assisted with research on using AI in mammograms (breast X-rays) to detect cancer, the New York Times reported. The study tested AI on roughly 90,000 X-rays from American and Canadian patients where a diganosis already had been made. With the U.S. images, the newspaper reported “the system produced a 9.4% reduction in false negatives, in which a mammogram is mistakenly read as normal and a cancer is missed. It also provided a lowering of 5.7% in false positives, where the scan is incorrectly judged abnormal but there is no cancer.”

This use of AI may be especially useful, perhaps to parse out the many negative screenings, so radiologists may focus on positive and challenging diagnoses, the New York Times reported, quoting one expert on machines’ advantages over people: “Unlike humans, computers do not get tired, bored or distracted toward the end of a long day of reading mammograms.”

The study went further, the newspaper reported:

“The researchers pitted AI against six radiologists in the United States, presenting 500 mammograms to be interpreted. Over all, AI again outperformed the humans. But in some instances, AI missed a cancer that all six radiologists found — and vice versa.”

Independent expert who looked at the study findings cautioned about its results, noting that devices that fare well in research do not always do so in actual use. Further, as the newspaper quoted one expert, “the patients studied might not be a true reflection of the general population. A higher proportion had cancer, and the racial makeup was not specified … [the]  ‘reader’ analyses involving a small number of radiologists — this study used six — were not always reliable.”

AI products carry other concerns, too, not the least of which is software developers’ reluctance to be transparent about their systems, Szabo reported. Doctors early on may plug in information into AI programs, but clinicians may not see nor know how devices process data and make medical determinations because those likely will be part of proprietary software. The inability to know what goes on in high tech “black boxes” already has become a nightmare for law enforcement and legal experts dealing with roadside sobriety testing devices.

In my practice, I see not only the harms that patients suffer while seeking medical services, but also the injury that can be inflicted on them by defective and deadly products, particularly of the medical kind. Pro-business politicians, particularly of the Republican stripe, have pushed the FDA both to hurry and ease up on its reviews of medical devices, arguing that valuable innovations should be rushed to benefit the public.

But the extremes to which this practice has been taken has turned the medical device field into what patient safety advocates term a “Wild, Wild West.” They have assailed the FDA for already giving makers huge leeway, zipping products through to the public because they maybe, sort of, possibly look or act like devices already there.

Szabo reported that with AI, under the leadership of former FDA commissioner Scott Gottlieb, regulators have cast yet more safety precautions to the wind, creating a “pre-certified” category for targeted makers and “low- and moderate-risk” devices.

The FDA, in effect, looks at certain companies and decides, hey, they seem to be nice folk, giving the likes of Apple, Fitbit (now Google) and Samsung a wave through tests, trials, and reviews. The firms also are supposed to monitor outcomes and harms of their own stuff, informing the public and the agency in an “honor” system. The agency also has smiled at the likes of Johnson & Johnson, Pear Therapeutics, Phosphorus, Roche, Tidepool and Verily Life Sciences.

Why exactly, and what precisely are low- or moderate-risk medical devices? As Szabo reported: “The FDA [already] has come under fire in recent years for allowing the sale of dangerous medical devices, which have been linked by the International Consortium of Investigative Journalists to 80,000 deaths and 1.7 million injuries over the past decade.”

We have a lot of work to do to cut through the hype about therapies, devices, and prescription medications. We also need to let politicians and regulators that they work for we voters, not fat cats at companies that may be donating richly to their campaigns. The upcoming elections would be an excellent time to send a message that patients health, well-being, safety, and finances need to be a first and foremost concern.

Patrick Malone & Associates, P.C. listed in Best Lawyers Rated by Super Lawyers Patrick A. Malone
Washingtonian Top Lawyer 2011
Avvo Rating 10.0 Superb Top Attorney Best Lawyers Firm
Contact Information