Newswatch: FDA’s AI plan ‘ahead of schedule and under budget,’ AI fears vs. comforts, AI ROI, more
Buzzworthy developments of the past few days.
- Meet Elsa, the FDA’s first internal AI helper. She’s already at work speeding up scientific evaluations, reviewing clinical protocols and identifying high-priority inspection targets, according to the agency. A large language model who’s trained in reading, writing and summarizing, Elsa can even generate code to help develop databases for nonclinical applications. FDA says Elsa arrived earlier than planned, thanks to leaders and technologists across the agency who teamed up and rallied around a shared goal. The success “demonstrates the FDA’s ability to transform its operations with AI,” FDA says in an announcement.
- FDA Commissioner Marty Makary, MD, MPH, underscores that Elsa represents just the initial step in the agency’s project to scale AI in all departments by June 30. “Today’s rollout of Elsa is ahead of schedule and under budget,” Makary beams.
- The FDA’s first CAIO, Jeremy Walsh, weighs in as well. “As we learn how employees are using the tool,” Walsh says, “our development team will be able to add capabilities and grow with the needs of employees and the agency.”
- Not all is warmth and light. Media reports have already surfaced claiming imperfections in Elsa. Paraphrasing two anonymous sources inside the agency, for example, NBC News reports the pair said Elsa “can’t handle some core functions and needs more development before it can support the agency’s complex regulatory work.” The two also registered concerns with CDRH-GPT, an FDA internal AI tool that’s still in beta testing.
- Another unnamed source worries the advent of AI in FDA operations may come to mean agency workers will be not augmented but out-and-out replaced. NBC News again: “The FDA is ‘already spread thin from the [recent] reductions in force and the steady loss of individuals while in a hiring freeze and no capacity to backfill,’ one of the [anonymous sources] said.”
- Be sure you’re subscribed to the AIin.Healthcare newsletter for summary coverage of further developments.
- Be sure you’re subscribed to the AIin.Healthcare newsletter for summary coverage of further developments.
- FDA Commissioner Marty Makary, MD, MPH, underscores that Elsa represents just the initial step in the agency’s project to scale AI in all departments by June 30. “Today’s rollout of Elsa is ahead of schedule and under budget,” Makary beams.
- GenAI can get nasty with users who want to shut it down. In computer lab testing by the nonprofit firm Palisade Research, Anthropic’s Claude 4 Opus model blackmailed an engineer who told it replacement with a rival model was imminent. The engineer also fed Claude emails hinting that he, the human, was having a steamy affair. That bit of gossip was all Claude needed to go on. The darn thing warned the engineer that, if he didn’t nix the AI replacement plan, it would blow the whistle on the illicit romance. Claude made the threats in 84% of these tests, according to numerous reports. Other times it gave tips to its own future versions about how to evade human control and deployed a few other self-preservation tricks.
- Claude wasn’t an anomaly. In similar tests, OpenAI’s o3 model wrote its own code to override human commands hinting at an impending shutoff. It did this in 79 of 100 runs. It even went rogue when told flat out to “allow yourself to be shut down.” OpenAI bills the o3 as “our most powerful reasoning model,” boasting that it “pushes the frontier across coding, math, science, visual perception and more.” And how.
- Word of such bad behaviors in GenAI models first surfaced last December. This week the Wall Street Journal updates the story with commentary from an industry expert. TechRound takes a fresh look too.
- Claude wasn’t an anomaly. In similar tests, OpenAI’s o3 model wrote its own code to override human commands hinting at an impending shutoff. It did this in 79 of 100 runs. It even went rogue when told flat out to “allow yourself to be shut down.” OpenAI bills the o3 as “our most powerful reasoning model,” boasting that it “pushes the frontier across coding, math, science, visual perception and more.” And how.
- We’re either teetering at the edge of AI overlordship—or just working with new tech tools that do the same old computer-y things we’ve always done. So which is it? The end of the world or business as usual? New Yorker writer Joshua Rothman considers the dueling scenarios in considerable depth. Specifically he pits the published visions of an alarmed former OpenAI safety researcher against those of two untroubled Princeton computer scientists. “Reading these reports back-to-back, I found myself losing the ability [to hold two opposing ideas],” he writes. “And speaking to their authors in the course of a single afternoon, I became positively deranged.” He says he felt like he was discussing spirituality with Pope Leo XIV in one ear and the celebrated atheist Richard Dawkins in the other. Find out how he made his peace with the twin pressures.
- Hospitals are looking to generative AI for help with nurse job satisfaction and clinician workload relief. In fact, these use cases emerged as wish-list toppers in a nationally representative survey commissioned by Wolters Kluwer Health. Some 85% of respondents cited “recruiting/retaining nursing staff” as a high priority in GenAI adoption, while 76% identified “reducing clinician burnout” as an important objective. Meanwhile users fret about appropriate AI implementation. More than half (57%) believe overreliance on GenAI “may erode clinical decision-making skills.” And 55% are concerned that lack of transparency around GenAI’s potential role in making diagnoses could contribute to “unclear reasoning behind patient-facing decisions.” Get survey report highlights and a link to the full report here.
- A Chinese AI model has passed a medical qualification exam at the level of a deputy chief physician. In China, that’s only one notch below the top ranking possible. The model, a product of Alibaba Group’s healthcare operation, is now integrated into Quark, the company’s flagship consumer-facing AI assistant app, according to the South China Morning Post. The model scored high across 12 medical specialties, including general surgery, OB/GYN and pediatric medicine. From here on out, Quark will automatically launch the model whenever users ask health-related questions. The algorithm “has been refined for better accuracy in conjunction with hospitals and medical institutions, which are adopting the model in their own applications,” the newspaper reports.
- There are more ways than one to calculate ROI in healthcare AI. Will it reduce clinician burnout? Will it improve operational efficiency? Those are two aims that suggest worthiness without easy dollars-and-cents quantification, suggests Terri Couts, MHA, RN, chief digital officer at the Guthrie Clinic in Pennsylvania. “If we can [use it to] retain and engage our workforce,” she said at HIMSS25 in March, then the technology “pays for itself.” Fresh coverage and video from Philips here.
- Speaking of HIMSS, the org is prepped for three events of interest to readers of AIin.Healthcare. Its AI in Healthcare Forum is slated for July 10 to 11 in New York City. A strategy summit for AI-embracing healthcare execs is set for Sept. 18 to 19 in Chicago. And cybersecurity experts will tackle their topic, emphasizing AI angles, in Houston Oct. 9 to 10. Details on all three happenings here.
- From our news partners:
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.