With many Americans turning to large language models for health advice, health systems around the country are eyeing and even rolling out their own branded chatbots in an attempt to harness this already popular tool and steer more people to their services. But the burgeoning trend is raising immediate questions and concerns for the country’s complicated and generally underperforming health care system.

Executives frame the new offerings as a convenience for patients, meeting people where they are and providing a service with digital equity. They also suggest their chatbots will be a safer alternative to commercial versions people are using now.

“We are at an inflection point in healthcare,” Allon Bloch, CEO of clinical AI company K Health, said in a statement. “Demand is accelerating, and patients are already using AI to navigate their lives.”

K Health is working with partner Hartford HealthCare, in Connecticut, to roll out its PatientGPT chatbot to tens of thousands of its existing patients.

“The question isn’t whether AI will shape healthcare, it’s about how we do it in a safe, transparent way, inside a health system that connects to your medical records and your care team. PatientGPT represents that turning point,” Bloch said.

But some experts are wary of the rollouts, raising concerns about whether chatbots are ready for such branded debuts, if there will be sufficient monitoring, what liability will look like, and also whether or not this is the answer to the care problems patients are really raising.

While these risks and questions swirl, the benefits to patients are still only hypothetical. “It’s a tempting idea,” Adam Rodman, a clinical reasoning researcher and internist at Beth Israel Deaconess Medical Center in Boston, told Stat News recently. But there isn’t yet evidence to show that integrating chatbots into health systems improves patient outcomes. “We’re not there yet,” he said.