Dr. Sina Bari, a practicing surgeon and AI healthcare leader, has seen firsthand how ChatGPT can lead patients astray with faulty medical advice. He recently had a patient who came in with a printed dialogue from ChatGPT claiming a recommended medication carried a 45% chance of pulmonary embolism. Upon investigation, Dr. Bari found the statistic was from a paper about the medication’s impact in a niche subgroup of people with tuberculosis, a context that did not apply to his patient.
Yet, when OpenAI announced its dedicated ChatGPT Health chatbot last week, Dr. Bari felt more excitement than concern. The new tool, rolling out in coming weeks, allows users to discuss health in a more private setting where their messages are not used as training data. Dr. Bari thinks this formalization is great, as it protects patient information and adds safeguards, making it more powerful for patients to use.
Users can get more personalized guidance from ChatGPT Health by uploading medical records and syncing with apps like Apple Health and MyFitnessPal. For the security-minded, this raises immediate red flags. Itai Schwartz, co-founder of data loss prevention firm MIND, notes that medical data would transfer from HIPAA-compliant organizations to non-HIPAA-compliant vendors, raising questions about regulatory approach.
But many industry professionals believe the cat is already out of the bag. Instead of Googling symptoms, over 230 million people already talk to ChatGPT about their health each week. Andrew Brackin, a partner at Gradient who invests in health tech, says this was one of the biggest use cases for ChatGPT, so building a more private, secure, and optimized version for healthcare questions makes sense.
AI chatbots have a persistent problem with hallucinations, a particularly sensitive issue in healthcare. According to evaluation models, OpenAI’s latest model is more prone to hallucinations than many from Google and Anthropic. But AI companies see potential to rectify inefficiencies in healthcare, with Anthropic also announcing a health product this week.
For Dr. Nigam Shah, a professor at Stanford and chief data scientist for Stanford Health Care, the inability of American patients to access care is more urgent than the threat of poor AI advice. Wait times to see a primary care doctor can be three to six months. Faced with a choice between a long wait or talking to an AI that can provide some assistance, many patients may choose the latter.
Dr. Shah believes a clearer route for AI in healthcare is on the provider side. Studies report administrative tasks can consume about half a primary care physician’s time, reducing the number of patients they can see. Automating that work could let doctors see more patients, potentially reducing the need for patients to rely solely on tools like ChatGPT Health.
Dr. Shah leads a team developing ChatEHR, software built into electronic health record systems to let clinicians interact with patient records more efficiently. Dr. Sneha Jain, an early tester, says making records more user-friendly means physicians spend less time searching for information and more time talking to patients.
Anthropic is also working on AI products for clinicians and insurers, not just its public chatbot. Its newly announced Claude for Healthcare aims to reduce time spent on administrative tasks like prior authorization requests. The company notes that saving even 20 to 30 minutes per case represents dramatic time savings.
As AI and medicine become more intertwined, a fundamental tension exists between a doctor’s primary incentive to help patients and a tech company’s accountability to shareholders. Dr. Bari acknowledges this tension is important, stating that patients rely on medical professionals to be cynical and conservative in order to protect them.