AI for Healthcare: Enterprise Use and Limits

Artificial intelligence is moving quickly from experimentation to enterprise healthcare use. We have been tracking these shifts closely across compliance, operations, and governance. This post shares what healthcare leaders should know right now, based on real-world experience and recent developments.

Why Enterprise Healthcare AI Deserves a Closer Look

A few months ago, we wrote about the use of AI in Healthcare and its impact on Compliance. In that post, we reviewed the guidance issued by the Joint Commission and the Coalition for Health AI entitled RUAIH – Responsible Use of Artificial Intelligence in Healthcare. This guidance outlines processes and issues for institutions to address when considering implementing artificial intelligence adoption in their healthcare institutions. 

This past month, a couple of the major players in Healthcare AI have released information on their latest AI models designed as “enterprise” applications. They include Anthropic’s Healthcare Enterprise AI offering built around its Claude model, and OpenAI’s ChatGPT for Healthcare. Given the recent controversy around the use of consumer-facing versions of both AIs and Apple Health data, what are these two companies offering to healthcare providers of various types? Let’s take a quick, high-level look. 

Keep in mind these summaries are based on information available from the companies themselves; none of the claims have been validated by The Fox Group or its consultants. And naturally, we used artificial intelligence tools to compile this information.

What are the Names and Models of the Enterprise Healthcare Artificial Intelligence Offerings Released in January 2026?

Anthropic says Claude for Healthcare is best understood as a packaged enterprise capability (not a single new standalone product) combining:

1. HIPAA-ready Claude for Enterprise/Work plans.
2. HIPAA-ready Anthropic API option (tied to Zero Data Retention).
3. Healthcare-oriented connectors + agent skills. These are intended to reduce hallucinations and improve workflow integration using authoritative medical and payer sources.

ChatGPT says its AI is best understood as a bundle called OpenAI for Healthcare, consisting of:

1. ChatGPT for Healthcare (a secure, governed ChatGPT workspace designed for regulated healthcare use)
2. OpenAI API for Healthcare (the developer platform + models used to embed AI into healthcare apps/workflows)
3. “GPT-5 / GPT-5.2 models optimized for healthcare” (the underlying models, evaluated and safety-tested using clinician-led methods and benchmarks)

What are Some of the Common Claims by both AI Companies?

Both companies make certain claims about their capabilities and protective features, but it is a short list.

• Both companies say that Business Associate Agreements are available to healthcare providers covered by HIPAA.
• Both companies say they do not use enterprise customer PHI for model training unless explicitly permitted by the customer-controlled configurations. 

What are Some of the Differences in Targeted Uses by Both AI Offerings?

The companies both emphasize features that have the potential to relieve the administrative and clinical burden on healthcare organizations. And they appear to recognize that there has to be a return on investment for healthcare professionals to invest in these applications. Automating administrative tasks is a high priority.

Most organizations will initially realize the fastest and safest return on investment is in documentation, workflow automation, and evidence retrieval before expanding into higher-risk clinical decision support use cases. 

Claude emphasizes the use of its model for clinical, administrative, and what Anthropic calls “life sciences” tasks. Clinical tasks could include summarizing patient charts/records and drafting clinical documentation with human review. When support begins to include autonomous recommendations for treatment plans or diagnostic findings, review by the FDA under its clinical decision support device regulations is likely.

Administrative tasks envisioned by the creators of Claude for Healthcare artificial intelligence include services like: 

• medical coding validation, 
• prior authorization and denial appeals drafting and evidence accumulation, and
• policy drafting and education.

The Life Sciences task described for Claude includes medical writing/synthesis and medical research trial operations support.

OpenAI for Healthcare artificial intelligence describes both clinical settings and administrative use cases. Clinical uses include evidence retrieval with citations and “trusted clinical data search”, plus what it calls institutional pathway alignment via internal resources like shared filing systems.

Administrative tasks include templates for discharge summaries, patient care instructions, and clinical letters. Such tasks also include support for prior authorization.

OpenAI for Healthcare also anticipates developer-built APIs for workflow. These could include activities such as chart summarization, care team coordination, and discharge workflows.

What do These Healthcare AI Systems Say They are Not?

Both of these companies stress descriptions of what “they are not”, and emphasize limitations health systems should keep in mind.

• These new models are not the same as the consumer-facing artificial intelligence bots by the same or similar names. 
• These AI systems are not electronic health records systems. Nor are they clinical decision support systems or medical devices as defined by the FDA.
• They are not autonomous clinical decision-making or fully autonomous PHI handling agents. Agentic AI tools, advanced forms of artificial intelligence that are designed for autonomous decision-making and action, are in the works, however.
• Despite comments about HIPAA compliance, no systems are human-proof when it comes to unauthorized disclosures of protected health information. So policies and procedures on use (and misuse) are essential.
• ChatGPT for Healthcare AI notes some limitations on capabilities, such as medical imaging and waveform pattern recognition for patient care. Claude for Healthcare AI notes that it may not always be up to date on regulatory changes and may even draw incorrect conclusions from reliable sources of clinical or administrative information(italics added).

What Should You be Doing if Healthcare AI Systems are in Your Immediate Future?

There may come a day when all (or at least most) of the bugs have been worked out of artificial intelligence models, and evaluating healthcare AI systems will be fairly straightforward. Until that happy day, here are some things to think about.

Considerations for Contracts with Healthcare AI Systems.

It almost goes without saying that you need a business associate agreement (BAA) with your Healthcare AI model. Your patient’s PHI will be accessible, so you want the AI organization to have the same responsibilities as you do as a regulated (covered) entity under HIPAA. Although many provisions of BAAs are pretty standard these days, make sure the provisions for breach notification and indemnity are robust. Also insist that the vendor has cyber insurance coverage at reasonable limits, e.g., at least $1 million per occurrence.

Other sensitive terms in a service agreement include suitability for the planned use cases and exceptions for clinical or administrative activities that are not supported, e.g., the ability to analyze medical imaging.

Security and Data Protection

If you are using Certified EHR Technology, your system may already be protected by encryption protocols such as encryption at rest and in transit. You may already have Role-based Access Control and Single Sign On capabilities. But if medical practices are using a stand-alone healthcare AI application, make sure you understand all of the data security and data retention features the vendor is offering. These include training on the model using your patient care data and comprehensive audit logs.

HIPAA and Regulatory Compliance

Ask for vendor documentation on how the anticipated uses of the AI tools are consistent with HIPAA privacy, breach notification, and security regulations. Determine what, if any, of the specific outputs of AI use are not PHI.

HIPAA compliance is not a static situation. The addition of AI tools into clinical workflows and clinical practice affects a great many people in healthcare systems. Make sure they do not inadvertently “leak” PHI due to insufficient internal controls or technical faults.

Review the state laws on privacy of individual health information, and determine how AI adoption may intersect with those laws. For instance, in most states, the medical records created by medical practices or hospital systems are legally owned by the creator. But in at least one state, the medical record belongs to the patient. Is the boundary clear between patient data and the use of AI systems in creating that patient care record?

Finally, make sure the AI for Healthcare vendor is clear on the question of FDA guidance on Artificial/Machine Learning medical devices. 

Technical Architecture & Integration Controls

Documentation on the solution should be specific enough to determine how it will integrate with internal systems such as EHRs or clinical policies. APIs used for integration should be secure, with a specific BAA. And APIs should also be subject to controls on using customer data for training data, and have usable audit logs.

Adopters of a specific a specific solution should be able to document flow diagrams showing where PHI travels, is used, or processed, and where it finally resides. There should also be controls for mobile devices and remote access settings.

Governance, Workforce, and Operations

Managing the use by staff of AI functions and the training required are very important. Most EHR systems already utilize Role-based Authorization schemes; access to an AI model embedded in an EHR may require even tighter controls on access to the AI-supported functions of an EHR application. 

Formal training for users is mandatory since the use of existing AI models like ChatGPT is not self-explanatory. Acceptable Use Policy provisions should be incorporated into the existing AUP for users and systems/capabilities, or a new policy should be written. 

Use of an AI model to compose things like chart content or patient letters may be a great time-saver, but there should be a process for validating the content before incorporating it into the medical record.

Finally, the roles and responsibilities for governance and management of the organization to evaluate AI technologies, select AI applications, and evaluate model performance should be clear. AI in healthcare will not be a simple or one-time activity any time soon. As models become more sophisticated and potentially handle more functions in modern healthcare systems, continuous evaluation of performance and reliability will be required.

Risk Management and Clinical Patient Safety

So-called “hallucinations” by digital systems are one of the significant bug-a-boos of current AI models. Vendors should be able to explain their efforts to reduce this phenomenon and also help users understand how to identify them. Limitations on the capability of the AI system chosen should be clearly identified.

Vendor and Contract Management

There are some usual and customary terms to look for in a contract for healthcare AI, plus some unique issues. Some usual items are performance metrics like uptime, support response times, and escalation processes. Unique terms include advance notice of model upgrades and feature changes. There should also be notice of new data sources for clinical search and audit rights to inspect security measures and compliance evidence from the vendor.

Documentation and Evidence

As you make your way through the processes of selecting and implementing a healthcare AI system, be sure to maintain a comprehensive evidence package. Such a file should include documents like the contract and BAA(s), security documentation (like penetration testing), system architecture documentation, policies, and audit logs. If possible, develop a dashboard of key performance indicators to monitor performance and compliance. These could include things like adverse events, anomalous usage, suspected hallucinations, and potential breaches of PHI related to the AI system components.

Evaluation and Continuous Review

As noted above, continuous monitoring of performance and reliability is part of the AI in healthcare experience. Not only should there be a post-rollout review of the model for compliance verification, clinical safety, and audit validation, but also there should be regular, periodic assessments of model performance, vendor performance, and changing regulatory or risk environments.

AI for healthcare: Promise, Performance, and Practical Limits

The developers of these emerging technologies for enterprise healthcare AI offerings are envisioning a brave new world of efficiency in healthcare delivery. They also hope to improve patient safety, improve health outcomes for patients, deliver precision medicine, and generally enhance patient care.

Right now, we should remember that these systems are probabilistic AI models that generate responses based on patterns learned from large data sets rather than true reasoning or memory. They don’t actually have human-like memory or understanding, and whether their responses stay consistent depends on how the system is configured and what data it can access. In other words, you may get a different answer each time you do the same inquiry.

They can also take in and incorporate healthcare data and present it in organized summaries. Consumer AI tools are often optimized for conversational engagement, which is different from how enterprise healthcare deployments are evaluated and controlled. How well these models will translate to healthcare applications as part of clinical care is yet to be determined.

Beyond the usability and reliability of these AI models are the issues of new risk areas for healthcare systems. For instance:

• Is AI assistance in medical coding consistent enough, given the risk of false claims to government healthcare programs?
• Will clinical documentation prepared with AI assistance stand up to the scrutiny that comes during a medical malpractice case?
• Will the effort required to validate AI-assisted output decline over time, allowing errors to go unnoticed before records are finalized or acted upon?

Strap in; there may be some big bumps and tough lessons on the road ahead!