HIPAA-Compliant LLMs: A 2026 Deployment Guide

You want to use an LLM somewhere in your healthcare workflow, but one question keeps stalling the project: Is this actually allowed with patient data? Although the vendor's website says “HIPAA-compliant”, you still don’t know what's covered and what you still have to configure yourself.

After helping many organizations build HIPAA-enabling apps and add LLMs, I created this guide to make the process easier. You’ll learn what HIPAA requires, explore popular LLM vendors, and see how to choose the right option for your organization.

What Is a HIPAA-Compliant LLM?

A HIPAA-compliant LLM is an AI language model used in a system that meets HIPAA rules for handling PHI (Protected Health Information). However, HIPAA rules apply to covered entities and business associates, so the AI model alone can’t make a system compliant. 

The system also requires HIPAA-enabling features, such as security controls, access limits, audit logs, and encryption. Your organization will also need to implement data policies and risk management. 

Plus, if your LLM vendor handles PHI on your behalf as a business associate, you’ll need a BAA (Business Associate Agreement). This contract defines how the vendor may use and disclose PHI. It also requires appropriate safeguards to protect PHI.

How Does HIPAA Apply to LLMs?

HIPAA applies to healthcare organizations and vendors when their LLM deployments handle PHI. These 4 major HIPAA requirements shape how organizations can use LLM deployments with PHI:

The Privacy Rule

The Privacy Rule governs when PHI may be used or disclosed. It exists to limit PHI access to permitted purposes and give patients rights over their information. You’ll need to create policies that define permitted PHI uses and appropriate workforce access within LLM workflows. 

Your team must know which prompts and records belong in approved systems. For example, a care coordinator uses an authorized deployment to summarize a patient record for treatment purposes.

The Security Rule

The Security Rule establishes safeguards for PHI. It protects the confidentiality, integrity, and availability of PHI. To follow this rule, you’ll need to apply administrative, physical, and technical safeguards around LLM deployments. 

In practice, this means you’ll need to manage permissions, review system activity, and secure PHI as it moves between authorized systems. For example, an administrator revokes an employee’s LLM access immediately after that employee leaves the organization.

The Breach Notification Rule

The Breach Notification Rule sets requirements for responding to breaches of unsecured PHI. When breaches occur, it requires notifying individuals and regulators. If you’re using LLMs that touch PHI, you’ll need to implement procedures for investigating potential exposures and documenting what happened. 

Your team also needs to assess the PHI involved, who received it, whether it was actually acquired or viewed, and how effectively the exposure was mitigated. For example, a compliance team investigates PHI accidentally submitted through an unapproved public LLM account.

Business Associate Agreements

A Business Associate Agreement is a contract governing how a vendor handles PHI on behalf of a covered entity. It defines permitted PHI uses and disclosures, safeguard responsibilities, breach duties, and subcontractor requirements. 

Covered entities need an appropriate BAA before allowing an LLM vendor acting as a business associate to handle PHI on their behalf. Staff then restrict workflows to covered services. For example, clinicians can use an organization-approved AI tool to summarize patient notes.

What Does a HIPAA-Compliant LLM Deployment Require?

A HIPAA-compliant LLM deployment requires a BAA and features like access controls and encryption to protect PHI. Here’s a closer look:

• Access controls: Only approved users should be able to view or use PHI. Set permissions by job role, such as limiting billing staff to claims workflows. For instance, front-desk staff shouldn’t be able to access session notes.
• Encryption: PHI should remain secure when stored or sent between systems. Encrypted connections help protect patient data as it moves to and from an LLM.
• Audit controls and logging: You need to keep records of who accessed PHI and what they did. Compliance teams can review these logs to spot unusual activity and investigate security problems.

• Appropriate data retention: Set clear limits for how long prompts, responses, logs, and other PHI remain stored. Remove data when you no longer need it.
• Secure integrations: Connections between an LLM and healthcare systems must protect PHI as it moves. For example, an EHR can send approved patient information through a protected API.
• Risk analysis and management: Regularly check where an LLM could put PHI at risk. You can then address security issues and review new risks as workflows change.

• Incident response procedures: Your team needs a clear plan for handling possible PHI exposure. Staff should investigate what happened, limit further access, document the incident, and take any required action.

What LLMs Can You Use in a HIPAA-Compliant Environment?

Several LLM vendors can support workflows involving PHI, but you need an appropriate BAA from your chosen vendor and must use services and features covered by that agreement. Your organization also remains responsible for configuring and using the service appropriately.

Here are some LLM options that support HIPAA-compliant deployments:

Azure OpenAI

Azure OpenAI gives organizations access to OpenAI models through Microsoft Azure, and it’s HIPAA-eligible under Microsoft’s applicable BAA framework. You still need to confirm that their Azure services and configurations meet your requirements. For example, developers can use an approved Azure OpenAI deployment to process clinical text.

Amazon Bedrock

Amazon Bedrock is an AWS service that provides access to foundation models from multiple providers. AWS lists Amazon Bedrock as HIPAA-eligible. But if you’re using it with PHI, you need appropriate AWS BAA coverage and must configure the environment securely.

Google Vertex AI

Google Vertex AI provides tools for building and running machine learning and generative AI applications. Certain Vertex AI services can support workloads involving PHI under applicable Google Cloud BAA coverage. Organizations need to confirm that the services and features they use are covered.

OpenAI API

The OpenAI API gives developers access to OpenAI models for applications and automated workflows. OpenAI offers BAAs for eligible API arrangements, and you'll need to confirm your configuration and covered services before processing PHI. Your developers can use an eligible API configuration to summarize patient charts inside a healthcare application.

Anthropic Claude

Claude is Anthropic’s family of language models available through its commercial services and APIs. Anthropic offers BAAs to qualifying customers for specific HIPAA-eligible services, and you’ll need to confirm that their service and configuration are covered before processing PHI.

Hathr AI

Hathr AI provides AI tools for organizations that work with PHI and other sensitive information. Hathr states that its HIPAA plans include a BAA and run on AWS GovCloud infrastructure. Organizations still remain responsible for meeting their own HIPAA obligations.

Self-Hosted Open-Weights LLMs

Self-hosted open-weights LLMs run on infrastructure controlled by your organization or contracted providers. HIPAA compliance depends on how you deploy, secure, manage, and use the model and supporting systems. You must manage safeguards such as access controls, logging, storage, and secure integrations.

3 Ways to Deploy an LLM in a HIPAA-Regulated Environment

You can deploy an LLM in a HIPAA-regulated environment through a cloud service, self-hosting, or a healthcare-specific AI vendor. Here’s a look at these 3 approaches:

1. Use a HIPAA-Eligible Cloud LLM Service

Cloud LLM services let you use AI without managing model hosting, computing infrastructure, and software updates. Your organization still controls user access, settings, and how PHI moves through the system. For example, a medical billing app can send PHI through an API covered by a BAA.

2. Self-Host an Open-Weights LLM

A self-hosted LLM gives your organization more control over where PHI is stored and processed. Your team manages security, access, logs, data storage, and system connections. To illustrate, a hospital can run an open-weights model in its own controlled cloud environment.

3. Use a Healthcare-Specific AI Vendor

A healthcare AI vendor provides AI software designed for healthcare workflows such as clinical documentation and medical record review. Your team still needs to check BAA coverage, approved services, user access, and PHI rules. For example, clinicians can use an approved vendor service to summarize patient notes.

How to Choose an LLM for a HIPAA-Regulated Application

Choose an LLM for HIPAA-regulated apps by confirming whether it’ll handle PHI and checking to see if the vendor offers a BAA. Follow these steps: 

Step 1: Determine Whether the LLM Will Handle PHI

Your team first needs to identify if the LLM will handle PHI. Review every source of data sent to the model, including prompts, uploaded documents, EHR integrations, claims data, and connected applications. If any of that information identifies a patient and relates to their health, care, or payment for care, treat the workflow as handling PHI.

Step 2: Check the BAA and Covered Services

Once you’ve identified potential LLM vendors, your team should check which vendor services the BAA covers. A vendor’s BAA may not cover every product or feature. You’ll need to make sure that the products and features that will touch PHI are covered and include HIPAA-enabling safeguards.

Step 3: Map Where PHI Travels

A PHI data map documents every system, service, integration, and storage location that receives protected health information during an LLM workflow. Use it to identify responsibility across the data path. Remember to document transfers and destinations and have your team review access points as integrations change. All connections require review before receiving PHI.

Step 4: Review Data Retention and Model Training Policies

Data retention and model training policies describe whether and how a vendor stores submitted information or uses it after processing. You need to review these terms because PHI handling continues beyond the model request when data persists. Configure approved retention settings and track policy changes.

Step 5: Evaluate Your Internal Technical Resources

Internal technical resources are the people and capabilities available to build, secure, integrate, and maintain an LLM application. Deployment choices create different levels of technical ownership. You should assign responsibility for infrastructure, access, logging, and updates. 

If you have limited infrastructure resources, you may find managed (cloud) services more practical. But if you have experienced engineers, you may choose to host and manage the LLM on your own servers.

Common HIPAA and LLM Mistakes

HIPAA problems can happen when teams lose track of where PHI goes, which services can handle it, or how it stays protected. Knowing these risks helps teams protect patient information.

• Using consumer AI tools with PHI: Consumer AI tools may not be approved for handling PHI. Only use services with HIPAA-enabling features from vendors that provide BAAs.
• Treating self-hosting as automatic compliance: Running your own LLM doesn’t automatically make it HIPAA compliant. Your organization must still protect PHI and set access controls before the model processes patient records.
• Overlooking prompts and outputs containing PHI: Prompts and AI responses can contain PHI and require proper protection. Your team must keep this information within approved systems. For example, patient summaries containing PHI need to stay within authorized tools.
• Exposing PHI through logs: System logs may capture PHI from prompts, responses, or patient details. Protect these logs from unauthorized access and limit unnecessary data. 
• Giving the LLM excessive data access: LLMs should only receive the patient information needed for their task. Limiting access to other apps reduces PHI exposure.
• Ignoring third-party integrations and subcontractors: Outside services may receive or handle PHI as part of an LLM workflow, so review these services and agreements first. 

Build Healthcare Applications With Blaze

Finding the right HIPAA-eligible LLMs solvesw only part of the problem. Blaze helps healthcare organizations build applications around AI while connecting the clinical systems and workflows that supply the data.

Here’s why more healthcare organizations choose Blaze:

• Healthcare software built for you: An expert-led three-person implementation team, including a project manager, healthcare developer, and integration engineer, builds production-ready patient portals, clinical databases, and AI-enabled applications. Blaze’s Integrations team can also connect your EHRs and other systems.
• Opt to self-build: Use Blaze’s visual builder to create custom healthcare dashboards, AI workflows, and applications without technical expertise.
• Replace repetitive administrative work: Automate patient intake, document routing, approvals, reminders, and other manual tasks while keeping your existing EHR in place.
• Faster implementation than traditional builds: Launch AI-enabled healthcare applications in weeks instead of months.
• AI integrations built for real clinical workflows: Support use cases such as automated patient intake, document extraction, and OpenAI integration alongside secure EHR and EMR connections built around how your team works.
• Built on compliance-ready infrastructure: Blaze is a HIPAA-enabling, HITRUST e1-certified healthcare app development platform that maintains SOC 2 Type II compliance.

Schedule a free build consultation call today and turn your HIPAA-regulated LLM use case into a working healthcare application connected to the systems your team already uses.

Frequently Asked Questions 

What Is HIPAA-Compliant LLM Inference?

HIPAA-compliant LLM inference is sending input to a trained LLM and receiving output, with PHI potentially present in prompts, context, and responses. Proper hosting, logging, and BAAs reduce unauthorized PHI exposure risk.

Can You Build a HIPAA-Compliant AI Agent?

Yes, you can build a HIPAA-compliant AI agent when the surrounding deployment meets HIPAA requirements. That includes appropriate BAAs, access controls, authentication, audit logging, risk management, and policies governing PHI. Since agents connect to APIs and records, limit each agent to the data and actions required for its job.

Can You Use an Open-Source LLM With PHI?

Yes, you can use an open-source LLM with PHI. You’ll need to deploy it within infrastructure that meets your organization’s HIPAA requirements, such as a controlled cloud environment or private servers. Your organization must secure the surrounding system with appropriate access controls, encryption, audit logging, data retention policies, and risk management. You also need BAAs with any service providers that handle PHI on your behalf.

Sources

1. U.S. Department of Health & Human Services. “Summary of the HIPAA Security Rule.” HHS.gov. https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html

2. U.S. Department of Health & Human Services. “Security Rule Guidance Material.” HHS.gov. https://www.hhs.gov/hipaa/for-professionals/security/guidance/index.html

3. National Institutes of Health: StatPearls. “Health Insurance Portability and Accountability Act (HIPAA) Compliance.” NCBI. https://www.ncbi.nlm.nih.gov/books/NBK500019/