Healthcare organizations can use generative AI without putting PHI at risk. Use approved enterprise model services under a business associate agreement (BAA), send only the data a task needs, restrict access, log every request and keep staff review in place. NeoTek Solutions in Nashville helps providers, payers and health tech companies design these setups for HIPAA requirements.
Clinicians and staff are already curious about generative AI. It can draft letters, summarize long records and answer policy questions in seconds. The risk is that someone pastes protected health information (PHI) into a tool that was never approved to hold it.
This guide is for healthcare leaders, compliance officers and IT teams. It explains where PHI tends to leak into AI tools and how to design a safer setup. It is general information, not legal advice, so consult your compliance or legal team before making decisions.
Why This Matters in Nashville
Nashville is one of the country’s major healthcare industry hubs. HCA Healthcare is headquartered here, and Brentwood and Franklin host many healthcare companies. That concentration means a large number of providers, payers, revenue cycle firms and health tech startups are weighing the same AI questions.
The pressure to use AI is real. So is the duty to protect patient data. The good news is that the two goals are compatible when you plan the architecture and the rules together.
What Counts as PHI in an AI Workflow
Generative AI tools touch data in more places than most people expect. PHI can appear in any of these:
- PromptsThe text a user types, such as “Summarize this discharge note for Jane Doe.”
- Uploaded filesRecords, faxes, images or spreadsheets attached to a request.
- OutputsThe summary or draft the AI returns, which often repeats identifiers.
- LogsConversation histories and system logs the vendor or your team keeps.
- Search indexesThe stored, searchable copies of documents used by retrieval systems.
Each of these is a place where PHI is created, received, maintained or transmitted. Each one needs the same protections as any other system that handles patient data.
The Rules That Still Apply
Generative AI does not get a special exemption. The same HIPAA framework applies to AI tools as to any other technology.
- HIPAA Privacy RuleLimits how PHI may be used and disclosed. Its minimum necessary standard means sharing only what a task requires.
- HIPAA Security RuleRequires administrative, physical and technical safeguards for electronic PHI.
- Breach Notification RuleSets out notification duties if unsecured PHI is compromised.
- Business associate agreements (BAAs)A BAA is generally needed when a vendor handles PHI on behalf of a covered entity or another business associate.
State privacy laws and your own contracts may add further requirements. Your compliance or legal team should confirm which rules apply to each use case.
A BAA is necessary in many cases, but it is not enough by itself. A tool can have a signed BAA and still be configured in a way that exposes data. Settings, access controls and staff behavior matter just as much.
Where PHI Leaks Into Generative AI Tools
Most exposure is not a dramatic breach. It is ordinary people trying to get work done with the tools in front of them. Watch for these common gaps:
- Consumer chat toolsStaff use personal accounts on public AI services that have no BAA.
- Browser extensions and plug-insAdd-ons that read page content, including patient portals and EHR screens.
- Transcription and note toolsApps adopted by a single department without a security review.
- Vendor data useTerms that allow the provider to retain prompts or use them to improve models.
- Unrestricted search indexesA document assistant that lets any employee retrieve any record.
- Over-shared outputsAI summaries saved to shared drives or emailed without the usual controls.
A Safer Architecture, Step by Step
This is the sequence we follow when we design a PHI-aware generative AI setup, and we adapt it to your environment with your security and compliance teams.
- Classify the use case. Decide whether it truly needs PHI. Many valuable uses, such as policy questions, need none at all.
- Choose approved model access. When PHI is involved, use enterprise model services covered by a BAA.
- Control data retention. Confirm how long prompts and outputs are kept, and turn off retention you do not need.
- Confirm no model training on your data. Get it in writing that your data will not train the provider’s models.
- Minimize and de-identify. Strip identifiers before sending data when the task allows it. HIPAA recognizes the Safe Harbor and Expert Determination methods for de-identification.
- Enforce access controls. If the AI retrieves documents, it should only return what that user is already allowed to see.
- Keep it inside your environment. Run the application in your own cloud tenant on Azure, AWS or Google Cloud where possible.
- Log and monitor. Record who asked what and what was returned, and protect those logs as PHI.
- Keep a human in the loop. Require clinical or staff review before AI outputs reach a patient or a chart.
- Test before launch. Try to make the system reveal data it should not. Fix what you find.
This is the kind of design work our generative AI and LLM solutions team does. We design for HIPAA requirements and work within your existing compliance program.
Lower-Risk Places to Start
You do not have to begin with the most sensitive workflow. These are the starting points we most often recommend, because they carry less PHI risk:
- An internal assistant that answers questions from policies, procedures and benefits documents.
- Drafting general patient education materials that staff review before use.
- Summarizing payer policy documents and contract terms for revenue cycle teams.
- Helping IT and help desk staff search technical documentation.
- Creating training content and quizzes for staff onboarding.
Once governance is proven on these, you can move to PHI-heavy uses like clinical documentation support. See how we approach these projects on our healthcare industry page.
Questions to Ask Any AI Vendor
Before any AI vendor touches patient data, get clear answers to these questions:
- Will you sign a business associate agreement for this service?
- Where is our data stored and processed, and for how long?
- Is any of our data used to train or improve your models?
- Can we turn off prompt and output retention?
- How is data encrypted in transit and at rest?
- What access controls and audit logs are available to us?
- How do you handle subprocessors, including the underlying model provider?
- What is your process if a security incident affects our data?
Be cautious of any vendor that claims to be “HIPAA certified.” HHS does not offer an official HIPAA certification. Look instead for clear contract terms and verifiable security controls.
Governance: The People Side
Technology controls only work if people know the rules. A few basics go a long way:
- Publish a short, clear policy on approved AI tools and what data may go into them.
- Give staff an approved option, so they are less tempted to use personal accounts.
- Train employees with real examples of safe and unsafe prompts.
- Name an owner who reviews and approves new AI use cases.
- Review AI systems on a set schedule, not just at launch.
The NIST AI Risk Management Framework offers a helpful structure for mapping and managing AI risks. Our AI governance, security and compliance service helps healthcare teams put a practical program in place.
How Can NeoTek Solutions Help?
NeoTek Solutions in Nashville helps providers, payers, revenue cycle firms and health tech companies use generative AI while protecting PHI. We design for HIPAA requirements and work within your compliance program and privacy office.
- AssessmentWe review workflows, data flows and vendor terms, and rank lower-risk use cases.
- Private deploymentWe build in your cloud tenant or through providers that sign BAAs where required, with PHI minimization.
- OversightWe add access controls, audit logs and human review, and your data is never used to train public models.
See our healthcare AI work.
Frequently Asked Questions
Can our staff use public AI chat tools with patient information?
Our advice is no. PHI should not go into any tool that lacks a BAA and approved settings, and personal or consumer accounts rarely meet that bar. We help you give staff an approved option instead, and your compliance team sets the policy.
Do you rely on de-identification to remove HIPAA concerns?
Only where it genuinely applies. Data de-identified under HIPAA’s Safe Harbor or Expert Determination methods is no longer PHI, and we treat free-text notes with particular care. We still design for re-identification risk and other laws alongside your compliance team.
Is a BAA with an AI vendor enough to protect PHI?
No, and we do not treat it as enough. A BAA sets responsibilities, while safe use depends on the configuration, access controls, retention settings and training we put in place around it.
Can NeoTek Solutions build generative AI that handles PHI?
Yes. We design these solutions for HIPAA requirements, with PHI minimization, private deployment, BAAs where required and audit logs. Your compliance and legal teams make the final calls.
Plan a PHI-Aware AI Project
If your organization wants the benefits of generative AI without adding privacy risk, we can help you design it right from the start. Book a free AI consultation to talk through your use case.