AI Fundamentals Through a Healthcare Lens

Context windows — explained through clinical trial protocols

One of the most important concepts in working with LLMs is the context window — the amount of text the model can process in a single interaction. Think of it as the model's working memory.

Here is the healthcare analogy: a clinical trial protocol for a Phase III oncology study is typically 150-200 pages. A model with a large context window (200,000+ tokens, roughly 150,000 words) can read the entire protocol in a single pass. It can then answer questions about any section, identify inconsistencies between the eligibility criteria and the statistical analysis plan, or compare the protocol against the investigator's brochure.

A smaller context window means the model can only see part of the protocol at a time — like a reviewer who can only look at 20 pages before forgetting the first 20. This matters enormously for healthcare workflows:

• Clinical documentation: A single patient's hospitalisation record might include admission notes, daily progress notes, consultant notes, lab results, imaging reports, medication lists, and nursing assessments. A large context window lets the model process the entire hospitalisation to generate an accurate discharge summary.

• Regulatory submissions: An NDA submission module might run to hundreds of pages. The model needs to hold the entire document to check cross-references and internal consistency.

• Literature review: A systematic review might involve comparing findings across 50 research articles. The model needs enough context to hold the key data points from all of them simultaneously.

The practical implication: when you are designing AI workflows for healthcare, always check whether your document fits within the model's context window. If a 200-page protocol fits, you can process it in one pass. If your data exceeds the window, you need to break the task into chunks — and that changes how you structure your prompts.

Capabilities for healthcare workflows

Here is what LLMs do well in a healthcare context:

Summarisation and synthesis. Give the model a 30-page inpatient record and ask for a structured discharge summary. Give it 15 research articles and ask for a comparative evidence table. Give it a 200-page clinical trial protocol and ask for a summary of key eligibility criteria. The model processes large volumes of text and extracts the relevant information based on your instructions.

Drafting structured documents. Prior authorisation letters, appeal letters for denied claims, CDI queries, patient communication templates, regulatory submission sections, adverse event narrative summaries. These are all documents that follow predictable structures and draw from source data. The model generates first drafts that a human reviews and finalises.

Cross-referencing and consistency checking. Does the primary endpoint in Section 6 of the protocol match the primary endpoint in the statistical analysis plan in Section 9? Does the ICD-10 code on the claim match the clinical documentation in the note? Does the eligibility criterion in the protocol align with the inclusion criteria in the informed consent form? Humans are inconsistent at this kind of cross-referencing across long documents. LLMs are systematic.

Data extraction from unstructured text. Extracting structured data from freeform clinical notes, pulling key safety findings from adverse event narratives, identifying medication names and dosages from physician orders. When the source is text and the output is structured data, LLMs perform well.

Translation between audiences. Translating a clinical trial result into a patient-friendly summary. Converting a dense radiology report into a referring physician summary. Drafting a regulatory response that addresses an FDA reviewer's specific questions. LLMs handle register shifts effectively.

Limitations — and the critical clinical boundary

LLMs have real limitations that matter enormously in healthcare:

Hallucination. LLMs can generate text that sounds authoritative but is factually incorrect. In a healthcare context, this means: a cited study that does not exist, a dosage that is wrong, a regulatory requirement that is fabricated, or an ICD-10 code that does not match the documentation. This is why every AI output in a healthcare workflow requires human review. No exceptions.

No access to real-time clinical data. Unless you explicitly provide patient data to the model (which raises HIPAA considerations we will cover shortly), it has no access to your EHR, lab systems, pharmacy systems, or any patient-specific information. The model works with what you give it.

No clinical judgment. This is the most important limitation for healthcare. An LLM can summarise a patient's lab trends, but it cannot decide whether those trends warrant a change in treatment. It can draft a discharge plan based on the clinical documentation, but a physician must review and approve it. It can suggest ICD-10 codes based on the note, but a certified coder must validate them.

The critical boundary: administrative AI vs clinical AI. This course covers administrative and operational AI — the kind that assists with documentation, coding, scheduling, and compliance. Clinical AI — the kind that supports diagnostic decisions, treatment recommendations, or patient risk stratification — falls under FDA regulatory oversight and requires a fundamentally different governance framework. Do not blur this line. If an AI output could directly influence a clinical decision about a specific patient's care, it is clinical AI and needs FDA-appropriate oversight.

Temporal limitations. LLMs are trained on data up to a certain date. They may not reflect the most recent CMS billing guidelines, the latest FDA draft guidance, or this year's HEDIS measure specifications. Always verify AI outputs against current regulatory sources.

HIPAA and data handling — the PHI distinction

This is the section that every healthcare professional needs to understand before using any AI tool. HIPAA's Privacy Rule protects Protected Health Information (PHI) — individually identifiable health information that relates to a patient's past, present, or future health condition, treatment, or payment.

The 18 HIPAA identifiers that make health information "individually identifiable" include: names, dates (except year), phone numbers, geographic data smaller than a state, fax numbers, email addresses, Social Security numbers, medical record numbers, health plan beneficiary numbers, account numbers, certificate/license numbers, vehicle identifiers, device identifiers and serial numbers, web URLs, IP addresses, biometric identifiers, full-face photos, and any other unique identifying number.

What this means for AI tools:

If you are inputting PHI into an AI tool, the AI vendor is acting as a Business Associate under HIPAA. You need a Business Associate Agreement (BAA) in place. Not all AI vendors offer BAAs. Not all AI deployment models support PHI processing. This is a non-negotiable requirement — using an AI tool with PHI without a BAA is a HIPAA violation, full stop.

De-identified data — health information with all 18 identifiers removed — is not PHI under HIPAA and can generally be processed through AI tools without a BAA. However, de-identification must be done properly: either through the Safe Harbor method (removing all 18 identifiers) or the Expert Determination method (a qualified statistical expert certifies that the risk of re-identification is very small).

The practical workflow options:

1. Use a HIPAA-compliant AI deployment with a BAA. Some vendors offer enterprise deployments specifically designed for healthcare with BAAs, data handling agreements, and no data retention for model training. This allows you to process PHI directly.

2. De-identify before processing. Strip all 18 identifiers from the data before sending it to the AI tool. The AI processes the de-identified data and returns results, which you then re-link to the patient record. This adds a step but works with any AI tool.

3. Use AI only for non-PHI workflows. Many healthcare AI use cases do not require PHI at all: literature review, protocol analysis, policy drafting, general communication templates, regulatory research. Start here while you establish HIPAA-compliant infrastructure for PHI workflows.

Prompt design for healthcare workflows

A prompt is the instruction you give to an AI model. In healthcare, prompt design is more consequential than in most industries because the stakes are higher: a poorly designed prompt can produce a discharge summary that misses a critical medication, a prior auth letter that omits the medical necessity justification, or a literature review that overlooks a key safety signal.

The principles of effective healthcare prompting:

Specify the role and context. Start every prompt by telling the model what role it is playing and what context it is working within. "You are a clinical documentation specialist reviewing inpatient records for discharge summary preparation" is far more effective than "Summarise this document."

Define the source data explicitly. Tell the model exactly what data you are providing. "The following is a 7-day inpatient record including admission H&P, daily progress notes, consultant notes, lab results, and medication administration records" gives the model the framework to organise its output.

Specify the output format. Healthcare documents have expected structures. A discharge summary has specific sections: diagnoses, hospital course, procedures, medications at discharge, follow-up instructions. Tell the model what structure to follow, and it will follow it.

Include explicit constraints. "Do not include any clinical recommendations or treatment suggestions. Only summarise what is documented." "Flag any ICD-10 code suggestions where the clinical documentation is ambiguous — do not assign a code if the documentation does not clearly support it." These constraints keep the AI within its appropriate role.

Require source attribution. "For each finding, cite the specific note date and author from the source record." This makes human review efficient — the reviewer can trace every AI-generated statement back to the source documentation.

Here is a basic template structure for healthcare prompts:

ROLE: [Clinical documentation specialist / Coding analyst / Regulatory affairs writer]
CONTEXT: [What you are working on and why]
SOURCE DATA: [Description of what you are providing]
TASK: [Specific instructions, numbered if multi-step]
OUTPUT FORMAT: [Exact structure of the expected output]
CONSTRAINTS: [What the AI must not do]
QUALITY CHECKS: [What the AI should flag for human review]

We will apply this structure to specific healthcare workflows in Modules 3 through 6.