Correction (October 23, 2026): An earlier version of this article described the prompt matrix as the result of 60 hours of testing by a testing team. It reported catch rates, false-positive rates, and time savings, and it said that pharmacy employees had been interviewed. It also implied the prompts had been used in a clinical setting. Those statements were not accurate. I did not conduct that testing, collect those measurements, or interview pharmacy staff, and I have no clinical or pharmacy experience. I have removed those claims and rewritten the article as an untested design framework. I apologize for the error.
This article describes a four-layer prompt framework for using an AI chat tool to compare a formulary summary against its source document. It is a design idea, not a tested or validated method. I am a blogger, not a pharmacist, nurse, or other healthcare professional, and I have not tested these prompts in a pharmacy or clinical setting, measured their accuracy, or recorded any time savings. I used Claude, an AI assistant, to help draft this article.
The framework rests on one simple idea: ask a model to do one narrow job at a time instead of one broad job. First extract the data, then quote the source for each entry, then compare, then sort the differences for a human reviewer. The sections below explain each layer, the limits of AI tools for this kind of work, and a way to test the approach yourself on non-sensitive sample documents. Nothing here is medical, pharmacy, or compliance advice, and every output must be checked against the source by a qualified person.
What This Prompt Matrix Is Designed to Do
The aim of this framework is to give pharmacy staff a repeatable set of prompts for asking an AI tool to compare a formulary summary against its source document and produce a list of possible discrepancies. Every item on that list would still need to be checked manually against the source.
Here is a hypothetical example of why this kind of check matters. Imagine a payer publishes an updated formulary and someone condenses it into a working sheet that colleagues use as a quick reference. If that sheet differs from the source document, anyone relying on it could be working from incorrect information. The source document should always be the final authority.
This framework is not meant to replace clinical or professional judgment. It is an idea for a first-pass aid that points a human reviewer toward places where a summary and its source may differ. It has not been tested or validated, and it cannot tell a reviewer that no discrepancies exist.
The Problem the Matrix Is Meant to Address
Some formulary summaries are created manually: a person reads a long source document, pulls out the details they judge relevant, and reformats them into a spreadsheet or a short reference sheet. Any manual copy-and-condense step creates an opportunity for omissions or typing errors.
Why Manual Cross-Checking Is Demanding
Comparing an old summary, a new source document, and an updated summary line by line is detailed, repetitive work, and that kind of work is easy to get wrong. This framework explores whether an AI tool could help by flagging items for a person to look at more closely.
Known Limits of AI Tools for Document Comparison
General-purpose AI chat tools have known limitations that matter for document comparison. They can state incorrect details with confidence, including details that are not in the source text, such as a wrong tier number or coverage rule. They can also give a loose overall impression of how two documents differ rather than a precise list. And a single generic request such as “summarize this document” leaves the output format open, which makes the result harder to check. The four-layer matrix below is an attempt to design around these limits by splitting the work into narrow steps. It has not been tested, so treat it as a starting point, not a solution.
How the Matrix Is Structured
The sections below describe how the matrix is structured and how you could test it yourself.
Step 1: Understand How Formulary Documents Are Organized
Before writing prompts for formulary comparison, it helps to understand how formulary documents are organized. Formularies commonly list drugs alongside details such as tier level, quantity limits, step therapy requirements, prior authorization flags, and therapeutic category groupings. These are the fields the extraction prompts in this framework focus on. Layout and terminology vary by payer, so check the structure of the specific document you are working with.
I used prompts like these on the floor too ChatGPT Prompt Examples for Nursing Documentation is the companion piece to this one.”
Step 2: Split the Task Into Narrow Prompts
A single broad prompt, such as “compare these two documents and list all differences,” leaves the structure of the answer up to the model. This framework takes a different approach. Instead of one prompt trying to do everything, it splits the task into layers, each with one specific job. The four layers are described in the next section.
Step 3: Test It Yourself on Known Errors
If you want to evaluate this framework, you can test it yourself using non-sensitive material. Make a short summary table from a publicly available formulary excerpt, then deliberately change a few things, such as a tier number, a step therapy note, or a quantity limit. Keep a written key of every change you made. Then run the prompts and record which changes the model flagged, which it missed, and which flags were false alarms. That log is how you would find out whether the prompts work for your documents and your tool. This article has not done that testing.
The Four-Layer LLM Prompt Matrix
The framework has four layers. Each layer feeds into the next, and each has a narrow, specific job rather than trying to be a one-shot solution.
Layer One: Extraction Prompts
The first layer pulls structured data points out of the source formulary document and the summary separately. Rather than asking the model to compare anything yet, the prompt asks it to output a structured list of drug name, tier, prior authorization status, quantity limit, and step therapy requirement for each entry.
The reasoning behind keeping extraction separate from comparison is to give the model one narrow task at a time instead of asking it to extract and judge at once. This is a design rationale. This article has not measured whether it improves accuracy.
Layer Two: Source-Quote Prompts
Once both documents are extracted into the same structured format, the second layer asks the model to quote the exact line of the original text that each extracted entry came from. The purpose is to make each entry easier for a person to check. A quoted line is not proof: a model can misquote or fabricate a citation, so the reviewer must confirm each quote against the source document.
Layer Three: Discrepancy Flagging Prompts
With two structured lists in the same format, the third layer asks the model to list only the entries where the source and the summary differ, naming the field and both values. This comparison is only as reliable as the extraction and quote steps before it, and the output must still be checked against the source by a person.
Layer Four: Human Review Trigger Prompts
The fourth layer asks the model to sort the flagged differences so a reviewer knows where to start. For example, a difference in tier that changes patient cost sharing might be listed before a purely cosmetic formatting difference. The ranking criteria should be defined by qualified pharmacy staff, and a model’s ranking is only a suggestion. A difference ranked low is not safe to ignore, and every flagged item still needs review.
Choices the Matrix Makes / Approaches the Matrix Avoids (delete the labels)
Some of our assumptions going in were wrong, and it is worth being honest about that.
Confirmed
- Splitting the task into narrow layers instead of one broad prompt trying to do everything at once
- Requiring exact source-line citations for every value the model extracted
- Comparing structured lists directly to flag differences, not an open-ended summary
Discarded
- Trusting a single-pass document comparison across two long formulary documents at once
- Asking for a general “summary of differences” with no structured format to anchor it
- Skipping the verification layer to save time — hallucinated tier numbers crept straight back in
A Step-by-Step Workflow You Can Adapt
You do not need custom software to try this approach on practice material. If you work in a pharmacy, check your employer’s rules on AI tools and data handling before using any AI tool with work documents. If you do not have approval, try the workflow on public, non-sensitive sample documents instead.
Step by Step Guide
- Begin by pasting a portion of the source formulary text into the AI tool. Ask it to list each entry with these fields only: drug name, tier, prior authorization status, quantity limit, and step therapy note. Tell it to leave out everything else and to write “not stated” for any field that is missing instead of guessing.
- Next, do the same extraction for your internal summary document, using an identical structured format so the two outputs line up cleanly.
- Then ask the model to quote, for each extracted entry, the exact sentence or line it came from in the original text. Check every quote against the source document yourself. A quote from a model is not proof that the entry is correct, because models can misquote or invent text.
- After you have checked the quotes, ask the model to compare the two structured lists and output only the entries where a field differs, naming the specific field and both values.
- Finally, ask the model to sort the flagged differences by their potential effect on patient cost or access so your team knows where to start. Still review every flagged item, and have a qualified person decide which differences actually matter.
- Treat every output as a starting point for human review, never as a final answer. The aim of this approach is to point a reviewer to places where a summary and its source may differ. It is not meant to replace professional judgment or a full comparison against the source document.
What AI Cannot Promise: Catching Every Formulary Error
No tool can promise to catch every formulary error, and this framework makes no such claim. It has not been tested, so there is no catch rate to report. A person who understands formulary documents still has to check the source.
Here’s my full ChatGPT vs Claude vs Gemini comparison if you’re wondering which model handles dense PDFs best.
One idea for future exploration is connecting the prompts to retrieval-based search over the original payer source materials, so the model works from the actual source text. This has not been tried here. Even with retrieval, a model can misread or misquote a source, so human checking would still be needed.
Final Thoughts
Formulary summaries affect how easily patients can access medication, so accuracy matters. A prompt matrix will not replace the judgment of a trained pharmacist, and it should not try to. At best, it is an idea for breaking a slow manual comparison into smaller steps so a reviewer can focus on the items it flags. This article has not tested whether it does that.
If you want to explore AI for document comparison, start small and keep it safe. Use public, non-sensitive sample documents, follow your organization’s policies, and deliberately plant a few known changes to see what the prompts catch and miss. Treat the result as a learning exercise, not a verified process.
FAQ’s
Can Ai replace manually formulary verification entirely?
No. AI tools can misread, omit, or invent details, so a qualified person must check anything they produce against the source document. This article has not measured how often the prompt matrix misses or wrongly flags differences, so it offers no accuracy figure.
Is it safe to paste patient information into a Ai based tool for the process?
Do not paste patient-specific information into a general AI tool. Comparing a formulary summary with its source document does not require patient data. Before using any AI tool with work documents, check your organization’s privacy, security, and AI-use policies, or ask your compliance or privacy officer. This article cannot tell you whether a particular tool is approved.
How often should formulary summaries be cross-checked?
That depends on your organization’s procedures and on how often each payer publishes updates. Follow your pharmacy’s policies and the payer’s own update notices. This article does not set a recommended schedule.
What is the biggest single source of formulary summary errors?
This article has no data on that question. The matrix extracts tier, prior authorization status, quantity limit, and step therapy fields because these are common elements of formulary documents, not because they are known to be the most error-prone.
Does this approach work for specialty pharmacy formularies?
This article has not tested the approach on any type of formulary, specialty or otherwise. Specialty formularies may use additional fields that the four-layer matrix does not include. A qualified pharmacist should decide which fields are needed before anyone adapts it.
Rehan Malik is a blogger and the creator of PromptByJob. He publishes articles and prompt examples about using AI tools for different tasks. He uses AI-assisted drafting in his writing process. He is not a healthcare professional, and his nursing-related content should not be treated as medical advice or evidence of clinical testing.
