AI can make health system pharmacy faster, safer, smarter. But adopters beware, say some experts

Alan Portnoy has a Pharm.D. and decades of experience as a clinical pharmacist, so he’s well aware of all that can go wrong when it comes to medication reconciliation. Recently, though, he learned about the problem in an all-too-personal way.

A family member was hospitalized in a community hospital that had recently been acquired by a larger health system. The health system knew — and its electronic health records (EHRs) showed — that the family member’s doctor had stopped a couple of her medications. But the newly acquired hospital had not yet switched over to the new EHR system, and therefore, clinicians there did not know the prescriptions had been stopped.

“She went into this small hospital that hadn't converted over yet, and her medications were resumed,” he says. Portnoy was able to catch the mistake and rectify it until she was moved to a rehab facility, and once again, the medications were restarted.

The episode was frustrating for Portnoy but not all that surprising.

“Healthcare is an area where, for a lot of different reasons, it has been a real challenge to be able to get precise, accurate, up-to-date, current, nonduplicated information,” says Portnoy, a clinical pharmacist with FDB (First Databank, Inc.), a technology firm that provides workflow-integrated drug databases and medication-decision support.

When it comes to medication reconciliation, a host of problems can get in the way of keeping information current. EHR systems may not be synced. Patients may forget to mention that they started an over-the-counter medication or supplement. Some patients purposely conceal that they haven’t adhered to a prescribed regimen.

There’s also messiness on the provider side, Portnoy says. EHR data are input by individual clinicians with individual nomenclatures and habits. “There are a lot of different ways to say ‘once a day,’ or ‘daily,’ or ‘QD,’” he says.

So while the world of prescriptions is one that depends on precision and accuracy, it is also one in which interpretation — interpreting instructions, interpreting doctor shorthand, interpreting the funny look on a patient’s face — frequently enters.

It turns out that interpretation — making sense of disparate information in disparate formats — is one of the things artificial intelligence (AI) does best. Except when it doesn’t.

So as AI pushes its way behind the pharmacy counter, health systems are trying to figure out how to balance the software’s competencies with the fundamentally human parts of the job.

Safety backstop

Z. Kevin Lu, Ph.D., an associate professor at the University of South Carolina’s College of Pharmacy, says one benefit of AI is that it can act as an extra layer of security to ensure patient safety.

“Machine-learning tools, particularly natural language processing and deep learning, can scan electronic prescriptions for potential issues such as incorrect doses, risky drug interactions or contraindications based on a patient’s history,” he says. “By interpreting the context behind the prescriptions, these systems can catch discrepancies that manual review might overlook.”

Portnoy noted that AI can also help identify important medical information, such as a medication change that is recorded in an EHR but not in an e-prescription.

“Sometimes it could be in a note, a discharge summary or just a comment in a progress note,” he says. “So where we see AI being able to help us is to be able to surface that.”

Moreover, AI can help analyze data from e-prescribing or medication history databases to generate insights that improve medication reconciliation.

For example, if a patient was prescribed a seven-day course of an antibiotic for a middle ear infection in July, and that medication still shows up on his medication list in October, AI systems can notice the error and recommend removing the drug from the patient’s prescriptions.

Automatic formulary substitution

AI can also be used to boost adherence to health system formularies. Portnoy says there are many different reasons formularies aren’t followed. Sometimes the clinician does not realize there’s a different recommended medication. Other times, patients will insist on staying on a product even if it’s not on their provider’s formulary. Sometimes the physicians themselves insist on a particular product.

Lu says he does not believe clinicians intentionally disregard formularies. Rather, they simply get caught up in the practical realities of their jobs.

“Clinicians often work under intense time pressure, managing many patients and navigating hundreds of medication options, which makes it easy to miss whether a medication is on the formulary,” he says.

Some hospitals have automatic formulary substitution protocols, Portnoy says, by which pharmacists are empowered to replace an off-formulary drug with a formulary drug without even calling the prescriber. Other health systems have implemented software that prompts providers whenever they try to prescribe a drug for which a formulary substitution is available.

Portnoy says such solutions have limits. “You can't do that with literally thousands of meds that are on the formulary,” he says. AI, however, can. He notes that AI can help identify formulary alternatives to nonformulary drugs, but it can also help ensure that the patient’s prescriptions are covered by their health insurance prior to their discharge from the hospital. AI can even help generate personalized patient education information about the drugs.

Forecasting demand

Another area where AI represents a major step forward, Lu says, is in managing pharmacy inventories.

“Before AI, most pharmacy inventory systems were built on fairly simple assumptions,” he says, such as the previous month’s or year’s usage. Those systems often missed real-world shifts in demand.

For example, if a health system saw a sudden spike in pneumonia cases, earlier forecasting systems would not project a need for additional antibiotics or nebulized therapies because they would be basing their forecasts on prior usage patterns, he says.

“That’s why pharmacies routinely ran out of fast-moving drugs during spikes and overstocked slow movers, because the system kept overestimating demand,” he says.

AI tools, though, can capture more nuanced information. The tools might recommend smaller orders of chemotherapy drugs, regardless of historical trends, after noticing that there are fewer chemotherapy infusion days scheduled in the upcoming months.

Not replacing pharmacists

AI’s boosters love to talk about its superhuman powers to streamline, recognize patterns, and speed up processes. But there are dangers in healthcare, where the stakes can be a matter of life or death. If a patient with a poor memory or a clinician with enigmatic shorthand can introduce faulty information into an algorithm, so might the internet.

“Large language models can really consume data from almost anywhere,” Portnoy notes. “It's not always accurate.”

That’s why Portnoy says it’s critically important to ensure the input, not just the output, of large language models is accurate. That’s where drug and device databases like FDB’s can add value by ensuring any information factored into AI-based decisions is vetted. He says EHR companies are likewise trying to develop their own large language models for clients that rely on vetted internal information.

Lu says such concerns are part of the reason there is a mix of viewpoints within the pharmacy industry about how quickly to embrace AI technology and how much “decision-making” should be delegated to machines.

“These concerns do not necessarily translate into resistance but do create a demand for thoughtful implementation, strong oversight and ongoing education to ensure AI enhances pharmacy practice rather than disrupts it,” he says.

Portnoy notes that most health systems have either a clinician or a committee in place to review any potential AI use case. “And the reason they do that is because they're concerned about any potential … hallucinations that could be caused by AI products,” he says.

For its part, the American Society of Health-System Pharmacists said in a policy statement that it sees a number of areas where AI has the potential to improve pharmacy. It does not, however, see a future in which AI could safely take over all human responsibilities.

“Operating these tools with human oversight is crucial,” they wrote. “AI should serve as a valuable aid to support the pharmacy workforce, rather than as a proxy for them.”