AI in Medication Identification: Opportunities and Challenges in Hospitals

Artificial intelligence could provide real help in medication safety, particularly in hospitals, where automated systems could significantly reduce healthcare workers' workload. However, a recently published international study warns that the accuracy of AI solutions measured in laboratories does not guarantee their performance in real-world hospital settings. According to a recent study published by researchers at the University of Pécs Faculty of Pharmacy and the University of Pannonia, the reliability of recognition is often influenced more by clinical conditions—lighting, device use, and human factors—than by the technological platform used.

We rarely stop to think—especially when we need hospital care—just how complex the process is that results in the particular pill or capsule we are given; we simply trust the healthcare system and its providers and accept that we need it. However, in everyday healthcare practice, medication identification often does not occur under ideal conditions, due to high patient volume, heavy workloads, and time constraints. For this reason, there is growing global attention on artificial intelligence, which could support medication safety through image recognition systems. However, a recent study published in JMIR Medical Informatics—conducted in collaboration between the University of Pécs Faculty of Pharmacy and the University of Pannonia Faculty of Information Technology—warns that AI can only be truly helpful if it is prepared for real-world use.

In the summer of 2024, the research team in Pécs has already reported on the development of an AI-based medication recognition system, which was tested in Pécs, Kaposvár, and Komló. The recently published study methodologically expands on this line of research: it compares four fundamentally different technological approaches using the same data to determine which performs with what levels of accuracy, cost, and error patterns in clinical practice.

From the UP Faculty of Pharmacy, the research involved Dr. Ashraf Amir Reza, senior lecturer, and Dr. András Fittler, associate professor and dean of the faculty, and from the University of Pannonia Department of Electrical Engineering and Information Systems Image Processing Research Laboratory, the participants were Dr. Zsolt Vörösházi, and Richárd Rádli research assistant and PhD student. They sought to determine whether medication identification using artificial intelligence can indeed be automated safely, cost-effectively, and reliably in everyday healthcare practice.

The study examined which artificial intelligence-based methods are suitable for the visual identification of pills and capsules in healthcare settings, such as when dispensing them at pharmacies or during clinical inspections. They compared two fundamentally different approaches in order to conduct the research. On the one hand, a code-centric, open-source image processing solution (YOLO11) was analysed, which offers a high degree of flexibility and control but requires significant developer and IT expertise. They simultaneously examined the so-called no-code or AutoML cloud-based systems, i.e., systems based on automated machine learning, of three major technology companies—Google, Amazon, and Microsoft—, which do not require the user to program in the traditional sense.

A total of thirty medications frequently used in clinical practice were included in the analysis. The individual AI models were trained on image datasets of varying sizes: the smallest training set contained 1,230 images, while the largest contained 26,880 images. The researchers aimed to explore how the performance of the algorithms changes as the amount of available data increases.

“We also examined deep learning-based systems that demonstrated accuracy rates exceeding 90 percent under controlled conditions, but their performance dropped dramatically when tested on data from a hospital setting,” said the study’s first author, Dr. Ashraf Amir Reza, senior lecturer at the UP Faculty of Pharmacy. 

According to the research, identification accuracy was often determined not by the name of the AI platform itself, but by factors such as lighting, the quality of the devices used, or how they were operated by humans.

The study paid particular attention to AutoML systems, one of whose greatest advantages is rapid implementability. However, these solutions often operate as “black boxes”: the users cannot see exactly what decisions are being made behind the scenes. The researchers also found examples where a system was so eager to avoid misidentification that it failed to recognise medications even when it should have. In practice, this kind of caution can be just as problematic as a false identification, especially when AI is intended to be introduced as a decision-support tool in healthcare.

“Our study emphasises that artificial intelligence is not an independent decision-maker, but a tool that must be used responsibly. There is no standardised solution in healthcare. Every system must be tested locally, using real-world data, and tailored to the specific environment,” highlighted Dr. Ashraf Amir Reza.

The testing was not conducted in a single, idealised environment. The systems were also tested on images from real clinical settings taken at three different hospitals, and their performance was measured using laboratory photos as well as images captured under controlled, “ideal” conditions. This approach enabled seeing not only what the algorithms are capable of under optimal conditions, but also how they behave when the hospital reality—such as varying lighting conditions, different devices, or human use—directly influences their performance.

In addition to recognition accuracy, the study also examined the stability of each system’s performance, how results vary across different environments, the costs associated with implementing these solutions, and the ease of their implementation and daily use in clinical practice, thereby identifying practical lessons relevant to medication safety.

“The knowledge gained from this research extends beyond this specific project; in the case of healthcare AI systems, neither the most advanced technology nor the largest training dataset alone can guarantee clinical reliability. Validation under local, real-world conditions is required before every implementation. Furthermore, developments should not aim to replace the professional competencies of doctors and pharmacists, but rather to provide them with maximum support in their daily workflows,” added Dr. Ashraf Amir Reza.

Article: PTE GyTK
Photo: Szabolcs Csortos