Retrieval Augmented Generation (RAG) for Evaluating Regulatory Compliance of Drug Information and Clinical Trial Protocols

The purpose was to evaluate retrieval-augmented generative (RAG) artificial intelligence (AI) methods for assessing the regulatory compliance of drug information and adherence to best practices in clinical trial protocols. Integrated systems containing RAG and large language model (LLM) components were employed to evaluate drug information and clinical trial protocols. The drug information for adalimumab, insulin glargine, atorvastatin calcium, sertraline, and alprazolam was evaluated for compliance with Food and Drug Administration (FDA) clinical pharmacology guidance for indications, use in specific populations, and warnings and precautions. The reasons for the withdrawal of rofecoxib, valdecoxib, and troglitazone were elicited. The clinical trial protocol evaluation system was used to assess a Phase-2a clinical trial protocol of Rifafour in tuberculosis with the FDA E9 and E9 (R1) guidance documents. The RAG system correctly identified the indications, use in specific populations, and warnings and precautions for adalimumab, insulin glargine, atorvastatin calcium, sertraline, and alprazolam. The drug information was evaluated against the requirements in the guidance documents, confirming compliance when present and providing explanations for deficiencies. The causes underlying the withdrawal of rofecoxib, valdecoxib, and troglitazone were explained. The clinical protocol summary included study design, population definitions, treatments, dose levels, and route of administration. The summary of the statistical analysis plan included primary/secondary endpoints, statistical tests, pharmacokinetic parameters, and handling of missing data and outliers. The findings aligned with manual protocol reviews. RAG-based AI methods can improve the usefulness of LLMs in document-restricted settings and are a promising approach for evaluating the compliance of clinical pharmacology documents.