Evaluation of prompt engineering strategies for pharmacokinetic data analysis with the ChatGPT large language model.

Publisher: Springer Country of Publication: United States NLM ID: 101096520 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1573-8744 (Electronic) Linking ISSN: 1567567X NLM ISO Abbreviation: J Pharmacokinet Pharmacodyn Subsets: MEDLINE

Publication: New York : Springer
Original Publication: Bristol, England ; New York, N.Y. : Kluwer Academic/Plenum Publishers, c2001-

Data Analysis* ; Language*; Administration, Intravenous ; Injections, Intravenous

To systematically assess the ChatGPT large language model on diverse tasks relevant to pharmacokinetic data analysis. ChatGPT was evaluated with prototypical tasks related to report writing, code generation, non-compartmental analysis, and pharmacokinetic word problems. The writing task consisted of writing an introduction for this paper from a draft title. The coding tasks consisted of generating R code for semi-logarithmic graphing of concentration-time profiles and calculating area under the curve and area under the moment curve from time zero to infinity. Pharmacokinetics word problems on single intravenous, extravascular bolus, and multiple dosing were taken from a pharmacokinetics textbook. Chain-of-thought and problem separation were assessed as prompt engineering strategies when errors occurred. ChatGPT showed satisfactory performance on the report writing, code generation tasks and provided accurate information on the principles and methods underlying pharmacokinetic data analysis. However, ChatGPT had high error rates in numerical calculations involving exponential functions. The outputs generated by ChatGPT were not reproducible: the precise content of the output was variable albeit not necessarily erroneous for different instances of the same prompt. Incorporation of prompt engineering strategies reduced but did not eliminate errors in numerical calculations. ChatGPT has the potential to become a powerful productivity tool for writing, knowledge encapsulation, and coding tasks in pharmacokinetic data analysis. The poor accuracy of ChatGPT in numerical calculations require resolution before it can be reliably used for PK and pharmacometrics data analysis.
(© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

OpenAI (2023) ChatGPT (June 26 version) Large language model.
Google AI (2023) Bard Large language model.
Kimko HC, Duffull SB (2003) Simulation for designing clinical trials: a pharmacokinetic-pharmacodynamic modeling perspective drugs and the pharmaceutical sciences, vol 127. Marcel Dekker, New York.
Kimko HC, Peck CC, American Association of Pharmaceutical Scientists (2011) Clinical trial simulations: applications and trends. AAPS advances in the pharmaceutical sciences series, vol 1. AAPS Press, Springer, New York.
Bonate PL, Barrett JS, Ait-Oudhia S, Brundage R, Corrigan B, Duffull S, Gastonguay M, Karlsson MO, Kijima S, Krause A, Lovern M, Neely M, Ouellet D, Plan EL, Rao GG, Standing J, Wilkins J, Zhu H (2023) Training the next generation of pharmacometric modelers: a multisector perspective. J Pharmacokinet Pharmacodyn. https://doi.org/10.1007/s10928-023-09878-4. (PMID: 10.1007/s10928-023-09878-43784863710460728)
Michelet R, Aulin LBS, Borghardt JM, Costa TD, Denti P, Ibarra M, Ma G, Meibohm B, Pillai GC, Schmidt S, Hennig S, Kloft C (2023) Barriers to global pharmacometrics: educational challenges and opportunities across the globe. CPT Pharmacometrics Syst Pharmacol 12(6):743–747. https://doi.org/10.1002/psp4.12940. (PMID: 10.1002/psp4.129403696063210272295)
White J, Fu Q, Hays S, Sandborn M, Olea C, Gilbert H, Elnashar A, Spencer-Smith J, Schmidt DC (2023) A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv:2302:11382.
Harrold JM, Abraham AK (2014) Ubiquity: a framework for physiological/mechanism-based pharmacokinetic/pharmacodynamic model development and deployment. J Pharmacokinet Pharmacodyn 41(2):141–151. https://doi.org/10.1007/s10928-014-9352-6. (PMID: 10.1007/s10928-014-9352-624619141)
R Core Team (2017) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna.
Rowland M, Tozer TN (1995) Clinical pharmacokinetics: concepts and applications, 3rd edn. Williams & Wilkins, Baltimore.
Alkaissi H, McFarlane SI (2023) Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus J Med Sci 15(2):e35179. https://doi.org/10.7759/cureus.35179. (PMID: 10.7759/cureus.35179)
Athaluri SA, Manthena SV, Kesapragada V, Yarlagadda V, Dave T, Duddumpudi RTS (2023) Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references. Cureus 15(4):e37432. https://doi.org/10.7759/cureus.37432. (PMID: 10.7759/cureus.374323718205510173677)
Beutel G, Geerits E, Kielstein JT (2023) Artificial hallucination: GPT on LSD? Crit Care 27(1):148. https://doi.org/10.1186/s13054-023-04425-6. (PMID: 10.1186/s13054-023-04425-63707279810114308)
Gabrielsson J, Weiner D (2007) Pharmacokinetic–pharmacodynamic data analysis: concepts and applications, 4th edn. Swedish Pharmaceutical Press, Stockholm.
Gabrielsson J, Weiner D (2016) Pharmacokinetic–pharmacodynamic data analysis: concepts and applications, 5th edn. Swedish Pharmaceutical Press, Stockholm.
Wickham H (2009) ggplot2: elegant graphics for data analysis. Use R:1–212. https://doi.org/10.1007/978-0-387-98141-3.
Frieder S, Pinchetti L, Chevalier A, Griffiths R-R, Salvatori T, Lukasiewicz T, Petersen PC, Berner J (2023) Mathematical capabilities of ChatGPT. arXiv:2301.13867v13862.
Yuan Z, Yuan H, Tan C, Wang W, Huang S (2023) How well do large language models perform in arithmetic tasks? arXiv:2304.02015.
Nair R, Mohan DD, Frank S, Setlur S, Govindaraju V, Ramanathan M (2023) Generative adversarial networks for modelling clinical biomarker profiles with race/ethnicity. Br J Clin Pharmacol 89(5):1588–1600. https://doi.org/10.1111/bcp.15623. (PMID: 10.1111/bcp.1562336460305)
Nair R, Mohan DD, Setlur S, Govindaraju V, Ramanathan M (2023) Generative models for age, race/ethnicity, and disease state dependence of physiological determinants of drug dosing. J Pharmacokinet Pharmacodyn 50(2):111–122. https://doi.org/10.1007/s10928-022-09838-4. (PMID: 10.1007/s10928-022-09838-436565395)
Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, Chi EH, Le QV, Zhou D Chain-of-thought prompting elicits reasoning in large language models. In: 36th conference on neural information processing systems (NeurIPS 2022). New Orleans, LA, 2022. vol 1. NeuroIPS Foundation, pp xvi, 538.
Shakarian P, Koyyalamudi A, Ngu N, Mareedu L (2023) An independent evaluation of ChatGPT on mathematical word problems (MWP). arXiv:2302.13814v13812.
Chen J, Chen L, Huang H, Zhou T (2023) When do you need chain-of-thought prompting for ChatGPT? arXiv:2304.03262v03262.
Cloesmeijer M, Janssen A, Koopman S, Cnossen M, Mathot R (2023) ChatGPT in pharmacometrics? Potential opportunities and limitations.

Keywords: Bioavailability; ChatGPT; Drug development; Graphing; PK/PD; Pharmacokinetics; Prompt engineering

Date Created: 20231111 Date Completed: 20240401 Latest Revision: 20240401

20240401

10.1007/s10928-023-09892-6

37952004