The potential and limitations of large language models in identification of the states of motivations for facilitating health behavior change.

Publisher: Oxford University Press Country of Publication: England NLM ID: 9430800 Publication Model: Print Cited Medium: Internet ISSN: 1527-974X (Electronic) Linking ISSN: 10675027 NLM ISO Abbreviation: J Am Med Inform Assoc Subsets: MEDLINE

Publication: 2015- : Oxford : Oxford University Press
Original Publication: Philadelphia, PA : Hanley & Belfus, c1993-

Motivation* ; Health Behavior*; Humans ; Transtheoretical Model ; Health Promotion

Importance: The study highlights the potential and limitations of the Large Language Models (LLMs) in recognizing different states of motivation to provide appropriate information for behavior change. Following the Transtheoretical Model (TTM), we identified the major gap of LLMs in responding to certain states of motivation through validated scenario studies, suggesting future directions of LLMs research for health promotion.
Objectives: The LLMs-based generative conversational agents (GAs) have shown success in identifying user intents semantically. Little is known about its capabilities to identify motivation states and provide appropriate information to facilitate behavior change progression.
Materials and Methods: We evaluated 3 GAs, ChatGPT, Google Bard, and Llama 2 in identifying motivation states following the TTM stages of change. GAs were evaluated using 25 validated scenarios with 5 health topics across 5 TTM stages. The relevance and completeness of the responses to cover the TTM processes to proceed to the next stage of change were assessed.
Results: 3 GAs identified the motivation states in the preparation stage providing sufficient information to proceed to the action stage. The responses to the motivation states in the action and maintenance stages were good enough covering partial processes for individuals to initiate and maintain their changes in behavior. However, the GAs were not able to identify users' motivation states in the precontemplation and contemplation stages providing irrelevant information, covering about 20%-30% of the processes.
Discussion: GAs are able to identify users' motivation states and provide relevant information when individuals have established goals and commitments to take and maintain an action. However, individuals who are hesitant or ambivalent about behavior change are unlikely to receive sufficient and relevant guidance to proceed to the next stage of change.
Conclusion: The current GAs effectively identify motivation states of individuals with established goals but may lack support for those ambivalent towards behavior change.
(© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: [email protected].)

Patient Educ Couns. 2017 Sep;100(9):1720-1729. (PMID: 28495391)
Annu Rev Public Health. 2011;32:381-98. (PMID: 21091195)
J Esthet Restor Dent. 2023 Oct;35(7):1098-1102. (PMID: 37017291)
JAMA Intern Med. 2023 Jun 1;183(6):589-596. (PMID: 37115527)
J Consult Clin Psychol. 1983 Jun;51(3):390-5. (PMID: 6863699)
NPJ Digit Med. 2024 Jan 11;7(1):6. (PMID: 38200151)
Am J Health Promot. 1997 Sep-Oct;12(1):38-48. (PMID: 10170434)
J Urol. 2023 Oct;210(4):688-694. (PMID: 37428117)
JMIR Form Res. 2021 Dec 3;5(12):e27956. (PMID: 34870611)
Patient Educ Couns. 2013 Aug;92(2):142-8. (PMID: 23763983)
JMIR Med Inform. 2016 Jan 21;4(1):e1. (PMID: 26795082)
J Dent Res. 1986 Feb;65(2):128-30. (PMID: 3455967)
JMIR Mhealth Uhealth. 2019 Dec 13;7(12):e13311. (PMID: 31833836)
J Med Internet Res. 2020 Aug 7;22(8):e17158. (PMID: 32763886)
Annu Rev Sociol. 2010 Aug 1;36:139-157. (PMID: 21921974)
JAMA. 2023 Mar 14;329(10):842-844. (PMID: 36735264)
JMIR Mhealth Uhealth. 2020 May 19;8(5):e15085. (PMID: 32427114)
J Med Internet Res. 2021 Mar 23;23(3):e24850. (PMID: 33755028)
Nat Med. 2023 Aug;29(8):1930-1940. (PMID: 37460753)
Annu Rev Clin Psychol. 2005;1:91-111. (PMID: 17716083)

Keywords: ChatGPT; Google Bard; Llama 2; behavior change; health promotion; large language models; motivations

Date Created: 20240325 Date Completed: 20240822 Latest Revision: 20240824

20240826

PMC11339501

10.1093/jamia/ocae057

38527272