"Going into the black box": a policy analysis of how the World Health Organization uses evidence to inform guideline recommendations.

Publisher: Frontiers Editorial Office Country of Publication: Switzerland NLM ID: 101616579 Publication Model: eCollection Cited Medium: Internet ISSN: 2296-2565 (Electronic) Linking ISSN: 22962565 NLM ISO Abbreviation: Front Public Health Subsets: MEDLINE

Original Publication: Lausanne : Frontiers Editorial Office

Evidence-Based Medicine*/methods ; Health Policy*; Policy Making ; Systematic Reviews as Topic ; World Health Organization ; Practice Guidelines as Topic

Background: The World Health Organization (WHO) plays a crucial role in producing global guidelines. In response to previous criticism, WHO has made efforts to enhance the process of guideline development, aiming for greater systematicity and transparency. However, it remains unclear whether these changes have effectively addressed these earlier critiques. This paper examines the policy process employed by WHO to inform guideline recommendations, using the update of the WHO Consolidated HIV Testing Services (HTS) Guidelines as a case study.
Methods: We observed guideline development meetings and conducted semi-structured interviews with key participants involved in the WHO guideline-making process. The interviews were recorded, transcribed, and analysed thematically. The data were deductively coded and analysed in line with the main themes from a published conceptual framework for context-based evidence-based decision making: introduction, interpretation, and application of evidence.
Results: The HTS guideline update was characterized by an inclusive and transparent process, involving a wide range of stakeholders. However, it was noted that not all stakeholders could participate equally due to gaps in training and preparation, particularly regarding the complexity of the Grading Recommendations Assessment Development Evaluation (GRADE) framework. We also found that WHO does not set priorities for which or how many guidelines should be produced each year and does not systematically evaluate the implementation of their recommendations. Our interviews revealed disconnects in the evidence synthesis process, starting from the development of systematic review protocols. While GRADE prioritizes evidence from RCTs, the Guideline Development Group (GDG) heavily emphasized "other" GRADE domains for which little or no evidence was available from the systematic reviews. As a result, expert judgements and opinions played a role in making recommendations. Finally, the role of donors and their presence as observers during GDG meetings was not clearly defined.
Conclusion: We found a need for a different approach to evidence synthesis due to the diverse range of global guidelines produced by WHO. Ideally, the evidence synthesis should be broad enough to capture evidence from different types of studies for all domains in the GRADE framework. Greater structure is required in formulating GDGs and clarifying the role of donors through the process.
Competing Interests: HI is employed by Unitaid, a hosted partnership of the World Health Organization. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
(Copyright © 2024 Ingold, Gomez, Stuckler, Vassall and Gafos.)

Health Res Policy Syst. 2006 Dec 08;4:28. (PMID: 17156460)
J Clin Epidemiol. 2011 Apr;64(4):401-6. (PMID: 21208779)
J Clin Epidemiol. 2011 Apr;64(4):395-400. (PMID: 21194891)
Health Res Policy Syst. 2018 Feb 7;16(1):7. (PMID: 29415735)
AIDS. 2014 Mar;28 Suppl 2:S205-16. (PMID: 24849480)
BMJ Glob Health. 2019 Jan 25;4(Suppl 1):e000893. (PMID: 30775016)
J Clin Epidemiol. 2017 Nov;91:23-30. (PMID: 28912002)
BMC Health Serv Res. 2017 May 11;17(1):344. (PMID: 28490325)
Soc Sci Med. 2010 Sep;71(6):1056-62. (PMID: 20678836)
BMJ Glob Health. 2019 Aug 19;4(4):e001683. (PMID: 31478014)
Public Health. 2014 May;128(5):444-74. (PMID: 24856197)
BMJ Evid Based Med. 2018 Jun;23(3):87-91. (PMID: 29615396)
BMJ. 2008 Apr 26;336(7650):924-6. (PMID: 18436948)
BMC Health Serv Res. 2014 Jan 03;14:2. (PMID: 24383766)
J Clin Epidemiol. 2016 Feb;70:123-8. (PMID: 26385188)
Health Res Policy Syst. 2006 Nov 29;4:15. (PMID: 17134482)
BMC Public Health. 2012 May 28;12:386. (PMID: 22640260)
J Clin Epidemiol. 2016 Apr;72:98-106. (PMID: 25618534)
PLoS One. 2013 May 31;8(5):e63715. (PMID: 23741299)
BMJ Glob Health. 2019 Jan 25;4(Suppl 1):e000899. (PMID: 30775017)
PLoS Med. 2014 Feb 18;11(2):e1001603. (PMID: 24558353)
J Clin Epidemiol. 2017 Nov;91:47-53. (PMID: 28911999)
BMJ Glob Health. 2019 Jan 25;4(Suppl 1):e000844. (PMID: 30775012)
Health Res Policy Syst. 2006 Nov 20;4:12. (PMID: 17116254)
Implement Sci. 2016 Jul 18;11:98. (PMID: 27430879)
J Virus Erad. 2018 Nov 15;4(Suppl 2):5-8. (PMID: 30515308)
J Clin Epidemiol. 2022 Jun;146:77-85. (PMID: 35271968)
Lancet Digit Health. 2021 Apr;3(4):e213-e216. (PMID: 33610488)
Ann Intern Med. 2012 Apr 3;156(7):525-31. (PMID: 22473437)
J Clin Epidemiol. 2014 Jun;67(6):629-34. (PMID: 24388966)
Glob Public Health. 2010;5(4):395-412. (PMID: 20155547)
Lancet Glob Health. 2017 Sep;5(9):e855-e856. (PMID: 28807174)
Implement Sci. 2013 Sep 04;8:101. (PMID: 24006933)
J Clin Epidemiol. 2018 Jun;98:1-8. (PMID: 29292204)
J Clin Epidemiol. 2013 Jul;66(7):726-35. (PMID: 23570745)
J Clin Epidemiol. 2015 Jun;68(6):703-7. (PMID: 25578218)
Lancet. 2007 Jun 2;369(9576):1883-1889. (PMID: 17493676)
BMC Pediatr. 2012 Jan 01;12:1. (PMID: 22208358)
Soc Sci Med. 2004 Jan;58(1):207-17. (PMID: 14572932)
J Clin Epidemiol. 2016 Feb;70:111-22. (PMID: 26399903)
J Clin Epidemiol. 2011 Apr;64(4):383-94. (PMID: 21195583)

Keywords: complex interventions; evidence-to-decision framework; expert opinion; public health guidelines; qualitative research; stakeholder engagement

Date Created: 20240408 Date Completed: 20240409 Latest Revision: 20240501

20240501

PMC10995388

10.3389/fpubh.2024.1292475

38584925