Using GRADE evidence to decision frameworks to choose from multiple interventions

Background and Objective: Guideline development groups or other health care decision makers frequently encounter situations that require a simultaneous comparison of multiple interventions. This sometimes becomes apparent either when they identify questions of interest, before they formulate recommendations, or it may surface only when recommendations have already been formulated based on pairwise comparisons. Methods: Using examples from the World Health Organization, the European Commission, and a professional society, we developed a flexible approach to developing recommendations when a multiple-intervention comparison (MC) is needed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to decision (EtD) frameworks. We iteratively refined this approach through user testing and then included a module in GRADE's official software GRADEpro to test the approach in two real and one theoretical guideline recommendations. Results: We found the approach feasible and that all EtD criteria should be considered in an MC approach. We judged that guideline development groups and other decision makers will benefit from the availability of a network meta-analyses (NMA) of intervention effects to support decisions; however, NMA supports only one of many criteria, that is, the balance of health benefits and harms, and is therefore helpful, but not essential to the approach we propose. When similar but not identical comparators are used to address MC, challenges may arise with intransitivity and the relative rankings of interventions. Conclusion: We successfully applied the MC approach and software module in generating recommendations across different scenarios and identified challenges. The MC approach allows guideline groups and other decision makers to transparently and critically assess multiple options for a given health question. Application of the approach by others may lead to refinement and allow for better understanding of its impact in developing recommendations and making choices.