Integrating evidence from randomized trials and prospective observational studies: an application of GRADE Guidance 44 to nutrition evidence syntheses

Objectives The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working group recently published guidance on integrating evidence from randomized and nonrandomized studies of interventions in systematic reviews. Our study aims to explore the application of this guidance by using cohort studies in nutrition systematic reviews as replacement, complementary, or sequential evidence to randomized controlled trials (RCTs). Study Design and Setting In this empirical application of the GRADE Guidance 44, we rated the certainty of evidence (CoE) for a convenience sample of bodies of evidence from nutrition RCTs and corresponding prospective cohort studies. We determined the congruency by comparing absolute effect estimates and 95% confidence intervals, before using cohort evidence as replacement, complementary, or sequential evidence. In case of similar CoE judgments and congruency, we estimated pooled effects of RCTs and cohort studies, and assessed whether integrating cohort studies influenced the conclusions drawn from RCT evidence. Results We included 26 pairs of body of evidence, of which 21 showed congruency. In 14/26 (54%) pairs, the CoE was higher for RCTs: high vs moderate ( n = 2), high vs low ( n = 1), moderate vs low ( n = 4), moderate vs very low ( n = 3), low vs very low ( n = 4). In 5/26 (19%) pairs, the CoE was higher for cohort studies: moderate vs low ( n = 4), low vs very low ( n = 1). For six pairs (23%), both RCTs and cohort studies were rated as “low” (downgraded for risk of bias and imprecision), and due to congruency effect estimates of five pairs were pooled, mitigating imprecision in two pairs. For one pair (4%), the CoE was rated as very low, making pooling inappropriate. Conclusion This example application provides practical knowledge on the usage of GRADE Guidance 44, but the findings are subject to methodological assumptions and warrant cautious interpretation.