Impact of methodological decisions when conducting systematic reviews to inform decision making in the context of a pandemic
After the COVID‑19 pandemic started, many researchers focused on assessing treatments for this disease. Global and local authorities (such as the World Health Organization (WHO) and the Public Health Agency of Canada), as well as experts, used the emerging data to make recommendations. Because these recommendations were needed as soon as possible, the groups faced additional challenges beyond the usual in deciding how to best collect and summarize these data. Special challenges included the large amount of data available, the need for a process that was feasible and fast, and the availability of data that was published but had not undergone review by independent researchers. These potentially daunting challenges resulted in groups looking for innovative strategies that could increase their productivity by reducing the time and resources needed to collect the evidence. The risk was, however, that these strategies could increase the risk of introducing errors in the process. To date, no research group has studied the real impact of these strategies on the recommendations these groups made. We will use the data we have collected during the development of one of the largest projects that has collected and summarized the data regarding treatments for COVID-19, the « COVID-19 Living Systematic Review and Network Meta-Analysis (LNMA) ». These data inform the development of recommendations by the WHO. Because we were unwilling to accept the risk of errors, we did not adopt the possible efficiency-enhancing strategies. Perhaps, however, we should have. To find out, we will re-analyze the data to explore if the results, conclusions, and WHO recommendations would have differed if we had used some of the strategies to increase productivity. Thus, we will provide evidence about the impact of such strategies that groups of experts can use in the future in scenarios in which they have limited time or resources to collect and synthesize the evidence.
