Network meta-analysis: a technique to gather evidence from direct and indirect comparisons

Main Article Content


Network Meta-Analysis, Evidence-Based Practice, Treatment Outcome, Decision Support Techniques


Systematic reviews and pairwise meta-analyses of randomized controlled trials, at the intersection of clinical medicine, epidemiology and statistics, are positioned at the top of evidence-based practice hierarchy. These are important tools to base drugs approval, clinical protocols and guidelines formulation and for decision-making. However, this traditional technique only partially yield information that clinicians, patients and policy-makers need to make informed decisions, since it usually compares only two interventions at the time. In the market, regardless the clinical condition under evaluation, usually many interventions are available and few of them have been studied in head-to-head studies. This scenario precludes conclusions to be drawn from comparisons of all interventions profile (e.g. efficacy and safety). The recent development and introduction of a new technique – usually referred as network meta-analysis, indirect meta-analysis, multiple or mixed treatment comparisons – has allowed the estimation of metrics for all possible comparisons in the same model, simultaneously gathering direct and indirect evidence. Over the last years this statistical tool has matured as technique with models available for all types of raw data, producing different pooled effect measures, using both Frequentist and Bayesian frameworks, with different software packages. However, the conduction, report and interpretation of network meta-analysis still poses multiple challenges that should be carefully considered, especially because this technique inherits all assumptions from pairwise meta-analysis but with increased complexity. Thus, we aim to provide a basic explanation of network meta-analysis conduction, highlighting its risks and benefits for evidence-based practice, including information on statistical methods evolution, assumptions and steps for performing the analysis. 

Abstract 4549 | PDF Downloads 2931


1. Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 2013;11:159. doi: 10.1186/1741-7015-11-159

2. Jansen JP, Trikalinos T, Cappelleri JC, Daw J, Andes S, Eldessouki R, Salanti G. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):157-173. doi: 10.1016/j.jval.2014.01.004

3. Bafeta A, Trinquart L, Seror R, Ravaud P. Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review. BMJ. 2013;347:f3675. doi: 10.1136/bmj.f3675

4. Bafeta A, Trinquart L, Seror R, Ravaud P. Reporting of results from network meta-analyses: methodological systematic review. BMJ. 2014;348:g1741. doi: 10.1136/bmj.g1741

5. Caldwell DM, Dias S, Welton NJ. Extending treatment networks in health technology assessment: how far should we go? Value Health. 2015;18(5):673-681. doi: 10.1016/j.jval.2015.03.1792

6. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159(2):130-137. doi: 10.7326/0003-4819-159-2-201307160-00008

7. Debray TP, Schuit E, Efthimiou O, Reitsma JB, Ioannidis JP, Salanti G, Moons KG, GetReal W. An overview of methods for network meta-analysis using individual participant data: when do benefits arise? S Stat Methods Med Res. 2016:962280216660741. doi: 10.1177/0962280216660741

8. Paul M, Leibovici L. Systematic review or meta-analysis? Their place in the evidence hierarchy. Clin Microbiol Infect. 2014;20(2):97-100. doi: 10.1111/1469-0691.12489

9. Leucht S, Kissling W, Davis JM. How to read and understand and use systematic reviews and meta-analyses. Acta Psychiatr Scand. 2009;119(6):443-450. doi: 10.1111/j.1600-0447.2009.01388.x

10. Mills EJ, Bansback N, Ghement I, Thorlund K, Kelly S, Puhan MA, Wright J. Multiple treatment comparison meta-analyses: a step forward into complexity. Clin Epidemiol. 2011;3:193-202. doi: 10.2147/CLEP.S16526

11. Sutton AJ, Higgins JP. Recent developments in meta-analysis. Stat Med. 2008;27(5):625-650.

12. Chalmers I. The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993;703:156-163.

13. Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making. 2013;33(5):607-617. doi: 10.1177/0272989X12458724

14. Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern Emerg Med. 2017;12(1):103-111. doi: 10.1007/s11739-016-1583-7

15. Berlin JA, Cepeda MS. Some methodological points to consider when performing systematic reviews in comparative effectiveness research. Clin Trials. 2012;9(1):27-34. doi: 10.1177/1740774511427062

16. Garg AX, Hackam D, Tonelli M. Systematic review and meta-analysis: when one study is just not enough. Clin J Am Soc Nephrol. 2008;3(1):253-260. doi: 10.2215/CJN.01430307

17. Catala-Lopez F, Tobias A, Cameron C, Moher D, Hutton B. Network meta-analysis for comparing treatment effects of multiple interventions: an introduction. Rheumatol Int. 2014;34(11):1489-1496. doi: 10.1007/s00296-014-2994-2

18. Hutton B, Salanti G, Chaimani A, Caldwell DM, Schmid C, Thorlund K, Mills E, Catala-Lopez F, Turner L, Altman DG, Moher D. The quality of reporting methods and results in network meta-analyses: an overview of reviews and suggestions for improvement. PLoS One. 2014;9(3):e92508. doi: 10.1371/journal.pone.0092508

19. Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, Boersma C, Thompson D, Larholt KM, Diaz M, Barrett A. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health. 2011;14(4):429-437. doi: 10.1016/j.jval.2011.01.011

20. Kim H, Gurrin L, Ademi Z, Liew D. Overview of methods for comparing the efficacies of drugs in the absence of head-to-head clinical trial data. Br J Clin Pharmacol. 2014;77(1):116-121. doi: 10.1111/bcp.12150

21. Fisher LD, Gent M, Buller HR. Active-control trials: how would a new agent compare with placebo? A method illustrated with clopidogrel, aspirin, and placebo. Am Heart J. 2001;141(1):26-32.

22. Pocock SJ, Gersh BJ. Do current clinical trials meet society's needs?: a critical review of recent evidence. J Am Coll Cardiol. 2014;64(15):1615-1628. doi: 10.1016/j.jacc.2014.08.008

23. Bhatnagar N, Lakshmi PV, Jeyashree K. Multiple treatment and indirect treatment comparisons: An overview of network meta-analysis. Perspect Clin Res. 2014;5(4):154-158. doi: 10.4103/2229-3485.140550

24. Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997;50(6):683-691.

25. Hasselblad V. Meta-analysis of multitreatment studies. Med Decis Making. 1998;18(1):37-43.

26. Hassan S, Ravishankar N, Nair NS. Methodological considerations in network meta-analysis. Int J Med Sci Public Health. 2015;4:588-594. doi: 10.5455/ijmsph.2015.210120151

27. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002;21(16):2313-2324.

28. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004;23(20):3105-3124.

29. Caldwell DM. An overview of conducting systematic reviews with network meta-analysis. Syst Rev. 2014;3:109. doi: 10.1186/2046-4053-3-109

30. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JPA, Straus S. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777-84. doi: 10.7326/M14-2385

31. Salanti G, Higgins JP, Ades AE, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008;17(3):279-301.

32. Veroniki AA, Vasiliadis HS, Higgins JP, Salanti G. Evaluation of inconsistency in networks of interventions. Int J Epidemiol. 2013;42(1):332-345. doi: 10.1093/ije/dys222

33. Greco T, Edefonti V, Biondi-Zoccai G, Decarli A, Gasparini M, Zangrillo A, Landoni G. A multilevel approach to network meta-analysis within a frequentist framework. Contemp Clin Trials. 2015;42:51-59. doi: 10.1016/j.cct.2015.03.005

34. Van Valkenhoef G, Dias S, Ades AE, Welton NJ. Automated generation of nodesplitting models for assessment of inconsistency in network meta-analysis. Res Synth Methods. 2016;7(1):80-93. doi: 10.1002/jrsm.1167

35. Efthimiou O, Debray TPA, Van Valkenhoef G, Trelle S, Panayidou K, Moons KGM, Reitsma JB, Shangg A, Salanti G. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods. 2016;7(3):236-263. doi: 10.1002/jrsm.1195

36. Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8(10):e76654. doi: 10.1371/journal.pone.0076654

37. Nikolakopoulou A, Mavridis D, Salanti G. Planning future studies based on the precision of network meta-analysis results. Stat Med. 2016;35(7):978-1000. doi: 10.1002/sim.6608

38. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3(2):80-97. doi: 10.1002/jrsm.1037

39. Riaz IB, Khan MS, Riaz H, Goldberg RJ. Disorganized systematic reviews and meta-analyses: time to systematize the conduct and publication of these study overviews? Am J Med. 2016;129(3):339.. doi: 10.1016/j.amjmed.2015.10.009

40. Greco T, Edefonti V, Biondi-Zoccai G, Decarli A, Gasparini M, Zangrillo A, Landoni G. A multilevel approach to network meta-analysis within a frequentist framework. Contemp Clin Trials. 2015;42:51-59. doi: 10.1016/j.cct.2015.03.005

41. Madden LV, Piepho HP, Paul PA. Statistical models and methods for network meta-analysis. Phytopathology. 2016;106(8):792-806. doi: 10.1094/PHYTO-12-15-0342-RVW

42. Ohlssen D, Price KL, Xia HA, Hong H, Kerman J, Fu H, Quartey G, Heilmann CR, Ma H, Carlin BP. Guidance on the implementation and reporting of a drug safety Bayesian network meta-analysis. Pharm Stat. 2014;13(1):55-70. doi: 10.1002/pst.1592

43. Kibret T, Richer D, Beyene J. Bias in identification of the best treatment in a Bayesian network meta-analysis for binary outcome: a simulation study. Clin Epidemiol. 2014;6:451-460. doi: 10.2147/CLEP.S69660

44. Uhlmann L, Jensen K, Kieser M. Bayesian network meta-analysis for cluster randomized trials with binary outcomes. Res Synth Methods. 2016 [Epub ahead of print] doi: 10.1002/jrsm.1210

45. Mavridis D, White IR, Higgins JP, Cipriani A, Salanti G. Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta-analysis. Stat Med. 2015;34(5):721-741. doi: 10.1002/sim.6365

46. Sturtz S, Bender R. Unsolved issues of mixed treatment comparison meta-analysis: network size and inconsistency. Res Synth Methods. 2012;3(4):300-311. doi: 10.1002/jrsm.1057

47. Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29(7-8):932-944. doi: 10.1002/sim.3767

48. Sobieraj DM, Cappelleri JC, Baker WL, Phung OJ, White CM, Coleman CI. Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review. BMJ Open. 2013 Jul 21;3(7):e003111. doi: 10.1136/bmjopen-2013-003111

49. Jansen JP, Crawford B, Bergman G, Stam W. Bayesian meta-analysis of multiple treatment comparisons: an introduction to mixed treatment comparisons. Value Health. 2008;11(5):956-964. doi: 10.1111/j.1524-4733.2008.00347.x

50. Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, Lee K, Boersma C, Annemans L, Cappelleri JC. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011;14(4):417-428. doi: 10.1016/j.jval.2011.04.002

51. Van Valkenhoef G, Lu G, Brock B, Hillege H, Ades AE, Welton NJ. Automating network meta-analysis. Res Synth Methods. 2012;3(4):285-299. doi: 10.1002/jrsm.1054

52. Greco T, Landoni G, Biondi-Zoccai G, D'Ascenzo F, Zangrillo A. A Bayesian network meta-analysis for binary outcome: how to do it. Stat Methods Med Res. 2016;25(5):1757-1773.

53. Stephenson M, Fleetwood K, Yellowlees A. Alternatives to Winbugs for network meta-analysis. Value Health. 2015;18(7):A720. doi: 10.1016/j.jval.2015.09.2730

54. Law M, Jackson D, Turner R, Rhodes K, Viechtbauer W. Two new methods to fit models for network meta-analysis with random inconsistency effects. BMC Med Res Methodol. 2016;16:87. doi: 10.1186/s12874-016-0184-5

55. Rucker G, Schwarzer G. Automated drawing of network plots in network meta-analysis. Res Synth Methods. 2016;7(1):94-107.

56. Neupane B, Richer D, Bonner AJ, Kibret T, Beyene J. Network meta-analysis using R: a review of currently available automated packages. PLoS One. 2014;9(12):e115065. doi: 10.1371/journal.pone.0115065

57. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. doi: 10.1136/bmj.b2700

Most read articles by the same author(s)