Reader TTS

THE EVOLUTION OF EVIDENCE SYNTHESIS Increasing interest in promoting evidence-based clinical practice has led to methodological advancements in evidence syntheses [1, 2]. Narrative reviews

have been superseded by systematic reviews, which may include meta-analysis—statistical pooling of treatment effect estimates across similar trials to improve precision [3,4,5]. Systematic

reviews minimize the risk of selection bias by considering all evidence relevant to a clinical question; however, an important limitation of conventional meta-analyses is that they only

inform treatments that have been directly compared in clinical trials. Moreover, many trials compare active interventions against placebo, usual, or standard care, whereas patients and

clinicians are typically concerned with the relative effectiveness of competing interventions. Network meta-analysis (NMA) has emerged to address these limitations by allowing for

calculation of the comparative effects of more than two competing interventions, even when they have not been directly compared in clinical trials [6, 7]. WHAT IS NETWORK META-ANALYSIS? NMA

requires the same steps as a conventional meta-analysis which include a systematic search of the literature, assessment of risk of bias among eligible trials, statistical pooling of reported

pairwise comparisons for all outcomes of interest, and assessment of the overall certainty of evidence on an outcome-by-outcome basis. This provides the “direct” evidence for treatments

that have been compared against each other, which is graphically represented by a network map. An NMA then identifies all interventions that are connected by virtue of a common comparator.

For example, two different active treatments may have been compared against placebo in different trials. An NMA allows for a theoretical trial to be created that compares these active

treatments against each other, based on their effect against a common comparator (placebo), which provides “indirect” evidence. Indirect comparisons provide an opportunity to fill knowledge

gaps within the available evidence, providing a more comprehensive understanding of treatment options for the clinician. The network estimate is the pooled result of the direct and indirect

evidence for a given comparison, or only the indirect evidence if no direct evidence is available [6, 8, 9]. Once all treatments have been compared within a network, there are different

methods for ranking treatments to convey their relative net effectiveness. Limitations and advancements in the ranking methodology will be discussed in greater detail within the example

provided below. NETWORK META-ANALYSIS IN PRACTICE An example network map on first-line medications effects on intra-ocular pressure (IOP) for primary open angle glaucoma (POAG) is shown in

Fig. 1, which represents all pharmacologic treatments that have been directly evaluated in 114 clinical trials for this condition [10]. Traditional meta-analysis would be limited in

comparing two of these treatments at a time, and could not inform the effectiveness of treatments that have not been directly compared; however, this NMA provides the relative effectiveness

of all 15 treatments in a single investigation, even when no RCT is available to make a direct comparison between two treatments. The network map uses circles, or nodes, for each included

treatment, that increase in size relative to the number of patients treated with that medication within included RCTs. The lines connecting different treatments are weighted by the number of

RCTs comparing them (i.e., thicker lines convey more direct trials) [10]. In this particular study, the authors color coded their treatment nodes by drug class to improve interpretation.

The network is specific to one outcome, in this case IOP, and the network assumes that the baseline characteristics of patients enrolled across trials are similar. As Fig. 1 demonstrates,

there are many RCTs assessing pharmacotherapy for POAG. Some treatments, such as Timolol or Latanoprost, have large bodies of evidence, while many others have far fewer – and smaller –

trials assessing their efficacy [10]. This network enables the comparison of 14 active medications, as well as placebo, for POAG. While the ability to summarize large bodies of evidence is

also possible for traditional meta-analyses, NMAs provide comparative effectiveness data between competing treatments. It is important to note that the evidence provided by an NMA is subject

to the limitations of the individual RCTs included within the network [11]. In addition, the ranking of interventions by NMAs using methods such as the Surface Under the Cumulative Ranking

Curve (SUCRA) approach is problematic – despite this currently being the most common form of treatment ranking in NMAs. This approach ranks all treatments within a network from “best” to

“worst” for each analyzed outcome, but only considers the effect estimate and not the associated precision or the certainty of evidence [12]. Thus, interventions supported by small,

low-quality trials that report large effects are ranked highly. Minimally or partially contextualized approaches, instead, consider the magnitude of effect in the context of patient

importance as well as the certainty of evidence [13, 14]. HOW CAN YOU HAVE CERTAINTY IN THE FINDINGS OF AN NMA? Like all study designs, there are considerations when evaluating the

credibility of the findings of an NMA. These include the same issues that should be considered when evaluating a traditional pairwise meta-analysis, such as the rigor of the literature

search, risk of bias among included trials, consistency of effect estimates contributing to pooled effects (heterogeneity), precision of the pooled effect estimate, publication bias, and

directness of the included evidence in relation to the primary research question [8, 9, 15, 16]. However, there are two additional considerations that are specific to NMAs: incoherence and

transitivity [8, 9, 15, 17]. Incoherence exists when the direct and indirect estimates for a comparison are not consistent with one another [6]. A meta-epidemiological study of 112 published

NMAs found inconsistent direct and indirect treatment effects in 14% of the comparisons made [18]. This means that while in most cases it is appropriate to combine indirect and direct

evidence, this is not always the case, and review authors should formally explore this issue. In the presence of incoherence, the higher certainty evidence should be presented rather than

the network estimate. If the direct and indirect effects are both supported by the same certainty of evidence, then the network estimate can be used but should be downgraded one level for

incoherence. The GRADE approach is increasingly used for rating the certainty in evidence for network estimates, which incorporates these aforementioned criteria [11, 15,16,17]. A GRADE

rating can assign high, moderate, low, or very low certainty in the evidence [11, 15,16,17]. Clinicians should take the certainty of the evidence in consideration when determining the impact

findings would have on their clinical practice, as lower certainty evidence provides less confidence in the results. Transitivity refers to the similarity between study characteristics that

allows indirect effect comparisons to be made with the assurance that there are limited factors that could modify treatment effects, aside from the intervention under investigation [6, 15].

Essentially, transitivity refers to the inclusion of studies that fundamentally address the same research questions within the same population [6]. Intransitivity can result in biased

indirect estimates, which would then impact the overall findings of the network estimates [15, 17]. As previously discussed, incoherence exists when discrepancies between direct and indirect

estimates are present, thus, transitivity is a common cause of incoherence [17]. Clinicians cannot be expected to evaluate transitivity and incoherence within an NMA and authors should

clearly report on these two important aspects. Indeed, the absence of reporting should lead readers to question the findings. Table 1 provides an example and overview of the core items for

readers to identify for critical appraisal of published NMAs, as applied to the Li et al. (2016) POAG study [10, 19]. These criteria are based on the Users’ Guides to the Medical Literature:

Essentials of Evidence-Based Clinical Practice [19]. CONCLUSION Rigorously conducted and reported NMA may provide helpful information for advancing evidence-based ophthalmology,

specifically in the common scenario in which multiple treatment options exist. However, clinicians should appraise the quality of NMAs before accepting the results, and even rigorously

conducted NMAs cannot provide high certainty evidence if the primary trials eligible for review are flawed. REFERENCES * Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a

quarter century on. Lancet Lond Engl. 2017;390:415–23. https://doi.org/10.1016/S0140-6736(16)31592-6. Article  Google Scholar  * Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred

reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097. Article  PubMed  PubMed Central 

Google Scholar  * Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315:1533–7. https://doi.org/10.1136/bmj.315.7121.1533. Article  CAS  PubMed  PubMed

Central  Google Scholar  * Crowther M, Lim W, Crowther MA. Systematic review and meta-analysis methodology. Blood. 2010;116:3140–6. https://doi.org/10.1182/blood-2010-05-280883. Article  CAS

  PubMed  Google Scholar  * Leucht S, Kissling W, Davis JM. How to read and understand and use systematic reviews and meta-analyses. Acta Psychiatr Scand. 2009;119:443–50.

https://doi.org/10.1111/j.1600-0447.2009.01388.x. Article  CAS  PubMed  Google Scholar  * Mills E, Thorlund K, Ioannidis J. Demystifying trial networks and network meta-analysis. BMJ.

2013;346:f2914 https://doi.org/10.1136/bmj.f2914. Article  PubMed  Google Scholar  * Neupane B, Richer D, Bonner AJ, Kibret T, Beyene J. Network meta-analysis using r: a review of currently

available automated packages. PLOS ONE. 2014;9:e115065. https://doi.org/10.1371/journal.pone.0115065. Article  CAS  PubMed  PubMed Central  Google Scholar  * Foote CJ, Chaudhry H, Bhandari

M, Thabane L, Furukawa TA, Petrisor B, et al. Network meta-analysis: users’ guide for surgeons: Part I – credibility. Clin Orthop Relat Res. 2015;473:2166–71.

https://doi.org/10.1007/s11999-015-4286. Article  PubMed  PubMed Central  Google Scholar  * Chaudhry H, Foote CJ, Guyatt G, Thabane L, Furukawa TA, Petrisor B, et al. Network meta-analysis:

users’ guide for surgeons: Part II - certainty. Clin Orthop. 2015;473:2172–8. https://doi.org/10.1007/s11999-015-4287-9. Article  PubMed  PubMed Central  Google Scholar  * Li T, Lindsley K,

Rouse B, Hong H, Shi Q, Friedman DS, et al. Comparative effectiveness of first-line medications for primary open-angle glaucoma: a systematic review and network meta-analysis. Ophthalmology.

2016;123:129–40. https://doi.org/10.1016/j.ophtha.2015.09.005. Article  PubMed  Google Scholar  * Puhan MA, Schünemann HJ, Murad MH, Li T, Brignardello-Petersen R, Singh JA, et al. A GRADE

Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ. 2014;349:g5630. https://doi.org/10.1136/bmj.g5630. Article  PubMed  Google

Scholar  * Mbuagbaw L, Rochwerg B, Jaeschke R, Heels-Andsell D, Alhazzani W, Thabane L, et al. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev.

2017;6:1–5. https://doi.org/10.1186/s13643-017-0473-z. Article  Google Scholar  * Brignardello-Petersen R, Florez ID, Izcovich A, Santesso N, Hazlewood G, Alhazanni W, et al. GRADE approach

to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ. 2020;371:m3900. https://doi.org/10.1136/bmj.m3900 * Brignardello-Petersen R, Izcovich A,

Rochwerg B, Florez ID, Hazlewood G, Alhazanni W, et al. GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework. BMJ. 2020;371:m3907.

https://doi.org/10.1136/bmj.m3907. Article  PubMed  Google Scholar  * Brignardello-Petersen R, Bonner A, Alexander PE, Siemieniuk RA, Furukawa TA, Rochwerg B, et al. Advances in the GRADE

approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol. 2018;93:36–44. https://doi.org/10.1016/j.jclinepi.2017.10.005. Article  PubMed  Google Scholar  *

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.

https://doi.org/10.1016/j.jclinepi.2010.04.026. Article  PubMed  Google Scholar  * Brignardello-Petersen R, Mustafa RA, Siemieniuk RAC, Murad MH, Agoritsas T, Izcovich A, et al. GRADE

approach to rate the certainty from a network meta-analysis: addressing incoherence. J Clin Epidemiol. 2019;108:77–85. https://doi.org/10.1016/j.jclinepi.2018.11.025. Article  PubMed  Google

Scholar  * Song F, Xiong T, Parekh-Bhurke S, Loke YK, Sutton AJ, Eastwood AJ, et al. Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological

study. BMJ. 2011;343:d4909. https://doi.org/10.1136/bmj.d4909. Article  PubMed  PubMed Central  Google Scholar  * Mills E, Ioannidis JPA, Thorlund K, Schünemann HJ, Puhan MA, Guyatt GH.

Chapter 24: network meta-analysis. In: Guyatt G, Rennie D, Meade MO, Cook DJ (eds). Users’ guides to the medical literature: a manual for evidence-based clinical practice. 3rd ed. McGraw

Hill Education; 2015. 327–56. Download references AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton,

ON, Canada Mark R. Phillips, Jason W. Busse, Lehana Thabane, Mohit Bhandari & Varun Chaudhary * Sunderland Eye Infirmary, Sunderland, UK David H. Steel * Biosciences Institute, Newcastle

University, Newcastle Upon Tyne, UK David H. Steel * Retina Consultants of Texas (Retina Consultants of America), Houston, TX, USA Charles C. Wykoff * Blanton Eye Institute, Houston

Methodist Hospital, Houston, TX, USA Charles C. Wykoff * Michael G. DeGroote National Pain Center, McMaster University, Hamilton, ON, Canada Jason W. Busse * Department of Anesthesia,

Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada Jason W. Busse * Center for Treatment Comparison and Integrative Analysis, Division of Rheumatology, Tufts Medical

Center, Boston, MA, USA Raveendhara R. Bannuru * Biostatistics Unit, St. Joseph’s Healthcare-Hamilton, Hamilton, ON, Canada Lehana Thabane * Department of Surgery, McMaster University,

Hamilton, ON, Canada Mohit Bhandari & Varun Chaudhary * NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, UK Sobha Sivaprasad * Cole Eye Institute, Cleveland

Clinic, Cleveland, OH, USA Peter Kaiser * Retinal Disorders and Ophthalmic Genetics, Stein Eye Institute, University of California, Los Angeles, CA, USA David Sarraf * Department of

Ophthalmology, Mayo Clinic, Rochester, MN, USA Sophie J. Bakri * The Retina Service at Wills Eye Hospital, Philadelphia, PA, USA Sunir J. Garg * Center for Ophthalmic Bioinformatics, Cole

Eye Institute, Cleveland Clinic, Cleveland, OH, USA Rishi P. Singh * Cleveland Clinic Lerner College of Medicine, Cleveland, OH, USA Rishi P. Singh * Department of Ophthalmology, University

of Bonn, Bonn, Germany Frank G. Holz * Singapore Eye Research Institute, Singapore, Singapore Tien Y. Wong * Singapore National Eye Centre, Duke-NUD Medical School, Singapore, Singapore Tien

Y. Wong * Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia Robyn H. Guymer * Department of Surgery (Ophthalmology), The University of

Melbourne, Melbourne, Australia Robyn H. Guymer Authors * Mark R. Phillips View author publications You can also search for this author inPubMed Google Scholar * David H. Steel View author

publications You can also search for this author inPubMed Google Scholar * Charles C. Wykoff View author publications You can also search for this author inPubMed Google Scholar * Jason W.

Busse View author publications You can also search for this author inPubMed Google Scholar * Raveendhara R. Bannuru View author publications You can also search for this author inPubMed 

Google Scholar * Lehana Thabane View author publications You can also search for this author inPubMed Google Scholar * Mohit Bhandari View author publications You can also search for this

author inPubMed Google Scholar * Varun Chaudhary View author publications You can also search for this author inPubMed Google Scholar CONSORTIA FOR THE RETINA EVIDENCE TRIALS INTERNATIONAL

ALLIANCE (R.E.T.I.N.A.) STUDY GROUP * Varun Chaudhary * , Mohit Bhandari * , Charles C. Wykoff * , Sobha Sivaprasad * , Lehana Thabane * , Peter Kaiser * , David Sarraf * , Sophie J. Bakri *

, Sunir J. Garg * , Rishi P. Singh * , Frank G. Holz * , Tien Y. Wong *  & Robyn H. Guymer CONTRIBUTIONS MRP was responsible for writing, conception of idea, critical review and

feedback on manuscript. DHS was responsible for critical review and feedback on manuscript. CCW was responsible for critical review and feedback on manuscript. JWB was responsible for

critical review and feedback on manuscript. RRB was responsible for critical review and feedback on manuscript. LT was responsible for critical review and feedback on manuscript. MB was

responsible for conception of idea, critical review and feedback on manuscript. VC was responsible for conception of idea, critical review and feedback on manuscript. CORRESPONDING AUTHOR

Correspondence to Varun Chaudhary. ETHICS DECLARATIONS COMPETING INTERESTS MRP: Nothing to disclose. DHS: Consultant: Gyroscope, Roche, Alcon, BVI; Research Funding for IIS: Alcon, Bayer,

DORC, Gyrsocope, Boehringer-Ingelheim – unrelated to this study. CCW: Consultant: Acuela, Adverum Biotechnologies, Inc, Aerpio, Alimera Sciences, Allegro Ophthalmics, LLC, Allergan, Apellis

Pharmaceuticals, Bayer AG, Chengdu Kanghong Pharmaceuticals Group Co, Ltd, Clearside Biomedical, DORC (Dutch Ophthalmic Research Center), EyePoint Pharmaceuticals, Gentech/Roche,

GyroscopeTx, IVERIC bio, Kodiak Sciences Inc, Novartis AG, ONL Therapeutics, Oxurion NV, PolyPhotonix, Recens Medical, Regeron Pharmaceuticals, Inc, REGENXBIO Inc, Santen Pharmaceutical Co,

Ltd, and Takeda Pharmaceutical Company Limited; Research funds: Adverum Biotechnologies, Inc, Aerie Pharmaceuticals, Inc, Aerpio, Alimera Sciences, Allergan, Apellis Pharmaceuticals, Chengdu

Kanghong Pharmaceutical Group Co, Ltd, Clearside Biomedical, Gemini Therapeutics, Genentech/Roche, Graybug Vision, Inc, GyroscopeTx, Ionis Pharmaceuticals, IVERIC bio, Kodiak Sciences Inc,

Neurotech LLC, Novartis AG, Opthea, Outlook Therapeutics, Inc, Recens Medical, Regeneron Pharmaceuticals, Inc, REGENXBIO Inc, Samsung Pharm Co, Ltd, Santen Pharmaceutical Co, Ltd, and Xbrane

Biopharma AB – unrelated to this study. JWB: Nothing to disclose. RRB: Research funds: Pfizer – unrelated to this study. LT: Nothing to disclose. MB: Research funds: Pendopharm, Bioventus,

Acumed – unrelated to this study. VC: Advisory Board Member: Alcon, Roche, Bayer, Novartis; Grants: Bayer, Novartis – unrelated to this study. ADDITIONAL INFORMATION PUBLISHER’S NOTE

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE

THIS ARTICLE Phillips, M.R., Steel, D.H., Wykoff, C.C. _et al._ A clinician’s guide to network meta-analysis. _Eye_ 36, 1523–1526 (2022). https://doi.org/10.1038/s41433-022-01943-5 Download

citation * Received: 02 January 2022 * Revised: 07 January 2022 * Accepted: 17 January 2022 * Published: 10 February 2022 * Issue Date: August 2022 * DOI:

https://doi.org/10.1038/s41433-022-01943-5 SHARE THIS ARTICLE Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not

currently available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative