Letter in Reply

Journal of Comparative Effectiveness ResearchVol. 10, No. 17 Letter in ReplyFree AccessLetter in replyImtiaz A Samjoo, Evelyn Worthington, Christopher Drudge, Melody Zhao, Chris Cameron, Dieter A Häring, Dee Stoneman, Luisa Klotz & Nicholas AdlardImtiaz A Samjoo https://orcid.org/0000-0003-1415-8055EVERSANA™, 204-3228 South Service Road, Burlington, Ontario, L7N 3H8, CanadaSearch for more papers by this author, Evelyn Worthington https://orcid.org/0000-0003-1659-2082EVERSANA™, 204-3228 South Service Road, Burlington, Ontario, L7N 3H8, CanadaSearch for more papers by this author, Christopher Drudge https://orcid.org/0000-0001-9721-3069EVERSANA™, 204-3228 South Service Road, Burlington, Ontario, L7N 3H8, CanadaSearch for more papers by this author, Melody Zhao https://orcid.org/0000-0002-3891-5367EVERSANA™, 204-3228 South Service Road, Burlington, Ontario, L7N 3H8, CanadaSearch for more papers by this author, Chris Cameron *Author for correspondence: E-mail Address: chris.cameron@eversana.comhttps://orcid.org/0000-0003-3613-760XEVERSANA™, 207-275 Charlotte Street, Sydney, Nova Scotia, B1P 1C6, CanadaSearch for more papers by this author, Dieter A HäringNovartis Pharma AG, Basel, SwitzerlandSearch for more papers by this author, Dee StonemanNovartis Pharma AG, Basel, SwitzerlandSearch for more papers by this author, Luisa KlotzDepartment of Neurology, University Hospital Münster, Westfälische-Wilhelms-University Münster, Münster, GermanySearch for more papers by this author & Nicholas Adlard https://orcid.org/0000-0001-6912-1685Novartis Pharma AG, Basel, SwitzerlandSearch for more papers by this authorPublished Online:5 Oct 2021https://doi.org/10.2217/cer-2021-0203AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinkedInRedditEmail Keywords: disease-modifying therapiesindirect treatment comparisonnetwork meta-analysisofatumumabrelapsing multiple sclerosisLetter in response to: 'Letter to the Editor: comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis' [1].We appreciate the insightful comments from the author(s) of the Letter to the Editor regarding our recent publication, ‘Comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis’ [2]. The Letter author(s) highlighted several important considerations that are common when conducting and reporting a network meta-analysis (NMA).The author(s) of the Letter commented that greater distances between treatments and a small number of trials per treatment comparison in NMA networks can increase the risk of violating key assumptions of NMAs and introduce bias into derived effect estimates. They also commented that although population-adjusted indirect comparisons are possible alternatives to NMAs and can provide a valid estimate of relative treatment effects, they rely on their own set of important assumptions that must be acknowledged. A key assumption underlying an NMA is the similarity of included trials with regards to effect modifiers. Violating this assumption undermines the validity of the NMA results and can lead to misleading conclusions. It is possible to limit the impact of cross-trial differences in effect modifiers by ensuring compliance with relevant NMA conduct and reporting guidance documents [3–5] and by carefully assessing NMA feasibility prior to undertaking such analyses to ensure the evidence base is sufficiently similar, as we did for our study (in the ‘Feasibility assessment’ section of the Results). In our publication, we noted as a limitation the presence of greater distances between treatments in the NMA networks because not all trials centered around a single common comparator (paragraph 11, lines 12–14 of the discussion) which is often the case in NMAs consisting of large networks. We also noted the small number of trials per treatment comparison in the networks as a limitation of our analysis, as this precluded using meta regression to adjust for potential effect modifiers (paragraph 3, lines 1–3 of the discussion). We believe that these stated limitations cover in sufficient detail the concerns raised by the author(s) of the Letter and agree with the author(s) that these limitations can increase the risk of violating the key assumptions of the NMA, which could bias effect estimates, as was clearly noted in our publication. We also note that these limitations are shared by the numerous other recently published NMAs evaluating the comparative efficacy of disease-modifying therapies (DMTs) for relapsing multiple sclerosis, as they are inherent to the treatment and clinical trial landscape for this disease [6–12]. Our study generally resembled these other recently published NMAs in terms of the evidence base and methodology used for the analyses. Our results regarding the comparative efficacy of ofatumumab and other monoclonal antibody therapies have also been corroborated by an NMA publication from an independent group [9], demonstrating that the limitations we outlined in our publication did not preclude generating NMA results that aligned with similar published studies. As described in our publication, we conducted multiple sensitivity analyses to investigate the influence of potential effect modifiers and found these results were consistent with our base case analyses, indicating that the investigated factors were not important effect modifiers. We agree that population-adjusted indirect comparisons can help address the identified network structure limitations. However, as we described in our publication (paragraph 3, lines 9–14 of the discussion), these methods only permit pairwise comparisons and so are not practical in therapeutic areas where multiple treatments exist (i.e., for large evidence base networks) such as those in our study. Further, NMAs are more widely accepted by health technology assessment agencies than population-adjusted indirect comparisons. We agree with the author(s) of the Letter that population-adjusted indirect comparisons come with their own set of strong assumptions that must be clearly expressed and investigated when undertaking such analyses.The Letter author(s) identified several concerns regarding the OPERA-aligned definition we used for the confirmed disability progression (CDP) outcomes analyzed in our study. The CDP definitions differed across trials included in the NMA based on the required increases in Expanded Disability Status Scale score required for progression and likely other infrequently reported definition components as well. Our rederivation of CDP data for the ASCLEPIOS I/II trials to align with the definition reported for the OPERA I/II trials was indeed a post hoc analysis and not prospectively registered. Only one OPERA-aligned definition was considered for our study based on what was publicly reported for OPERA I/II [13]. The OPERA-aligned CDP data for ASCLEPIOS I and ASCLEPIOS II were provided in our publication (Supplementary Tables 3.2 & 3.3 & Supplementary Figures 3.1 & 3.2). However, we acknowledge that the corresponding pooled data for ASCLEPIOS I/II were previously presented at investor relations calls that were not referenced in our publication. The Letter author(s) aptly note that making outcome definitions more consistent across trials included in an NMA can reduce bias; however, the OPERA-aligned CDP definition could only be applied to the ASCLEPIOS I/II trials, meaning that CDP data for the several linking trials between ASCLEPIOS I/II (ofatumumab) and OPERA I/II (ocrelizumab) was not based on a consistent definition. In our study, we assessed the impact of using rederived post hoc OPERA-aligned CDP data for the ASCLEPIOS I/II trials by conducting NMAs using CDP data based on the prespecified per protocol CDP definition for these trials. The results of this analysis were reported in our publication alongside those for the NMA using the OPERA-aligned CDP definition (Figures 4 & 6). Both analyses were given equal footing in our publication, and we intentionally did not refer to either analysis as a base case. The results of the two analyses were generally consistent for both the 3 and 6-month CDP outcomes. Ideally, the effect of using different CDP definitions in the NMAs would be assessed using additional sensitivity analyses with CDP data for all DMTs (i.e., all trials included in the NMA) using the OPERA-aligned definition or using the ASCLEPIOS-defined definition. However, as acknowledged by the author(s) of the Letter, without access to patient level data for all included trials these analyses are not possible.The Letter author(s) suggested that our NMAs of discontinuation or adverse event (AE) related outcomes belonged in the main text instead of the appendix. The principal aims of DMTs for multiple sclerosis, as reflected in the primary scope and end points for the trials included in our NMA networks, are to slow progression of disability, reduce the number and severity of relapses and reduce the impact of the disease on quality of life. With the recent emergence of multiple DMTs, the choices for healthcare providers and patients are now even greater. However, this also makes clinical treatment management rather complicated and complex. The choice of DMT for each patient depends on many factors, including disease activity, socioeconomic status, living environment and considerations for safety, adherence, tolerability, flexibility and route of administration/use and patient preference. We agree with the Letter author(s) that DMT selection requires consideration of the balance between the efficacy and safety/tolerability profiles, which vary across therapies. This cross trial heterogeneity for discontinuation or AE related outcomes contributes to several confounding factors, which we noted in our publication (in the ‘Additional analyses’ section of the Materials & methods and in Appendix G where the discontinuation or AE related NMAs were provided). Specifically: induction therapies tend to have a low rate of discontinuation due to the inpatient administration schedule (as opposed to because they are safer or more tolerable); many trials are not adequately powered to analyze safety/tolerability outcomes given their placement on the statistical hierarchy; recorded AEs reflect only those occurring during the study period (and so do not provide a meaningful understanding of long-term safety/tolerability); and it can be very difficult to interpret quantitative AE related outcomes when safety/tolerability profiles between therapies are very dissimilar and outcome definitions are infrequently reported (e.g., most trials included in our NMAs did not report definitions for AE or serious AE). Given these limitations, we believe quantitative analyses incorporating discontinuation and AE related outcomes may lead to misinterpretation of the NMA; and hence, could be potentially misleading to the reader should they be presented alongside the efficacy outcome NMAs. For these reasons, in our publication we clearly stated the limitations of using NMA to assess discontinuation and AE related outcomes and provided the full NMA results in the publication appendix. We encourage readers to consider the totality of the evidence presented in our publication (incorporating both efficacy and harms) when determining individualized treatment and believe we have sufficiently and transparently presented this evidence in our publication.Given the importance of NMAs for healthcare decision making, we agree with the author(s) Letter that it is imperative that reporting of NMA methods, results and assumptions are clear and transparent to allow accurate interpretation of findings. We believe the limitations of our analyses, both methodological limitations of the NMAs and data limitations related to efficacy and/or safety/tolerability outcomes, have been clearly described and are fully transparent and balanced throughout our publication. Although NMAs allow for the comparison of products where no direct evidence exists, it is important to note that they should be used as only one tool, among many others, to assist decision making by payers and other stakeholders. Patient characteristics and preferences, clinical heterogeneity of disease course, unmet need and many more factors should all be considered when deciding on individual treatment decisions.Financial & competing interests disclosureThis work was funded by Novartis Pharma AG. IA Samjoo, E Worthington, C Drudge, M Zhao and C Cameron are employees of EVERSANA™. EVERSANA receives consultancy fees from major pharmaceutical and device companies, including Novartis Pharma AG. C Cameron is also a shareholder of EVERSANA. DA Häring, D Stoneman and N Adlard are salaried employees of Novartis Pharma AG. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the letter apart from those disclosed.No writing assistance was utilized in the production of this letter.References1. Parekh K, Watkins C. Letter to the Editor: comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis. J. Comp. Eff. Res. 10(17), 1265–1266 (2021).Abstract, Google Scholar2. Samjoo IA, Worthington E, Drudge C et al. Comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis. J. Comp. Eff. Res. 9(18), 1255–1274 (2020).Link, Google Scholar3. Hoaglin DC, Hawkins N, Jansen JP et al. Conducting indirect treatment comparison and network meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health 14(4), 429–437 (2011).Crossref, Medline, Google Scholar4. Jansen JP, Fleurence R, Devine B et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health 14(4), 417–428 (2011).Crossref, Medline, Google Scholar5. Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 11(1), 159 (2013).Crossref, Medline, Google Scholar6. Giovannoni G, Lang S, Wolff R et al. A systematic review and mixed treatment comparison of pharmaceutical interventions for multiple sclerosis. Neurol. Ther. 9(2), 359–374 (2020).Crossref, Medline, Google Scholar7. Hamidi V, Couto E, Ringerike T, Klemp M. A multiple treatment comparison of eleven disease-modifying drugs used for multiple sclerosis. J. Clin. Med. Res. 10(2), 88–105 (2018).Crossref, Medline, Google Scholar8. Li H, Hu F, Zhang Y, Li K. Comparative efficacy and acceptability of disease-modifying therapies in patients with relapsing-remitting multiple sclerosis: a systematic review and network meta-analysis. J. Neurol. 267(12), 3489–3498 (2020).Crossref, Medline, Google Scholar9. Liu Z, Liao Q, Wen H, Zhang Y. Disease modifying therapies in relapsing-remitting multiple sclerosis: a systematic review and network meta-analysis. Autoimmun. Rev. 20(6), 102826 (2021).Crossref, Medline, CAS, Google Scholar10. Lucchetta RC, Tonin FS, Borba HHL et al. Disease-modifying therapies for relapsing-remitting multiple sclerosis: a network meta-analysis. CNS Drugs 32(9), 813–826 (2018).Crossref, Medline, Google Scholar11. McCool R, Wilson K, Arber M et al. Systematic review and network meta-analysis comparing ocrelizumab with other treatments for relapsing multiple sclerosis. Multiple Scler. Relat. Disord. 29, 55–61 (2019).Crossref, Medline, Google Scholar12. Siddiqui MK, Khurana IS, Budhia S, Hettle R, Harty G, Wong SL. Systematic literature review and network meta-analysis of cladribine tablets versus alternative disease-modifying treatments for relapsing-remitting multiple sclerosis. Curr. Med. Res. Opin. 34(8), 1361–1371 (2018).Crossref, Medline, CAS, Google Scholar13. Hauser SL, Bar-Or A, Comi G et al. Ocrelizumab versus interferon beta-1a in relapsing multiple sclerosis. N. Engl. J. Med. 376(3), 221–234 (2017).Crossref, Medline, CAS, Google ScholarFiguresReferencesRelatedDetails Vol. 10, No. 17 Follow us on social media for the latest updates Metrics Downloaded 355 times History Received 23 August 2021 Accepted 26 August 2021 Published online 5 October 2021 Published in print December 2021 Information© 2021 Future Medicine LtdKeywordsdisease-modifying therapiesindirect treatment comparisonnetwork meta-analysisofatumumabrelapsing multiple sclerosisFinancial & competing interests disclosureThis work was funded by Novartis Pharma AG. IA Samjoo, E Worthington, C Drudge, M Zhao and C Cameron are employees of EVERSANA™. EVERSANA receives consultancy fees from major pharmaceutical and device companies, including Novartis Pharma AG. C Cameron is also a shareholder of EVERSANA. DA Häring, D Stoneman and N Adlard are salaried employees of Novartis Pharma AG. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the letter apart from those disclosed.No writing assistance was utilized in the production of this letter.PDF download