Using clinical practice successfully for comparative effectiveness research: Lessons learned from surgical prehabilitation research.

Interest is growing for prehabilitation, an intervention to optimize preoperative functional capacity and resilience [[1]Molenaar C.J.L. Minnella E.M. Coca-Martinez M. ten Cate D.W.G. Regis M. Awasthi R. et al.Effect of multimodal prehabilitation on reducing postoperative complications and enhancing functional capacity following colorectal cancer surgery: the PREHAB randomized clinical trial.JAMA Surg. 2023; 158: 572-581Crossref PubMed Scopus (14) Google Scholar]. However, the effectiveness of this intervention in daily clinical practice is unknown [2Falz R. Bischoff C. Thieme R. Lässing J. Mehdorn M. Stelzner S. et al.Effects and duration of exercise-based prehabilitation in surgical therapy of colon and rectal cancer: a systematic review and meta-analysis.J Cancer Res Clin Oncol. 2022; 148: 2187-2213Crossref PubMed Scopus (12) Google Scholar, 3Heger P. Probst P. Wiskemann J. Steindorf K. Diener M.K. Mihaljevic A.L. A systematic review and Meta-analysis of physical exercise Prehabilitation in major abdominal surgery (PROSPERO 2017 CRD42017080366).J Gastrointest Surg. 2020; 24: 1375-1385Crossref PubMed Scopus (85) Google Scholar, 4Molenaar C.J. van Rooijen S.J. Fokkenrood H.J. Roumen R.M. Janssen L. Slooter G.D. Prehabilitation versus no prehabilitation to improve functional capacity, reduce postoperative complications and improve quality of life in colorectal cancer surgery.Cochrane Database Syst Rev. 2022; 5 (Cd013259)PubMed Google Scholar, 5Zhang X. Wang S. Ji W. Wang H. Zhou K. Jin Z. et al.The effect of prehabilitation on the postoperative outcomes of patients undergoing colorectal surgery: a systematic review and meta-analysis.Front Oncol. 2022; 12958261Google Scholar]. Previous studies were heterogeneous in terms of study design, intervention composition (including content, duration, intensity and location of the program) and target population. Moreover, all studies were hampered by limited sample size [2Falz R. Bischoff C. Thieme R. Lässing J. Mehdorn M. Stelzner S. et al.Effects and duration of exercise-based prehabilitation in surgical therapy of colon and rectal cancer: a systematic review and meta-analysis.J Cancer Res Clin Oncol. 2022; 148: 2187-2213Crossref PubMed Scopus (12) Google Scholar, 3Heger P. Probst P. Wiskemann J. Steindorf K. Diener M.K. Mihaljevic A.L. A systematic review and Meta-analysis of physical exercise Prehabilitation in major abdominal surgery (PROSPERO 2017 CRD42017080366).J Gastrointest Surg. 2020; 24: 1375-1385Crossref PubMed Scopus (85) Google Scholar, 4Molenaar C.J. van Rooijen S.J. Fokkenrood H.J. Roumen R.M. Janssen L. Slooter G.D. Prehabilitation versus no prehabilitation to improve functional capacity, reduce postoperative complications and improve quality of life in colorectal cancer surgery.Cochrane Database Syst Rev. 2022; 5 (Cd013259)PubMed Google Scholar, 5Zhang X. Wang S. Ji W. Wang H. Zhou K. Jin Z. et al.The effect of prehabilitation on the postoperative outcomes of patients undergoing colorectal surgery: a systematic review and meta-analysis.Front Oncol. 2022; 12958261Google Scholar]. The Dutch PreColo comparative effectiveness research (PreColo-CER) aimed to answer the question on the effectiveness of prehabilitation in specifically older patients with colorectal cancer. The research question of the PreColo-CER was considered relevant and urgent because of the contradictory evidence on the effectiveness of prehabilitation, the large number of patients involved and the potential cost reduction that structural implementation of prehabilitation could entail [2Falz R. Bischoff C. Thieme R. Lässing J. Mehdorn M. Stelzner S. et al.Effects and duration of exercise-based prehabilitation in surgical therapy of colon and rectal cancer: a systematic review and meta-analysis.J Cancer Res Clin Oncol. 2022; 148: 2187-2213Crossref PubMed Scopus (12) Google Scholar, 3Heger P. Probst P. Wiskemann J. Steindorf K. Diener M.K. Mihaljevic A.L. A systematic review and Meta-analysis of physical exercise Prehabilitation in major abdominal surgery (PROSPERO 2017 CRD42017080366).J Gastrointest Surg. 2020; 24: 1375-1385Crossref PubMed Scopus (85) Google Scholar, 4Molenaar C.J. van Rooijen S.J. Fokkenrood H.J. Roumen R.M. Janssen L. Slooter G.D. Prehabilitation versus no prehabilitation to improve functional capacity, reduce postoperative complications and improve quality of life in colorectal cancer surgery.Cochrane Database Syst Rev. 2022; 5 (Cd013259)PubMed Google Scholar, 5Zhang X. Wang S. Ji W. Wang H. Zhou K. Jin Z. et al.The effect of prehabilitation on the postoperative outcomes of patients undergoing colorectal surgery: a systematic review and meta-analysis.Front Oncol. 2022; 12958261Google Scholar]. The PreColo-CER was divided into three parts: (I) A report on practice variation in preoperative care between Dutch hospitals [[6]Heil T.C. Melis R.J.F. Maas H. van Munster B.C. Olde Rikkert M.G.M. de Wilt J.H.W. et al.Technical efficiency evaluation of colorectal cancer care for older patients in Dutch hospitals.PloS One. 2021; 16e0260870Crossref Scopus (2) Google Scholar], (II) A multicenter non-randomized, prospective, cost-utility study, comparing observational data from hospitals with prehabilitation to similar data from hospitals without prehabilitation, and (III) A qualitative study on barriers and facilitators of prehabilitation implementation [[7]Heil T.C. Driessen E.J.M. Argillander T.E. Melis R.J.F. Maas H. Olde Rikkert M.G.M. et al.Implementation of prehabilitation in colorectal cancer surgery: qualitative research on how to strengthen facilitators and overcome barriers.Support Care Cancer. 2022; : 1-14Google Scholar]. The PreColo-CER faced many challenges during planning, elaboration and conduction, ultimately leading to an incomplete sample size for part II of the study, which included 53 of the originally intended sample size of 500 patients. Here, we summarize the barriers faced by the PreColo-CER and shares lessons learned about the conditions needed to perform such CER, as many other trials on prehabilitation in older patients faced similar challenges but did not publish a process analysis [[1]Molenaar C.J.L. Minnella E.M. Coca-Martinez M. ten Cate D.W.G. Regis M. Awasthi R. et al.Effect of multimodal prehabilitation on reducing postoperative complications and enhancing functional capacity following colorectal cancer surgery: the PREHAB randomized clinical trial.JAMA Surg. 2023; 158: 572-581Crossref PubMed Scopus (14) Google Scholar,[3]Heger P. Probst P. Wiskemann J. Steindorf K. Diener M.K. Mihaljevic A.L. A systematic review and Meta-analysis of physical exercise Prehabilitation in major abdominal surgery (PROSPERO 2017 CRD42017080366).J Gastrointest Surg. 2020; 24: 1375-1385Crossref PubMed Scopus (85) Google Scholar]. In the PreColo-CER, it appeared not feasible to connect all relevant stakeholders. The knowledge gap on prehabilitation before colorectal cancer surgery was identified and prioritized by geriatricians as part of the scientific research agenda of the Dutch Geriatrics Society. As a result, the development of the scientific research agenda was mostly done within the discipline of geriatrics itself, as it was urged in this development by governmental policies. Consequently, other stakeholders in the field of prehabilitation and colorectal cancer care, including surgeons and health insurances, were not sufficiently involved from the start of the PreColo-CER and, therefore, the complete network of care was not covered in preparing the study. As a result, there was no top-down incentive for surgeons to participate. Moreover, the PreColo-CER researchers had no access to knowledge about competing studies organized by surgeons, and thus lacked opportunities to collaborate with these. Health insurance companies were also relevant stakeholders, but were not partnering in the PreColo-CER. Therefore, there was no opportunity to tackle funding issues, such as the lack of structural funding for prehabilitation implementation in participating hospitals. This lack of stakeholder involvement is not new to CER in general [[8]Concannon T.W. Fuster M. Saunders T. Patel K. Wong J.B. Leslie L.K. et al.A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research.J Gen Intern Med. 2014; 29: 1692-1701Crossref PubMed Scopus (286) Google Scholar]. Including involved healthcare providers and their scientific associations, as well as patients and policymakers, from the start of CER is crucial to provide insight into different interests and to get input on how to make CER actionable and relevant [[9]Selby J.V. Forsythe L. Sox H.C. Stakeholder-driven comparative effectiveness research: an update from PCORI.Jama. 2015; 314: 2235-2236Crossref PubMed Scopus (68) Google Scholar]. As such, funders should also play a role in organizing and facilitating stakeholder engagement early in CER. The implementation rate of prehabilitation in daily practice was too limited to realistically evaluate prehabilitation practice. The research question and observational study design originated from the assumption of the initiators that a significant number of the 79 hospitals in the Netherlands performing colorectal carcinoma surgery already had structured prehabilitation programs in place or were actively implementing them. This assumption was verified during Part I of the PreColo-CER with a survey among clinicians. This survey showed among participating Dutch hospitals, 24% (7/25) offered prehabilitation as standard care, and 36% (9/25) only on indication [[6]Heil T.C. Melis R.J.F. Maas H. van Munster B.C. Olde Rikkert M.G.M. de Wilt J.H.W. et al.Technical efficiency evaluation of colorectal cancer care for older patients in Dutch hospitals.PloS One. 2021; 16e0260870Crossref Scopus (2) Google Scholar]. In reality, though, during a more thorough evaluation of Part II of the PreColo-CER it was revealed that the large majority of hospitals did not have a program in place that qualified as a predefined prehabilitation, containing at least a minimum of two weeks of exercise therapy and an optimisation of the patient's nutritional status at least two weeks before surgery. Most hospitals were only in the planning phase and still negotiating with health insurers about the conditions under which to implement a prehabilitation program. It turned out that this phase mostly took longer than expected, or that plans were renounced. The hospitals indicating that they offer prehabilitation did so mostly on an ad hoc basis. Only two hospitals participating in Part I of the PreColo-CER had a predefined prehabilitation program. At least four other hospitals that did not participate in the inventory of practice variation did have ongoing studies on prehabilitation, but were not willing to participate in the PreColo-CER because of competing interests. As mentioned, the Dutch Surgery Society was not a stakeholder involved from the start of the PreColo-CER, and consequently there was no incentive to promote and mandate study participation. During the implementation of Part II of the PreColo-CER there was one additional hospital that implemented prehabilitation and was willing to participate, resulting in a prospective descriptive CER study on only three hospitals. Although the content of the prehabilitation program was the same in these hospitals, there was variation in the inclusion and exclusion criteria and the setting where prehabilitation took place. In addition, because prehabilitation was not reimbursed as standard care, funding for prehabilitation was locally arranged by grants or by support coming from the local hospital itself. In retrospect, the question can be raised whether the CER approach as it was intended for the evaluation of prehabilitation was feasible at all, considering the limited implementation rate of prehabilitation in daily practice. In future, it is therefore necessary to confirm in advance that the intervention under study is carried out as hypothesized, for example, by using existing databases such as delivered care databases combined with quality registrations [[10]Heil T.C. Olde Rikkert M.G.M. Maas H. van Munster B.C. Willems H.C. de Wilt J.H.W. et al.Practice variation in prehabilitation for colorectal cancer surgery in older patients in the Netherlands Submitted.2023Google Scholar]. The Dutch government is now working on such a registry as CER in other domains fell short on similar issues. The process for local legal and regulatory study approval turned out to be time-consuming. Part II of the PreColo-CER did not fall within the remit of the Medical Research Involving Human Subjects Act (WMO) [[11]Medical research involving human subjects act.Bull Med Ethics. 1999; 152: 13-18Google Scholar]. In the Netherlands these studies are referred to as nonWMO-studies. Typically, this implies that a study can be conducted with less strict procedures, which should make it easier to conduct the study. However, while being more strict, the framework of regulations one has to comply with when conducting a WMO obligatory study is also highly standardized. This standardization creates clarity for all parties involved, that is in fact missing for nonWMO studies. For the PreColo-CER this meant that each institutional review board (IRB) of the (potentially) participating hospitals started a separate approval process with its own requirements and time schedule. This resulted in a long process for the study team, because for each hospital separate arrangements needed to be made, advice sought upon from legal support, (re)negotiated and implemented before a study site could start patient inclusion. To ensure that a sufficient number of hospitals will participate in a future CER, care facilities (including hospitals) should be obligated to participate in CER. A way out might be to reframe participation in CER as an obligatory part of quality assessment and improvement by care facilities. Researchers probably should be rewarded specifically, as CER does not allow to evaluate innovative interventions (which by definition are not part of regular practice), in which researchers are often most interested. Moreover, a centralized regulatory board and cooperative agreements for CER are needed. Traditional ethical and regulatory standards need to be reconsidered and redefined specifically for CER to make CER feasible in daily clinical practice and to make up for the huge diversity in how hospitals implement the regulations currently [[12]Fiore L.D. Lavori P.W. Integrating randomized comparative effectiveness research with patient care.N Engl J Med. 2016; 374: 2152-2158Crossref PubMed Scopus (93) Google Scholar]. The PreColo-CER also faced many barriers during study implementation in daily practice. It was observed that – in their oncosurgical routines – hospital staff did not succeed in sufficient patient inclusion, data collection and all other tasks needed to conduct the study, despite these being reimbursed by the research budget. In the future, it should be more strictly considered together with all stakeholders (and carefully piloted) which data collection is really feasible in daily clinical practice. Data reported in daily practice, including electronic patient file data, patient reported outcome measures (PROMs) and patient reported experience measures (PREMs), should be included in a future CER as much as possible. This can limit the burden for both patients, healthcare professionals and researchers as much as possible. To make this possible, funders should invest in facilitating the collection and aggregation of volume of multisource data to enable comparisons across care settings, patient populations and treatment combinations [[13]Gray E.A. Thorpe J.H. Comparative effectiveness research and big data: balancing potential with legal and ethical considerations.J Comp Eff Res. 2015; 4: 61-74Crossref PubMed Scopus (20) Google Scholar]. In addition, funders should work with developers of electronic health record systems (EHR) to configure these EHRs to accommodate study-specific workflows (such as the creation of specialized study-based pop-ups and order-entry screens) with back-end data linked to other databases (such as outpatient record systems) [[12]Fiore L.D. Lavori P.W. Integrating randomized comparative effectiveness research with patient care.N Engl J Med. 2016; 374: 2152-2158Crossref PubMed Scopus (93) Google Scholar]. Together with all stakeholders it should also be considered which research design is most appropriate for a specific CER. An emulated target trial methodology turned out to be feasible and probably second best if a randomized controlled trial (RCT) is not feasible [[14]Heil T.C. Verdaasdonk E.G.G. Maas H. van Munster B.C. Rikkert M. de Wilt J.H.W. et al.Improved postoperative outcomes after Prehabilitation for colorectal Cancer surgery in older patients: an emulated target trial.Ann Surg Oncol. 2023; 30: 244-254Crossref PubMed Scopus (6) Google Scholar]. This is true for a complex intervention like prehabilitation, in which an RCT is usually costly and challenging [[15]Craig P. Dieppe P. Macintyre S. Michie S. Nazareth I. Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance.Bmj. 2008; 337a1655Google Scholar], and even more so in frail older study populations, where an RCT is often complicated by difficult recruitment and high study dropout rates [[16]Diener L. Hugonot-Diener L. Alvino S. Baeyens J.P. Bone M.F. Chirita D. et al.Guidance synthesis. Medical research for and with older people in Europe: proposed ethical guidance for good clinical practice: ethical considerations.J Nutr Health Aging. 2013; 17: 625-627Crossref PubMed Scopus (24) Google Scholar]. In the latter case, the causal model for treatment assignment of an RCT in a less frail population could be used as model for an emulated target trial in a frail population. In the PreColo-CER the study protocol was mainly described in the grant application and later refined when writing the research protocols for each individual phase. In a future CER, the protocol for the evaluation should be more adaptable depending on the findings during CER. A pilot study or repeated pilots should be performed to investigate and safeguard feasibility of the chosen research design and should be adapted where necessary. Preplanned qualitative research on the barriers and facilitators experienced during implementation of the pilot study (and implementation of prehabilitation) is of added value. In addition standardized operation procedures (SOPs) should be created [[17]van der Marck M.A. Olde Rikkert M.G.M. Guideline on medical research in older persons.Tijdschr Gerontol Geriatr. 2017; 48: 278-286Crossref PubMed Scopus (3) Google Scholar]. These SOPs can then be used during the implementation of the evaluation trial. Once a future CER can finally be carried out, a process evaluation alongside the prospective trial is essential. As a result, barriers with regard to study conduction (and prehabilitation implementation, if part of the study design) can be identified timely. In the network of stakeholders these barriers can be discussed and resolved to make a future CER successful. In addition, recommendations from the process evaluation may be important in the final phase of CER, the (de)implementation of prehabilitation in daily practice, as this is often the most difficult to accomplish. Actively involving clinical research nurses in CER in each participating hospital probably adds value by improved direct communication about CER between clinical staff and participants, by increased recruitment and by improved patient compliance as nurses have closer contacts with patients and relatives compared to researchers and physicians [[18]Isaacman D.J. Reynolds E.A. Effect of a research nurse on patient enrollment in a clinical study.Pediatr Emerg Care. 1996; 12: 340-342Crossref PubMed Scopus (30) Google Scholar]. There were many barriers to CER on an intervention linked to CRC care for older patients. There were many practical challenges involved in conducting the PreColo-CER, which ultimately resulted in an insufficient sample size. Challenges mainly arose from the lack of stakeholder involvement and the limited implementation rate of the intervention of interest (prehabilitation) in contrast to what was expected. In addition, the process for local legal and regulatory study approval turned out to be time-consuming and there was a general lack of (daily) discipline needed for conducting a study such as the PreColo-CER. Moreover, hospitals were often insufficiently equipped to become a research facility for CER.