Breaking the RWE Acceptability Barrier
January 25, 2023
If real-world evidence is to shape healthcare decisions, it must tick several boxes.
Real-world evidence (RWE), while hardly a new idea, is on everyone’s mind these days. As specialty treatments continue to increase in complexity, stakeholders are discovering how RWE can shape a novel medication’s destiny. By filling evidence gaps, RWE can help get the right drug to the right patient at the right time – and optimize treatment. Most importantly, use of RWE can inform approval and reimbursement decisions, giving patients faster access to life-changing medications while rewarding manufacturers for creating value and ensuring that payers are supporting the most valuable and efficacious treatments. What’s not to like?
Exciting possibilities await RWE in this country. Canada has joined the global push toward RWE and policymakers agree on the need, though hesitation about when and how to use the data has stretched out the implementation timeline. A big priority, at this juncture, is data acceptability – ensuring decision makers trust the data enough to use it. We have alignment on what has to happen and are ready for lift-off.
OUTLOOK AND OPPORTUNITIES
RWE ranks number one on the list of top health economics and outcomes research (HEOR) trends identified by ISPOR in 2022 – ahead of such major priorities as health value assessment, health equity, healthcare financing, and patient engagement – and continues to grow in importance.12 By all accounts, the specialty pharmaceutical space views data from real-world studies as a meaningful source of information and the quest for RWE as a valuable undertaking. As management guru Peter Drucker has famously said, “what gets measured gets managed.”13
Why go to the trouble of generating RWE? By definition, RWE sheds light on how a medical intervention performs “in the field,” as opposed to the more controlled conditions of a clinical trial. As noted by Dr. Mark Fendrick, a professor of internal medicine and health management at the University of Michigan, “the benefit of [real-world data] is embedded in its name. The data come from the real-world, where diverse people live, work and play.”14
“Real-world data, by definition, come from the real world, where diverse people live, work and play.”
RWE bridges the evidence gaps left by clinical trials, allowing us to answer such questions as: Is the medication working for the patients it is intended for? How is it performing in specific populations that may have been underrepresented (or absent) in clinical trials? Does it work better for some populations than others?
We’ve made good headway in using RWE to answer such questions, which helps clinicians better understand patient populations and optimize treatments. As an example of this application of RWE, consider the INFORM study, which evaluated the benefits of targeted multiple myeloma therapy in real-world Canadian patients, both as front-line and maintenance treatment.15 In the study, patients who received novel agents such as REVLIMID were significantly less likely to die within a year compared to those treated before these agents entered the market. According to Tara Cowling of Medlior, which conducted the study, “There were some differences noted between real-world practice and clinical trials/treatment guidelines, particularly in the use of maintenance monotherapy.”15
RWE can go still further. Used to its full potential, it can help establish efficacy and value to HTA assessors and payers. This holds especially true for the growing number of novel therapies, often within precision oncology or rare diseases, that show great promise but have limited data from clinical trials. These data gaps stretch out the time to access, leaving patients in limbo as they wait for medications that could change their lives. Payers and HTA assessors around the world are seeing value in leveraging RWE to support decision making, which in turn helps the timeliness of such reimbursement decisions.
In the UK, for example, Managed Access Agreements (MAAs) enable time-limited access to promising new treatments that would otherwise not be recommended for routine use. The RWD collected through the MAAs helps establish the value of the treatments. Since the launch of the first MAA, the National Institute for Health and Care Excellence (NICE) has reevaluated 22 medications after a period of managed access, 20 of them cancer drugs. The mechanism works: 19 of the 22 medications received approval for routine use following the MAA period. 16 This high success rate led NICE to conclude that “managed access is [now] an established mechanism in England for early patient access to promising new treatments, where significant evidential uncertainty remains.”
In Canada, RWE hasn’t reached this level of integration into HTA. The data itself is out there: from health claims, hospitals, administrative databanks, patient registries, and patient support programs (PSPs), among other sources. What’s missing is the consistent use of the data to support regulatory, HTA recommendations, and listing decisions. So where’s the bottleneck?
A lot has to do with concerns about the quality of the data. To this end, ISPOR has identified some key RWE characteristics that can make or break stakeholders’ confidence in the data. 14 These include:
Source: How reliable and robust is the source of the data?
Approach: What was the purpose of collecting the RWE? Was the research design and process transparently communicated?
- Analysis: Did the statistical methods suit the investigation and were they properly applied? *
- Reproducibility: Can the results of the study be replicated?
Canadian HTA assessors and payers, for their part, have identified trust and bias as key barriers to RWE adoption. 17 Reviewers are more likely to trust data that can be audited and validated. They also need the assurance the data hasn’t been cherry picked. Under the bias umbrella, selection bias ranks as a top concern: the study population must reflect the target population for the drug.
The hurdles don’t stop there. Lack of relevance, completeness, or sufficient time span of the data can also weaken an RWE submission. In its recent recommendation against reimbursing SPINRAZA for adult spinal muscular atrophy, CADTH took issue with the short period (months) for measuring RWE outcomes, which they deemed insufficient for a lifelong disease. 10 Citing the NICE RWE framework document in its rationale, the recommendation also noted that the submission “had large amounts of missing data, [which] may not be missing at random but possibly due to lack of efficacy.”
Bottom line: quality comes first. If the data doesn’t meet quality standards, efforts to shape it into acceptable RWE will stall.
IN PROGRESS
Such challenges have not prevented Canadian researchers and other stakeholders from moving the RWE agenda forward, with inspiring guidance documents and initiatives under way throughout the country. After formalizing a life-sciences strategy that incorporates RWE a few years ago, Quebec reaffirmed its commitment to the strategy in a 2022 paper called “Using our Ingenuity to Promote Health.” 18 The document asserts that RWE can help demonstrate a treatment’s value and that “the evidence obtained can guide the decisions of healthcare administrators and reimbursement agencies.” As a testament to this position, in May 2022 INESSS issued a positive recommendation for the CAR-T cell therapy BREYANZI, which targets large B-cell lymphoma, on the condition of RWE generation. 19RWE Exhibit A: INESSS Lung Cancer Study
INESSS put its RWE strategy to the test in an exploration of administrative clinical data as a vehicle for evaluating epidermal growth factor receptor (EGFR) tyrosine kinase inhibitors (TKI) in the treatment of lung cancer in Quebec.20 Data sets in the study included drugs dispensed, lab results, and lines of therapy, among others. Investigators used the data to estimate the overall survival of patients receiving EGFR-TKI therapy and compare the figure to outcomes reported in published studies.
Overall, the real-world performance of the EGFR-TKI agents paralleled the results seen in most clinical studies. The RWE investigation confirmed that “EGFR-TKI should be offered to all eligible patients, based on the approved indications.” The study also revealed that it took approximately 5 years to fully integrate EGFR-TKI into Quebec practice, prompting INESSS to call out the need to promote education about new therapies to enable faster uptake. By illustrating the power of RWE to establish value and identify access gaps, this study can serve as a model for using RWE to improve access and care.
It should be noted that initiatives of this type only became achievable since 2016, when INESSS gained access to anonymized patient data from other provincial databases and began assigning a unique identifier number to each patient. As Michèle de Guise, president and general manager of INESSS, explains, “it means we don’t just leverage data from the literature, but can see how new technologies play out in the Quebec context.”21
Echoing INESSS’s forward march, CADTH has expanded its scientific advice program to allow pharmaceutical companies to request advice on RWE generation.22 CADTH’s new post-market drug evaluation program,23 meanwhile, brings together a network of experts (called CoLab) to supply RWE to submission reviewers. Along similar lines, CIHI’s CanREValue framework lays out parameters for generating and using actionable RWE for cancer drugs.24 Within the hospital sector, Hamilton Health Sciences has partnered with data technology firm Pentavere to help translate unstructured clinical data on breast cancer patients into RWE that can guide clinical decisions—one of many examples of the creative energy being invested in RWE.25
RWE in Canada took an especially significant step forward in November 2022, when CADTH released the draft of its RWE guidance document.11 The much-anticipated document outlines best practices for RWE submissions and helps demystify what regulators and HTA bodies are looking for in these submissions. Principles covered by the document include how to evaluate data sources, how to eliminate bias, and how to communicate study protocols transparently. A 92-item checklist advises submitters on what to report and what limitations to address, including potential sources of bias. Following a stakeholder consultation period, CADTH expects to release the final document in the spring of 2023.
RWE Exhibit B: CADTH patient registry analysis
Patient registries can shed light on the epidemiology of a disease, its impact, and its prognosis as new treatments become available, thus playing a key role in the RWE ecosystem. CADTH has been looking closely at Canadian rare disease registries as sources of RWD. One of these is the Canadian Bleeding Disorders Registry (CBDR), owned and operated by the Association of Hemophilia Clinic Directors of Canada and hosted at McMaster University. A national database created to support best practices, individualized treatment, and engagement of Canadian patients with bleeding disorders, the registry integrates data collected from various sources, including the Canadian bleeding disorders community.26
In its review of the CBDR, CADTH described the registry as “secure, centralized, encrypted” and identified formal processes to enable the collection of both clinical and patient-reported outcomes.26 There’s room to expand on these capabilities by collecting data that has the robustness to support decision making. This could help compensate for a key limitation of RWE – lack of comparators – and take registry RWE to the next level.
ACCEPTABLE OR NOT?
While CADTH’s scientific advice program on RWE remains separate from its submission and review process, the RWE guidance document can help investigators improve the quality of their RWE studies. That said, certain parameters lie outside the document’s current scope. As Mina Tadrous, a scientist at Women’s College Hospital in Toronto and Lead of the Core Working Team for the RWE Guidance Working Group, explains in a webinar about the guidance,27 “There won’t be a point score, a way to say this data is acceptable and this is not.” Also out of scope is “information on the weight given to RWE in decisions. This is going to vary.” Most importantly, the guidance doesn’t cover “when RWE should be, can be, and will be used.”These as-yet unanswered questions create a challenging scenario for manufacturers, who must decide whether to invest in RWE generation without knowing whether their efforts will move the access needle. Even if a submitter follows all the rules, including the 92-item checklist, reviewers may deem the evidence insufficiently robust or relevant.17
Fortunately, stakeholders can look forward to more granular and targeted guidance on RWE generation from CADTH, as stated in the draft guidance document: “Future efforts can leverage these core reporting standards to provide guidance on how and when RWE can be used in HTA and regulatory decision-making.”11
Acceptability of RWE: An oncology case study
What makes RWE acceptable for decision making? Manufacturers are working hard to answer the question, as exemplified by this real-world study of lorlatinib. An oral medication developed by Pfizer, lorlatinib is used to treat a subset of patients with non small-cell lung cancer (NSCLC). To strengthen the evidence for the treatment, IQVIA Solutions, which conducted this RWE study, turned to the patient support program (PSP) associated with the medication. Analysis of the PSP data showed that lorlatinib yielded a meaningful increase in quality of life within 3 months, which patients largely maintained throughout the year-long data capture period.28
To gain insight into the acceptability of this evidence, study authors have outlined RWE-related questions for stakeholders such as CADTH, INESSS, provincial health ministries and departments, and patient groups to consider,28 including:
- Is the evidence useful?
- Is the data collection methodology sufficiently robust?
- What applications can it be used for (for example, product listing agreements)?
- What are the limitations of the data?
Guidance from successful initiatives and from experts in RWE acceptability can also point investigators in the right direction. As a start, the NICE real-world evidence framework lists a number of scenarios that call for evidence sources other than RCTs.4 These include when:
- Randomization is considered unethical, which can happen in therapeutic areas of high unmet need.
- Patients refuse allocation to one of the treatment arms.
- Healthcare professionals refuse to randomize patients to what they consider a less effective treatment arm.
- The low number of eligible patients precludes conducting an adequately powered RCT.
Investigators can also learn from each other by sharing their RWE research protocols and study outcomes. Dr. Winson Cheung, a professor of medicine at the University of Calgary and principal director of the Oncology Outcomes (O2) research program, proposes “a registry of all Canadian RWE studies that are being conducted, regardless of the RWE study outcome.” In addition to facilitating learning exchange, such a registry would “reduce publication bias and thus increase transparency and credibility.” In fact, Dr. Cheung and his O2 colleagues suggest that there would be significant value in creating “an online portal where RWE studies can be registered —something similar to the clinicaltrials.gov site, but for real-world investigations.” Beyond Canada’s borders, ISPOR and a few partners have launched an RWE registry called the Real-World Evidence Transparency Initiative, hoping it will build trust that “[study] results can be used for decision-making purposes.”29
“A registry of Canadian RWE studies would help reduce publication bias and thus increase transparency and credibility.”
TRANSFORMING OUR DATA CULTURE
In an ideal world, investigators would not have to take guesses about RWE acceptability. Before conducting an RWE study, all stakeholders would discuss the evidence gaps and agree on a study protocol that fills them. While this level of collaboration won’t happen overnight, decision makers’ increasing engagement with RWE bodes well for the coming years.The CADTH RWE guidance document has given RWE in Canada a big push forward, but implementation still stalls at the acceptability stage. To get the gears moving, pharmaceutical manufacturers could build on their efforts to address two major acceptability barriers – trust and bias – by systematically publishing study protocols and study results. Registering studies in RWE databases, a strategy recommended by ISPOR,30 signals a commitment to such transparency.
RWE has to provide value not just to patients and HTA assessors, but also to industry and payers, and much work remains to be done in this area. Collaborative partnerships, multistakeholder engagement, and guidance on RWE submissions can move us toward this objective.31 In an inspiring application of such collaboration, CADTH’s pediatric low-grade glioma (pLGG) learning project has engaged 7 different stakeholder groups, including industry, payers, and patients, to find out which RWE elements provide the most useful information to decision makers.31 Stakeholders agreed that RWD and RWE can “play a role in decision making by providing additional, complementary evidence” and that the data in Canadian registries such as POGONIS and CYP-C meets the quality standards to generate actionable RWE. Looking ahead, they envisioned that these registries could allow for the collection of prospective data – a currently untapped functionality.31
Where do we go from here? And how do we make bigger strides in the use of RWE for efficacy? We need to continue to transform the culture around RWE in Canada.
Changing a research culture requires both the vision to recognize the need and the support to facilitate the transition, as encapsulated in ISPOR’s culture-building pyramid. In Canada, we have just begun our ascent. Having reached the “possible” stage, we can now turn our attention to streamlining the process and removing barriers – making it “easy.”
The Canadian RWE ecosystem is rapidly maturing. As more and more stakeholders join the RWE journey, the new data culture will solidify and the barriers will fall away. We’re on our way. Let’s keep going.
References
4. NICE real-world evidence framework. June 23, 2022. https://www.nice.org.uk/corporate/ecd9/resources/nice-realworld-evidence-framework-pdf-1124020816837
11. RWE guidance document (draft). CADTH. November 2022. https://www.cadth.ca/sites/default/files/RWE/pdf/RWE%20Reprting%20Guidance%20-%20Draft%20for%20Consultation.pdf
13. Klaus P. Measuring Customer Experience (2015). Chapter 7. https://link.springer.com/chapter/10.1057/9781137375469_7
14. Fendrick MA. Real-world evidence: additional tool to support clinical decision making. Bristol Myers Squibb 2021. https://www.ispor.org/docs/default-source/strategic-initiatives/ispor-rwe-byline-article_10-25-21.pdf
15. Case study: multiple myeloma. Medlior. April 30, 2020. https://www.medlior.com/insights/case-study-multiple-myeloma/
16. Bee C et al. What goes in must come out (poster). NICE https://www.ispor.org/docs/default-source/intl2022/ispor-2022-ma-posterfinal-pdf.pdf?sfvrsn=86df95f1_0
17. 20Sense original research.
18. Using our ingenuity to promote health: 2022 to 2025. Government of Quebec. May 2022. https://cdn-contenu.quebec.ca/cdn-contenu/adm/min/economie/publications-adm/politique/PO_strategie_sciences_vie_2022-2025_MEI_EN.pdf
19. INESSS recommendation for Breyanzi: list with conditions. May 2022. https://www.inesss.qc.ca/fileadmin/doc/INESSS/Inscription_medicaments/Avis_au_ministre/Juin_2022/Breyanzi_2022_06.pdf
20. Utilisation en contexte québécois des inhibiteurs de la tyrosine kinase du récepteur du facteur de croissance épidermique (EGFR) pour le traitement du cancer du poumon. Coup D’Oeil. INESSS. https://www.inesss.qc.ca/fileadmin/doc/INESSS/Rapports/Oncologie/CoupDoeil_Cancer_poumon.pdf
21. A bold new vision for value in healthcare. 20Sense. Oct. 19, 2022. https://www.20sense.ca/articles/22-03
22. CADTH news release. April 28, 2022. https://www.cadth.ca/news/cadth-expands-scientific-advice-program-include-advice-real-world-evidence
23. Post-market drug evaluation program (PMDE). CADTH. Last updated Dec. 12, 2022. https://www.cadth.ca/post-market-drug-evaluation-pmde-program
24. Value-based decisions from real-world evidence. CIHR. https://cc-arcc.ca/canrevalue/
25. Daniel D. AI making progress in collaborative projects across Canada. Canadian Healthcare Technology. Nov. 4, 2021. https://www.canhealth.com/2021/11/04/ai-making-progress-in-collaborative-projects-across-canada/
26. Iorio A et al. Canadian Journal of Health Technologies. August 2022, Volume 2, Issue 8. https://www.canjhealthtechnol.ca/index.php/cjht/article/view/MC0019/MG0019
27. CADTH webinar: consultation on real-world evidence guidance. Dec. 6, 2022. https://www.youtube.com/watch?v=QS0lDUyax8A
28. On PV et al. Leveraging patient support program infrastructure to gather data to supplement HTA for rare tumors: What constitutes quality real-world evidence? https://scientificpubs.congressposter.com/p/4bwylsdtcec8k3po
29. Real-World Evidence Transparency Initiative. https://www.ispor.org/strategic-initiatives/real-world-evidence/real-world-evidence-transparency-initiative
30. Shaking the myth of real-world evidence: updates from the RWE transparency initiative. ISPOR webinar. Dec. 17, 2020. https://www.ispor.org/docs/default-source/strategic-initiatives/rwe_combined-slides_final.pdf?sfvrsn=4564f4dc_0
31. Dialogue: optimizing the use of real-world evidence for decision-making for pediatric low-grade
glioma in Canada. https://www.cadth.ca/sites/default/files/RWE/pdf/multi-stakeholder_dialogue_optimizing-the_use-of_real-world_evidence_for_decision-making_for_pediatric_low-grade_glioma_in%20_canada.pdf