Bringing Real-World Evidence to the Decision Table
January 20, 2023
Dr. Nicole Mittmann recognizes the need for leadership in real-world evidence generation. CADTH is on it.
As CADTH’s Chief Scientist and Vice-President of Scientific Evidence, Methodologies and Resources, Dr. Mittmann manages a diverse portfolio that ranges from scientific methods and health economics to scientific publishing and library information services. Under her direction, CADTH is currently finalizing a Canadian framework for real-world evidence (RWE) generation. In addition to her leadership role at CADTH, Dr. Mittmann holds dual faculty appointments at the University of Toronto. Her past positions include Chief Research Officer at Cancer Care Ontario and Executive Director at the Health Outcomes and Pharmacoeconomics Research Centre at Sunnybrook Hospital. Throughout her career, Dr. Mittmann has maintained a passionate belief in the power of data to inform good healthcare decisions. She explains more in this conversation with 20Sense.
How would you evaluate CADTH’s ability to integrate RWE into health technology assessment today?
CADTH has been working steadily to evolve the process of incorporating RWE into our drug reimbursement reviews. Over the years, we have already considered non-randomized forms of evidence for drug utilization, health preference and patient-reported outcomes. We have also trained our reviewers in areas related to the integration of RWE into our work. We have seen, and continue to see, an increasing interest among sponsors in taking advantage of the opportunities we have created. At the same time, evidence generated from non-randomized studies has methodological limitations and therefore cannot fully replace RCT data for evaluation of efficacy.
What are the largest hurdles to overcome with RWE to support healthcare decision making?
A key challenge is how to evaluate a medication’s performance without designs that randomize and control for variables, which are built into RCTs. With RWE, we can’t fully demonstrate causality because external known and unknown factors can influence the outcome. With statistical analyses, such as matching, we can try to control for these factors or get closer to it. Also, all clinical trials have formal processes to handle informed consent, transparency of study protocols, and other important concerns. We need to ensure this is also happening with RWE studies.
“We can’t fully demonstrate causality with RWE, though we may be able to get closer to it.”
In November 2022, CADTH published a draft guidance document on the use of RWE, to be finalized following stakeholder consultation. What is the vision behind this document?
In Canada, as our capacity and expertise in generating RWE grows, so too does the need to standardize reporting for RWE studies that are submitted to inform regulatory and HTA decision-making. Our vision for the RWE Reporting Guidance is to begin laying down principles for robust study protocols and transparent reporting, building on work done by organizations such as the FDA, NICE, and the International Society for Pharmacoepidemiology. We’re explaining requirements such as “list the comparator” and “provide an analysis plan” in greater detail, so submitters will have a better understanding of what we’re looking for.
Are there any plans for alignment between HTA bodies and Health Canada to determine how and when RWE can be used to inform decision making?
We thought it was important to include the regulatory perspective in the guidance document and we involved a number of people from Health Canada in the development process to ensure our principles are aligned. Regulators and HTA assessors have some common needs, but they serve different functions and may use the guidance in slightly different ways.
I will add that collaboration in the RWE space is essential to our progress. That’s why CADTH and Health Canada are chairing the Real-World-Evidence Steering Committee, which also includes members from INESSS, the pCPA, CIHR, CIHI, industry groups, and other stakeholders.
There is a lack of guidance for industry about when to generate RWE—a question that falls outside the scope of the current guidance document. What are your thoughts on the “when” question?
RWE is not necessarily appropriate for every scenario. Circumstances that may warrant the generation of RWE include RCTs with a high level of uncertainty about the studied medication’s effectiveness and safety. We also need to know if high-quality RWE data is even available to answer research questions. In many cases, the first step of obtaining the data requires significant collaboration between different organizations, data holders, and stakeholders.
Do you have any recommendations to help industry tailor their RWE studies and submissions to CADTH’s needs?
I think the draft RWE reporting guidance is an excellent place to start. It lays out key principles and offers insight on a range of issues, from data governance and study design through to data sources, statistical methods, interpretation, and limitations. Transparency in governance, structure and design are paramount. Watch for and address information gaps: telling us what is missing improves the quality and, most importantly, the transparency of the submission.
“Watch for and address information gaps: telling us what is missing improves the quality and, most importantly, the transparency of the submission.”
At present, how willing are reviewers to use RWE as an input in a drug reimbursement review?
There’s always a willingness to look at RWE, if you can describe it in a robust and transparent way. We already look at drug utilization, health preference values, and patient input, meaning real patient experiences. These are forms of RWE.
How has industry performed in terms of RWE submissions to CADTH or other organizations?
The pharmaceutical industry has been using RWE for decades and has developed strong expertise in how they employ it. Within the industry, there are different approaches to using RWE, and not all the studies we look at are suitable to inform decision making. CADTH is trying to help bridge this gap through our RWE reporting guidance document and by including industry representatives in the RWE Steering Committee. We know that industry is looking for guidance and we are taking steps to better understand their needs.
What are your thoughts about the potential to integrate CADTH’s Scientific Advice program with the drug reimbursement review process?
Our scientific advice program is a distinct, voluntary, non-binding and confidential fee-for-service consultation that provides pharmaceutical companies with advice on their early drug development plans from a Canadian HTA perspective. In 2022 we expanded the program for a 1-year learning period, ending in March 2023, to invite applications for RWE generation plans after protocols for pivotal trials have been finalized.
We see the value of connecting the advice and review areas. It will allow us to dialogue with manufacturers and sponsors about the best approaches to generating RWE. We’re currently exploring different ways to carry knowledge throughout the life cycle of a medical technology.
Do you engage HTA bodies from other countries in discussions about RWE?
One of the aspects of this work I’m most excited about is the level of international cooperation that is happening across regulatory and HTA spaces. Last October, CADTH hosted a panel to talk about global collaborations to optimize the use of RWE in decision-making, and I really encourage your readers to watch it. CADTH is also fortunate to count a number of global experts as members of our Real-World Evidence and Real-World Data Guidance working group, which oversaw the development of our RWE reporting guidance document.
CADTH is in a similar position to many other international agencies in terms of learning and considering potential approaches. We’re aiming to harmonize our processes, as it would be a shame for one country to collect a certain type of data and not another. These are important discussions that will continue throughout 2023, as we work toward information sharing, consensus building, and the arguably more difficult step of implementation.
Do you think of registries as sources of RWE?
The term “registry” is broad. We have grant-funded, industry-funded, government-funded, and patient/donor-funded registries. We’re now having much-needed conversations with registry communities to discuss minimum quality standards for the use of registries. We see high-quality registries as a powerful tool that can help us better understand the course of a disease, collect clinical outcomes, and observe potential treatment outcomes and harms, along with capturing the real-world benefits of a new product.
What is the opportunity for patient support programs (PSPs) to serve as real-world data sources in Canada?
PSPs are definitely on our radar. We’re having a lot of discussion around them. We also understand the potential bias in using a data source developed and funded by industry. There is the potential to use processes to get past this type of challenge, such as independent analyses to ensure the reliability of the data. Overall, we’re receptive to using PSPs as one source of data among others, as long as the data meets the quality standards described in our guidance document.
Anything else you would like to say to industry or other stakeholders about RWE?
First and foremost, I would like to thank our stakeholders in industry and across the life sciences sector for their continued desire to sit at our table. Throughout the development of the RWE reporting guidance, we had many meetings and productive conversations that brought real value to our process, and I know that cooperation will continue.
The primary task ahead of us is to continue fostering collaboration and transparency. We’re excited to engage with industry stakeholders, patient stakeholders, and all stakeholders involved in the regulatory and HTA process on the next phase of the RWE journey.