EMA provides mid-point progress report on real-world evidence development strategies

Background: Real-world data (RWD) and evidence (RWE) for regulatory use in the E.U.

• What is real-world data (RWD) and evidence (RWE)? Just as a quick reminder: Real-world Data (RWD) is data generated by the health care system outside clinical trials. Principle sources include “electronic health records, claims data, and data from registries.” RWD can be used to generate evidence, known as real-world evidence (RWE). As E.U. regulators have previously discussed, using RWE “for regulatory decision-making is already a reality, and there is increasing interest in using big data and advanced analytics as a complementary source of evidence.” [See AgencyIQ’s landscape analysis on RWD/RWE in the U.S. and E.U. here.]
• In the E.U., work to advance RWE in regulatory decision-making is underway. In 2019, the European Union published the Operational, Technical, and Methodological (OPTIMAL) Framework for RWE. This publication, issued in April 2019, noted the “increasing interest in the use of real-world data (RWD) to support regulatory decision-making across the product life cycle” – and cited the FDA’s 2018 RWE Framework. The OPTIMAL Framework laid out EMA’s perspective on challenges in using RWD for regulatory use and offered “possible solutions” in an EU context. However, that framework also outlined the significant work needed before regulators could utilize these data for decision-making, even as it acknowledged the changing digital data landscape that would expand RWD sources in the E.U.
• Regulators have been investing in ways to understand better how RWE can be used and how RWD is generated. In September 2021, the EMA’s Pharmacovigilance Risk Assessment Committee (PRAC) conducted a pilot program as a landscape analysis to “explore the need for and usefulness of RWE generated by EMA to support regulatory decisions.” Examples of these pilot activities in assessing RWE and its generation include work from committees such as the Pediatric Committee (PDCO), the Committee for Orphan Medicinal Products (COMP), the Committee for Advanced Therapies (CAT), as well as the Committee for Medicinal Products for Human Use (CHMP) and its Scientific Advice Working Party (SAWP).
• Regulators at EMA have outlined their intent to “enable” RWE and “establish its value for regulatory decision-making” by 2025. EMA and the Heads of Medicines Agencies (HMA) outlined following the 2021 pilot programs to outline the goals of the regulatory network through 2025. This included a strategic action item to enhance the technical capabilities and understanding of how Big Data could be used in life sciences research.
• HMA’s Big Data Steering Group (BDSG) has taken point on RWE issues. In July 2022, the steering committee adopted a new iteration of its work plan, outlining its goals through 2025. As of 2023, several test pilot programs are ongoing; the goal is a “strengthened regulatory system that can efficiently integrate data analysis into its assessment processes to improve decision making,” based on “knowing when and how to have confidence in novel technologies.” The project aims to assess RWE’s ability to help improve treatment outcomes, facilitate early patient access to new treatments and accelerate drug development. As shown in the individual work plans by BDSG and the EMA, these plans included a new pilot program specifically developed to understand better the framework needed to support regulatory decisions leveraging RWD/RWE.

The EMA and BDSC have just released a status report on their RWD pilots

• This new report covers the BDSG and EMA pilot’s findings from September 2021 (i.e., the last PRAC pilot report) to February 2023. The report outlines BDSG’s RWD-based research programs’ results and previous experience. As highlighted in the BDSG work plans, the EMA is coordinating and piloting RWE-based research projects to identify successes and challenges and get a broader sense what RWE analyses look like and how they can potentially be used in regulatory contexts. The EMA’s RWD/E pilot programs are about halfway through and are intended to run into 2025 when an official “regulatory report is published on RWE decision making.” In effect: This report provides a status check on the information that will be leveraged for the 2025 report, which is intended to support an RWE framework for the E.U.
• In the report, BDSG identified the ongoing EMA RWD pilots to categorize them and explore their use and progress. Research questions for these specific studies were either requested by members of the EMA or identified by Committee members, CMDH, NCA, or SAWP members. Research topics could also be proposed by the EMA RWE team “based on proactive screening or scientific advice requests, or by a screening of meeting agendas and minutes of certain committees.” In the end, 61 RWD research projects were selected in the categories of design and feasibility, evaluation of incidence and prevalence of the disease, drug utilization, and safety.
• There are three basic categories of EMA RWD pilots, which use different data sources: In-house, DARWIN EU, and framework contract (FWC). The “in-house” studies, via the BDSG RWE generation pathway, leverages EMA’s “direct access” to six medical databases of patient medical records throughout the EU. These records contain a collection of “patient encounters from primary care, general practitioner data, and some also cover information from specialists.” Out of the 61 research projects in the report, most were conducted via this pathway, with 49 “in-house” studies, which is over 80% of the topics considered for the entirety of the BDSG projects. The second option for RWE generation, DARWIN EU, as AgencyIQ has previously discussed, is a “Data Analytics and Real World Interrogation Network” with a series of “data holders working under common governance using a common data model.” Unlike the “in-house” category, only eight topics were considered for projects via this method. The last category with only one topic is the FWC framework. This network has eight research organizations and academic institutions, with a “wide network of 59 data sources in 21 countries” available for RWE analysis.
• One key upfront issue: Were the studies selected actually feasible? This issue of finding feasible topics for the RWE pilots was discovered early in the pilot programming. As the report identified, out of 61 RWD topics selected as an RWE generation project, only 30 could be executed. For the “in-house” studies, 25 out of 49 were considered for analysis, with only 24 completed and one still active by the publishing of this report. In the DARWIN EU studies, 4 out of the eight were considered, and a total of 3 completed, with one still active in 2023. By the end of the reporting period for the FWC category, the only study found feasible has yet to be completed and is expected to continue into late 2023. This feasibility issue has significantly impacted the number of studies active for analysis, as almost 50% of the chosen topics were declared “not feasible.” These studies are often determined to be infeasible due to a medical product not being “prescribed in the database setting or not authorized to be used in the respective countries. Another frequent reason was that the outcomes of interest (condition or adverse event) were not adequately captured in databases.” It’s important to note that because of the significant number of studies labeled “not feasible,” the amount of new data obtained will need to be further tested and supplemented in future programs.

The pros and cons of each RWE generation method in the pilot

• With over 80% of the total reported pilot studies being completed “in-house,” this was by far the most utilized study method. As previously mentioned, all RWD for “in-house” studies was obtained through six databases containing patient records from various primary and secondary care settings across the E.U. Primary care data was accessed through the IQVIA UK, France, and the THIN Italy and Romania databases. In contrast, secondary care settings or specialist-related care activities were sourced from IQVIA Germany and THIN Spain. The accessibility of the data allowed for more topics to be conducted through the “in-house” pilot programs, despite limitations in the data itself, including “the need for more specificity (such as age), granularity (e.g., coding, information on indication, dosing, duration of use, etc.) and completeness of the data.” This method also lends itself to quicker-turnaround projects, as RWE can be generated with a “56 calendar day average” if “suitable data sources exist.” Overall, the report concluded that the in-house pilot studies were, in fact, a good option for obtaining RWD and RWE, even as the EMA continues to invest in DARWIN EU.
• During its first year, DARWIN EU conducted four RWD research projects, three being completed and one still ongoing in 2023. As previously explained by AgencyIQ, this method is a federal network of data holders working under a shared governance and a similar data model. Although getting DARWIN EU up and running is a top priority for the EMA, it was not considered a good option for time-sensitive pilot projects as the data generation process is lengthy and complicated. One notable example of the DARWIN EU projects called out in the report was a completed pilot on the “use of antibiotics in the ‘Watch’ category of the [World Health Organization] WHO AWaRe classification,” which focuses on antimicrobial resistance (AMR). This study was intended to “investigate the incidence and prevalence of use of antibiotics (from the WHO watch list) and to explore the duration of antibiotic use as well as indication.” Researchers used five data sources from the Netherlands, the United Kingdom, Spain, Germany, and France. Researchers tracked the utilization of antibiotics “at risk of AMR,” assessing trends in utilization; for example, the pilot found that the use of antibiotics in children was lower than that of adults, and use would statistically increase with age. The trends observed in this pilot were leveraged for regulatory use in an ongoing impact study for fluoroquinolone use commissioned by the EMA. Other DARWIN EU research projects include work on the prevalence of rare blood cancers in Europe and the use of valproate-containing products in women of childbearing potential. Though seen as impactful, the number of studies determined feasible and complete compared to the in-house studies demonstrates there is still significant work to do to fully realize DARWIN EU’s intended usability, especially for research requests on short-to-medium timelines. Per the report: “the DARWIN EU network is expected to become more agile,” although use cases like the AMR study, which used a 10-year time horizon, may be more feasible in the near term.
• The Framework Contract (FWC) method played a small part in the BDSG pilot studies. “During the proof-of-concept and pilot period covered by this report, four RWD study requests were received via the RWE platform,” though only one epidemiology study on the disease and treatment patterns of spinal muscular atrophy was found to be feasible and is still ongoing 17 months later. This was surprising, as “40 pharmacoepidemiology studies have been commissioned” with the FWC method since 2014. It’s also important to note that the only feasible study has yet to be finished, which means the report considers this method in the “do not use” category for time-sensitive investigations. The FWC databases coordinated through the EMA were selected initially in a public competition where eight private and public organizations were chosen to analyze and conduct RWD information. Due to this, these RWD studies can be initiated through different routes, such as a request, a response from a public health emergency, or a Pharmacovigilance impact study. According to the report’s results and lack of pilot studies, this RWD option may have received the least favorable compared to the DARWIN EU and in-house options.

What we learned and the gaps they identified:

• RWD-based analyses have a long way to go. As demonstrated in the report, the EMA’s pilot projects show that there are still significant gaps in RWD analysis methods; in particular, the still-ongoing buildout of DARWIN EU. The report suggested options to help increase the feasibility of these projects through enhanced dialogue and close collaboration with current data partners and coordinating centers.
• In-house studies can rapidly generate RWE and provide suitable data sources, which could help support decision-making. While this method allowed for quicker turnaround times than either FWC or the DARWIN EU projects, using patient records has limitations, including a need for granularity in the data (e.g., dosing).
• To expand coverage of different outcomes and exposure, additional data sources such as number, size, and type are readily accessible to EMA. According to the report, additional data is needed to address the current gaps, including certain demographic information (e.g., birth dates) and mother-child linkages. Sources of these data could include electronic health records from additional settings, ideally linked to primary care data along with biobanks, extensive claim databases, disease or patient registries, and broader geographical representatives’ data sources.
• The EMA says it will continue exploring if and how relevant research questions can be identified earlier, and even ahead of the submission of the regulatory application, by considering the dynamic development pipeline, expanding screening approaches for suitable procedure types, and leveraging expert input from the RWE liaison groups, the product teams’ committees, and other relevant information. The report also cites work that could potentially “accelerate the generation of RWE” beyond the in-house pathway. This includes expanding the technical capacity of DARWIN EU and analytical methods, including development of phenotype libraries, a standard catalogue of executable analyses, precomputed searchable dashboards, and “increased automation of repeated tasks.”

What’s Next?

• At a high level, the report provides a pretty good idea of where the EMA is with understanding RWD-based analyses, including its own understanding of what the data ecosystem in the E.U. looks like for research purposes and what kinds of studies can – or, importantly, can’t – be done. The status of the pilot programs highlights existing gaps in both the data available and the methods by which it can be analyzed for potential use as RWE. Notably, much of the report highlights future capabilities of DARWIN EU, indicating EMA’s position that the system will help mitigate the issues identified by the researchers. All that to say: There is still significant work to be done by the regulator to understand the RWD/E landscape in the E.U.
• There is more planned in the BDSG and EMA work plan frameworks in the next two years. For example, the goal for DARWIN EU is to have an additional 20 data partners and 66 studies by the end of 2024. For 2025, the work plan states it will add at least 100 studies and another 10 data partners. These goals are essential in reaching the upcoming goals for the E.U. network and making it possible to gain enough info on RWE and RWD to publish more official documentation. Additional planned work includes a glossary of key terms in Q3 of 2023, and publishing of more use cases by Q4, as well as a publication of RWD quality considerations and the launch of the “European . By Q2 of 2024, there are plans to publish the CHMP and CAT pilot reports, and by Q4, the work plan states that the BDSG will have more use cases revised and work on understanding new sources of RWD including eHealth data and social media. By 2025, the group plans to publish a report on RWE in regulatory decision-making, which will help outline the framework on more official grounds that has been in the works over the last couple of years.
• The EMA recognizes that their baseline needs work. One of the goals in the upcoming years is to obtain “wider access to additional, more diverse and complementary data sources.” This could increase the options to generate RWE and the network of pilot tools for testing. As suggested by the information provided by the report, more pilot testing will need to occur over the next two years to meet these goals and further expand and grow these networks.

To contact the author of this item, please email Sierra Milam (smilam@agencyiq.com).
To contact the editor of this item, please email Laura DiAngelo ( ldiangelo@agencyiq.com)

Key Documents and Dates

• Use of real-world evidence in regulatory decision making – EMA publishes review of its studies – published June 23, 2023.
• Real-World Evidence in EU Medicines Regulation: Enabling Use and Establishing Value – published November 19, 2021