When Worlds Collide: The Theory of Real-World Evidence Meets Reality

FDA has long touted the use of real-world evidence (RWE).  Extolling RWE, FDA has said “RWE can be leveraged to bring new products to market, evaluate the safety and effectiveness of existing products for new uses, and assess the continued performance and safety of products once on the market.”  FDA recognizes the potential of RWE to support regulatory submissions of medical devices and to inform benefit-risk analysis of such products, while assuring patients have timely access to devices.  FDA has even gone so far as to maintain that the “real-life clinical performance of a medical product might be more clearly demonstrated through RWD/RWE because a controlled clinical trial often cannot evaluate all applications of a product in clinical practice across the full range of potential users.”  However, in our experience, there is a large gap between FDA’s lauding the value of RWE and practice.

FDA defines real-world data (RWD) as “data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.”  RWE is “the clinical evidence regarding the usage, and potential benefits or risks, of a medical product derived from analysis of RWD.” RWD can be collected from a wide range of sources, such as electronic health records, registries, administrative claims, pharmacy data, and feedback from wearables and mobile technology.

FDA acknowledges that such data offer opportunities to generate evidence and better understand clinical outcomes.  In fact, in 2018, FDA emphasized that leveraging the use of RWD and RWE in regulatory decision-making is “a top strategic priority for the FDA.”  FDA believes that RWE has the power to “accelerate medical product development and bring new innovations and advances faster and more efficiently to the patients who need them, without compromising patient safety.”  Further, FDA has stated that enabling advanced data analytics, including RWD, is one of the objectives incorporated in the FDA’s Information Technology Strategy for FY 2024-2027.

We, and many stakeholders, agree that RWD and RWE can play an invaluable role.  However, based on our experience reviewing device premarket submissions and engaging with FDA, we have found that reviewers have not embraced it in practice.  Rather than looking at how RWD/RWE provide meaningful information on safety and effectiveness, reviewers often focus on perceived gaps.  But by their very nature, RWD/RWE will have some gaps compared to randomized controlled trials.

Congress has recognized the challenges with FDA’s acceptance of RWD and RWE.  It was Congress that intervened in 2022, to mandate FDA to issue or revise its 2017 guidance on considerations for the use of RWD and RWE to support regulatory decision-making for devices to clarify its regulatory expectations.  Also, under the MDUFA V commitment letter, FDA was directed to continue development of RWD and RWE methods and policies to advance regulatory acceptance for premarket submissions by updating and clarifying the recommendations in the 2017 guidance, and providing RWD/RWE training for FDA review teams, among other things.

In December 2023, FDA issued a new draft guidance on RWE, which updates and clarifies how FDA evaluates RWD to determine if it is sufficient to be used in regulatory decision-making for devices and provides updated recommendations for sponsors collecting RWD.  This guidance, when finalized, will replace the original version of this document finalized in 2017.

Of course, as FDA has itself acknowledged, RWE can offer some significant compensatory advantages.

FDA states it has a long history of using RWD and RWE in its regulatory decision-making for devices and conducted a sample analysis of the range of RWE that have been used in regulatory decisions in premarket submissions.  Based on a review of premarket submissions made in 2012 through 2019, FDA identified 90 examples of 510(k)s, De Novos, HDEs, and PMAs approved/cleared which utilized RWE in support of regulatory decision-making.  During this period the estimated total number of such submissions was approximately 26,121.  Based on this data set, RWD/RWE were identified by FDA as being used in a trivial 0.34 percent of examples.

This statistic is consistent with our experience.  Similar to FDA’s assessment, we have found very few examples where FDA has used RWE as the basis for making a positive regulatory decision in a premarket submission.  Not only has FDA not accepted RWD in some cases, it has even raised the evidentiary bar by requiring a clinical study as the only adequate means.

FDA’s proposed ban of electrical stimulation devices (ESD) is a case in point. FDA is proposing to ban the device, citing the absence of “large, randomized, and controlled trials, or even any large or randomized trials.”  (89 FR 20882 at 20889, March 26, 2024).  In taking this approach, FDA expressly rejected the adequacy of the substantial amount of RWD available.  Patients that use ESD undergo continuous 24/7 monitoring.  There is ample data collected on them during this 24/7 monitoring period such as case notes, contemporaneous patient medical records, and patient tapes.  Every use of ESD is recorded, as is every behavior that falls within the intended use.  Patients are continually monitored for safety.  Far more data are collected at the site than would ever be available in a registry or other source of RWD.  Nonetheless, FDA entirely discounted this information and has concluded that the available evidence is not enough to establish ESDs are effective and that instead “randomization, control, [and] large numbers of subjects” should be relied on for understanding the benefit-risk profile of ESDs (89 FR 20882 at 20890).  FDA’s rejection of all RWD supporting GED in favor of randomized controlled trials cannot be squared with its broader policies.  Device stakeholders would be right to be troubled by the inconsistency between FDA’s professed support for RWE and its very public rejection of it in the proposed ban.

We recognize that there can be challenges with using RWD/RWE in regulatory decision-making.  However, despite FDA leadership’s efforts to recognize and encourage use of RWD and RWE, reviewers have not fully embraced RWD/RWE.  FDA’s brusque rejection of data obtained from 24/7 monitoring of patients, as set forth in its recent proposed ban, will do nothing to encourage reviewers to embrace RWD/RWE.

As FDA has noted, RWD/RWE has the potential to be relied on to evaluate the safety and effectiveness of products.  Yet that potential will never be realized unless reviewers are willing to accept RWD/RWE despite their limitations.  FDA’s recent public dismissal of the use of medical information obtained from continuous, 24/7 monitoring of patients does nothing to encourage reviewers to rely upon RWD/RWE when reviewing marketing submissions.