CDRH Issues Final Guidances Regarding Use of Real-World Evidence and Procedures for Advisory Committee Meetings

It appears that CDRH has now resumed issuing guidances after not issuing any in the first six months of the new administration. In late August and early September CDRH has issued several final guidances, including “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,” issued on August 31, 2017, and “Procedures for Meetings of the Medical Devices Advisory Committee,” issued on September 1, 2017.  Below, we discuss key points of each of the guidances.

Real-World Evidence

The draft of this guidance was issued in 2016, and it discusses types of clinical information, apart from traditional clinical studies, that could be used to support regulatory decisions, including premarket clearance/approval. According to the guidance, Real-World Evidence (RWE) is information regarding usage and potential risk/benefit of a device, which is derived from real-world data (RWD), including electronic health records (EHRs), claims and billing data, data from product and disease registries, and patient-generated data including in home-use settings, among others.  The guidance specifically excludes non-clinical data, adverse event reports, secondary use of clinical trial data (e.g., post-hoc analyses), and literature reviews.  RWE can be used in a variety of situations, according to the guidance, including in a premarket submission, postmarket surveillance study (Section 522), post-approval study, or as a control in a clinical study.

In our experience, FDA has often resisted acceptance of RWE because of concerns there was a bias in the data. The purpose of this guidance is to describe the factors that FDA will consider when reviewing RWE, namely (i) relevance, and (ii) reliability.

With regard to relevance, the guidance states that overall a sponsor should consider whether the data is directly relevant to the regulatory question at issue (e.g., a premarket submission, 522 order). The long list that follows certainly provides FDA with considerable flexibility in deciding whether to accept RWE and how much weight to give it.  The factors FDA will use to assess relevance include:

• the RWD contain sufficient detail to capture the use of the device, exposures, and the outcomes of interest in the appropriate population (i.e., the data apply to the question at hand);
• the data elements available for analysis are capable of addressing the specified question when valid and appropriate analytical methods are applied (i.e., the data are amenable to sound clinical and statistical analysis); and
• the RWD and RWE they provide are interpretable using informed clinical/scientific judgment. Important considerations for the assessment of this factor include:
  • whether the use of the device in a real-world population is representative as captured within the data source, and is generalizable to the relevant population being evaluated;
  • whether the RWD source is used regionally, nationally and/or internationally;
  • the overall percentage of patient exposure to the device that are captured in the RWD source;
  • the validation protocols and resultant data that are used to evaluate how well the RWD source reflects the patient population’s experience;
  • whether the RWD study design, study protocol, and/or analysis plan is appropriate to address the regulatory question and capable of being accomplished in a sufficiently timely manner;
  • whether the RWD contains elements to capture specific device identification information (e.g., unique device identifier);
  • whether the RWD adequately captures patient medical history and preexisting conditions, as well as follow-up information needed to evaluate the question being addressed (e.g., whether administrative claims data have adequate continuity of coverage);
  • whether sufficient data elements are collected to adjust for confounding factors that may impact the exposure or outcomes of interest;
  • whether any linkages performed are scientifically appropriate and account for differences in coding and reporting across sources;
  • the RWD source reporting schedule, including time interval between database close and release, and length of reporting periods;
  • the prior documented (e.g., peer reviewed publications or practice guidelines) use of the RWD source for determining outcomes-based quality assessments, validated predictive risk modeling, signal detection, performance improvement, benchmarking, and other clinically-meaningful uses;
  • whether the data elements collected are sufficient for assessing outcomes (including adjudication, if necessary); and
  • whether supplemental data sources are available and sufficient to provide any missing information or evidence required for an informed decision.

With regard to reliability, the guidance indicates that the Center will focus on two primary elements (1) how the data is collected (data accrual); and (2) whether the people and processes in place during the data collection and analysis are adequate to ensure data integrity (data assurance). In our experience, the question of reliability is the primary issue during FDA’s review of RWE, and if the sponsor does not have a relationship with the collectors of the data, it can often be next to impossible to provide the level of detail FDA needs.  The data indicates that it will consider the following factors when assessing data accrual:

• the preparedness of individual sites for complete and accurate collection of RWD (e.g., whether there are defined processes, site training and support, and qualified personnel);
• whether a common data capture form was used;
• whether a common definitional framework (i.e., data dictionary) was used;
• adherence to a common temporal framework for collection of key data points;
• the timing of establishing the study plan, protocol, and/or analysis plan relative to collection or retrieval of the RWD;
• the sources and technical methods used for data element capture (e.g., chart abstraction, point of care entry, EHR integration, UDI capture, data records from the device, and linkage to claims data);
• whether patient selection and enrollment criteria minimize bias and ensure a representative real-world population (e.g., all-comer’s design, consecutive patient enrollment);
• the timeliness of data entry, transmission, and availability; and
• whether necessary and adequate patient protections were in place (e.g., methods to protect patient privacy, and need for informed consent as determined by the reviewing IRB and in compliance with FDA regulations).

Finally, FDA will consider the following factors when assessing data assurance.

• the quality of data element population (e.g., whether abstracted from a verifiable source to assess transcription errors or automatically populated through a data extraction algorithm);
• adherence to source verification procedures and data collection and recording procedures for completeness and consistency;
• completeness (i.e., minimized missing or out of range values) of data necessary for specified analyses, including adjustment for confounding factors;
• data consistency across sites and over time;
• evaluation of on-going training programs for data collection and use of data dictionaries at participating sites;
• evaluation of site and data monitoring practices; and
• the use of data quality audit programs.

The guidance encourages sponsors to submit a pre-submission if it plans to use RWE in a premarket submission. While this might be the Center’s preference, there can be advantages to simply submitting the data if it is readily available to the company, the company is not collecting any new information, and it was not otherwise planning to submit a pre-sub (e.g., to address other regulatory questions).  Submitting the RWE in a premarket submission without a pre-sub will force the review team to provide definitive feedback during the review process, unlike the pre-submission process which is non-binding and often results in the reviewers asking for information it would like to see rather than what might be realistic or available. The pre-sub process can, however, be useful to get FDA’s informal feedback regarding the RWE, if it does not delay the company’s premarket submission.

The guidance also explains that an IDE is generally not required to collect RWE. However, according to the guidance, an IDE may be required “if data are being gathered to determine the safety and effectiveness of the device, and the process for gathering the data would influence treatment decisions, such administration would likely not be within the normal course of medical practice, and an IDE may be required.”

Whether this guidance will result in CDRH reviewers being more open to RWE remains to be seen. RWE can provide data that is more representative of a device in actual use.  This, however, may be the very reason why CDRH reviewers would be skeptical of it because it is not as clean as the data sources CDRH generally expects.

Procedures for Advisory Committee Meetings

This guidance is less noteworthy than the RWE guidance. The draft guidance was issued in 2015; however, the draft was intended to replace a 1991 Bluebook memo on PMA panels. The procedures in this guidance are not new. The guidance states that it applies to all advisory committee meetings, including PMA panel meetings, but not dispute resolution panels.

The Center can convene an advisory committee meeting for a variety of issues, including advice on a premarket submission, e.g., PMAs, de novos. The guidance includes factors to consider when a panel for a premarket submission would be appropriate. It also provides model questions to be posed to the panel when it considers a submission. A panel can also be convened for classification or reclassification and other general issues. The guidance provides details regarding the level of expertise of the panel members, and the various procedures for panel meetings.

We hope that these guidances are a sign that we will see additional information coming out of CDRH in the future and that the freeze on guidances has been lifted. Regardless of your political views, guidance is often helpful to industry to help them better understand the Center’s current thinking.