Are You Talking? Because CDRH Says It’s Listening (At Least If You Are In the Digital Health Space): Notes from A Two Day Workshop
February 11, 2018The Center for Devices and Radiological Health (CDRH) says it has a plan for fostering digital health innovation while reimagining the regulatory oversight to provide patients with access to safe and effective digital health products. One step in the plan is developing a Digital Health Software Precertification (Pre-Cert) Program. We blogged on the overall digital health plan here.
As part of developing the Pre‑Cert Program, Dr. Jeff Shuren, Center Director of CDRH, welcomed the audience to a two day workshop at the end of January. Bakul Patel, Associate Director for Digital Health, introduced the program in detail, summarizing its progress to date and sharing lessons learned. The workshop was divided into five panels with opportunities for the audience to ask questions on the first day and three breakout sessions with time to share breakout discussions with the larger group on the second day.
The panels included representatives from among Pre-Cert pilot participants, CDRH Pre-Cert team members (many of whom doubled as workshop moderators), patients, payers, providers, investors, assessors, trade associations, and academic institutions.
The Pre-Cert Program has been touted as an organization-based streamlined regulatory approach for Software as a Medical Device (SaMD) that relies on a demonstrated Culture of Quality and Organizational Excellence (CQOE), where SaMD is defined as software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device. The excellence principles are:
- Patient Safety
- Product Quality
- Clinical Responsibility
- Cybersecurity Responsibility
- Proactive Culture.
One of the overarching workshop themes was that FDA intends to work closely with stakeholders to develop the Pre-Cert Program. The idea is CDRH could “precertify” companies that exhibit CQOE, based on objective criteria. These organizations would qualify to market their lower-risk devices without additional FDA review or with a more efficient premarket review (depending on the product). This focus on the manufacturer, and not the product, would be a significant departure from the Agency’s historical approach to premarket product review.
The following were interesting or notable aspects of the discussion:
The notion of using CQOE principles for evaluation of software firms was received well, but a lot of questions were raised about what metrics, as a practical matter, would be suitable. There was also a question about how frequently companies would need to re-certify. The moderators also asked participants to consider what the certification review would look like and emphasized that it needed to be feasible for both small and large organizations and align with how software firms conduct themselves.
No consensus answers emerged. True to form for a technological crowd, some panelists suggested crowdsourcing to identify key performance indicators (KPIs metrics). The crowdsourcing approach was also suggested for collecting real world data from users on product performance, which would then be shared with developers, who would use this feedback to improve safety and effectiveness in the next software version. This iterative life‑cycle approach has the potential to tie in nicely with the Center’s efforts to promote the National Evaluation System for health Technology (NEST), which is being developed to generate better evidence for medical device evaluation across the total product lifecycle and regulatory decision-making.
We learned that the nine pilot participants each had hosted CDRH for a two-day site visit. The visits were the first steps from CDRH to engage stakeholders. Representing small and large organizations, profit and non-profit, the participants agreed on ensuring high‑quality products reached market sooner and promoting a concept of transparency to ensure confidence and credibility of both the program and its participants.
During these visits, CDRH and the pilot participants had discussed common organizational traits such as having agile processes and a culture that recognizes and supports efforts to ensure quality, but also recognized the difficulty in aligning the five excellence principles with the four validating perspectives (i.e., organizational resource, customer, learning and growth, and process). Another outcome of these interactions was that FDA postulated that a library of KPIs could be used to determine a CQOE, acknowledging that no one set would fit every company.
The panels at the workshop provided interesting perspectives on how a precertification program would be understood and received by their communities. For instance, one panelist from the healthcare stakeholder perspectives panel noted the need for clarification between software reviewed under a product based pathway such as the 510(k) versus software considered under a firm based approach as imagined in the Pre-Cert. She also noted that payers will have to consider reimbursement of apps judiciously. The reimbursement process will take time and could frustrate software developers.
Another panelist from the same group pointed out that healthcare is moving towards a value based system that considers clinical outcomes. He expressed concern about constantly evolving algorithms in software and the potential disruption to clinical workflow. The same panelist wondered how a clinician would obtain information to help choose one company’s software over another.
Another panelist brought up recent experience with electronic health records (EHR). The EHR implementation included cybersecurity, lost data, power failure, and interoperability issues. These same issues could begin to plague digital health software.
A panelist challenged the notion that digital health should be subject to traditional processes. He cited the tradeoffs patients are willing to make in exchange for faster access to technology and highlighted the challenges of costly, time‑consuming clinical studies which could lag well behind innovation.
In general, there was much spirited discussion that showed there is still a long way to go in standing up the Pre-Cert program, something the CDRH moderators acknowledged as well. The good news is that CDRH clearly is interested in developing the Pre‑Cert program with input from the community.
It seems to us that the key issue is the KPI metrics that will be used to assess companies. It is easy to sketch out the broad CQOE principles. Figuring out how to operationalize these principles with concrete KPI metrics is much harder. Some of the metrics that were introduced and discussed during breakout sessions included employee performance, customer engagement, and brand reputation. However, even these metrics are fairly abstract and hard to measure. One wonders how “brand reputation” would even apply to smaller start ups. All in all, it is probably fair to say that the workshop shed light on how difficult it will be to adopt a binding list of KPI metrics and how far off those decisions are.
CDRH has invited those who are willing to share their thoughts on the Pre-Cert Program, whether it be in the form of questions or requests for clarification, to do so in an email to FDAPre-CertPilot@fda.hhs.gov or through a comment on the open docket entitled “Fostering Medical Innovation: A Plan for Digital Health Devices; Software Precertification Pilot Program”.
* Senior Medical Device Regulation Expert