The COVID-19 pandemic is having a substantial impact on many ongoing clinical studies in all phases of product development. Numerous difficult decisions are being made and steps are actively being taken to ensure the safe execution, or future resumption, of ongoing studies. While patient safety is paramount and should drive all study conduct related decisions, many of these decisions can impact the interpretability of estimates of efficacy at study conclusion. Changes that may seem innocuous on the surface can have a substantial impact on trial integrity, including the validity and reliability of results. Careful consideration, in consultation with a statistician, should be given to the impact that protocol changes, visit schedule amendments, collection methods, and incomplete or missing information will have on the final analysis and interpretation of results.
The FDA Guidance on Conduct of Clinical Trials of Medical Products during the COVID-19 Pandemic makes several thoughtful recommendations regarding methods to maintain the integrity of ongoing clinical studies through the COVID-19 pandemic. While the considerations raised are important to ongoing studies in all phases of clinical research, many of the issues raised take on added importance in the randomized phase 3 confirmatory trial setting. Changes to study design, assessment methods, and visit schedules, in addition to the possibility of higher rates of missing or incomplete information, may make it difficult to obtain an unbiased estimate of differences between treatment and comparator groups in these pivotal efficacy studies.
It is heartening to recognize that some of the study conduct and data-related issues we are presently confronting, including a few of the concepts discussed in the Guidance document, are not new to clinical research and are issues that investigators, protocol sponsors, and statisticians confront frequently, albeit under less difficult circumstances.
While a statistician should be consulted, we are providing some statistical rules of thumb (some are covered directly in the Guidance document) surrounding considerations related to data collection and missing/incomplete information in ongoing studies during the COVID-19 pandemic.
1. Keeping in mind that patient safety is paramount, efforts should be made to collect as much efficacy data as possible within the parameters of the current protocol. Though there will be exceptions, collecting data outside of a visit window or after treatment discontinuation is preferable to collecting no efficacy information whatsoever.
2. Changes to the protocol design may be needed to limit the amount of missing or incomplete efficacy information. However, some changes in study conduct may warrant changes to the planned primary analysis or additional sensitivity analyses.
3. It is important that the reasons for missing data, incomplete data, and patient discontinuations are captured directly, and in an easily identifiable manner, in the case report form. More specifically, this information should be collected in a manner that is readily accessible for analysis and at an appropriate resolution for the degree of missingness (e.g., instrument, visit, patient).
4. Previously unplanned analysis to assess the power of the study before continuing with enrollment may be appropriate. Mature studies which are close to planned enrollment may be sufficiently well-powered to stop early.
5. In many cases, it is likely that incomplete or missing information as a result of COVID-19 conveniently fall into the category of ignorable missing data. The plan for handling missing data due to COVID-19 should be described in the SAP. Sensitivity analyses that explore the missing data space should be planned and documented in the SAP prior to database lock.
6. Documentation in the protocol and SAP are of critical importance. For blinded studies, all decisions and changes to planned data collection, assessment, and analysis should be finalized in advance of database unblinding.
As described in the Guidance, amendments to key elements of efficacy data collection, assessment, and/or analysis should be discussed with the appropriate reviewing division. In consultation with a statistician, study sponsors should prepare now for regulatory interactions to discuss and gain agreement on any proposed changes.
Rob Woolson, MS, JD, Chief Strategist, Biostatistics & Standards for Regulatory Submissions, has 18 years of experience as an applied statistician. Mr. Woolson brings an extensive background of statistical and project leadership experience on US and ex-US regulatory submissions, having led the biostatistical and technical aspects of 12 CDISC-compliant marketing applications, having guided the creation of ISS/ISE statistical analysis plans; integrated analysis dataset design and production; integrated display design and production; and submission-related documentation development. He has conducted statistical analyses in all phases of drug development (Phase I through IV, NDAs, and BLAs) and has led SDTM/ADaM dataset conversion projects in multiple therapeutic areas. Rob works extensively as a consultant advising sponsors on integrated statistical analysis planning, integrated database design, regulatory data submission requirements, and CDISC standards application and implementation. He has authored responses to numerous FDA queries and has represented sponsors at numerous FDA face-to-face meetings, including Advisory Committee meetings. Mr. Woolson’s educational background includes a Bachelor’s degree in mathematics from Northwestern University, a Juris Doctor degree from DePaul University, and a Master’s degree in applied statistics from DePaul University.
Ben Vaughn, MS, RAC, Chief Strategist, Biostatistics & Protocol Design, has over twelve years of experience in clinical research. He has participated in over 25 regulatory submissions and is an expert on CDISC standards. His work has included serving as lead statistician to complete displays and datasets for ISS/ISEs and co-producing the ISS/ISE for multiple products, including six NDAs reviewed by DAAAP. Ben also co-produced the ISE for two opioid products; and provided statistical consultation, display generation and submission work for four separate products for OA knee pain. He has authored responses to FDA queries regarding NDAs, PMAs, IDEs, and SPAs and has represented sponsors in FDA meetings. In the past three years, he has supported five sponsors at DAAAP FDA advisory committee meetings. Additionally, he has represented sponsors in FDA teleconferences and face-to-face meetings for both OA knee pain products and opioid products. His analytic experience includes cross-over studies, survival analysis, non-parametrics, and extensive work with linear and non-linear repeated measure models.