To those of us who regularly create or review adverse event (AE) incidence tables for randomized controlled trials, it may come as a surprise that your typical AE incidence table can be misleading if data was combined from more than one trial. This is due to “Simpson’s paradox,” which, simply put, is the phenomenon that the mere grouping of data can introduce confounding or bias otherwise not present. Getting a bit more technical, this occurs because a confounding variable is introduced in the act of grouping the data; in the case of stacking datasets from studies with differing randomization ratios, the “probability” of being a treatment group is now related to study. Essentially, you could be giving a disproportionate weight to a particular arm/study combination. This means a crude unadjusted AE percentage can look larger or smaller than was apparent from the study-level percentages.

As of late, FDA has placed a renewed emphasis on this phenomenon and may expect you to report AE percentages that are “adjusted for study size” in your Integrated Summary of Safety (ISS). This is particularly true if you are stacking data from trials with differing randomization ratios and/or studies that are of significantly different sizes. There are various ways to address this (including constructing test statistics based on certain weighted averages), but one simple method is to create a “study-size” weighted average, as shown below (Chuang-Stein & Beltangady).

AdjPᵢ = Σⱼnⱼpᵢⱼ / Σⱼnⱼ

where AdjPᵢ is the study-size adjusted proportion for the i ^th arm, nⱼ is the number of subjects in the j ^th study, and pᵢⱼ is the proportion of subjects in the i ^th arm in the j ^th study with the event (i.e., your “unadjusted” percentage). Note that Σⱼnⱼ is simply the size of your pooled population; so this is essentially weighing by the proportion of subjects in a given study out of the pooled population.

Planning your ISS analyses?  Here are some questions to consider:

• Ask the sponsor/ISS author if they want both adjusted and unadjusted percentages reported. Unadjusted percentages may not be necessary, but some sponsors may like those shown as well.
• Decide which analyses/displays should include adjusted percentages and what methodology to use.
• See if you can get FDA buy-in on your plan. It may come down to a “review issue,” but if you have an upcoming Type C or pre-NDA meeting and are submitting your ISS Statistical Analysis Plan anyways, it may behoove you to clearly lay out your plan and see if they remark ahead of time.

Need additional clarification? Contact us to speak with one of our Biometrics Regulatory Experts.

Sources:

• Chuang-Stein C, Beltangady M. Reporting cumulative proportion of subjects with an adverse event based on data from multiple studies. Pharm Stat. 2011 Jan-Feb;10(1):3-7. doi: 10.1002/pst.397. PMID: 20073040.
• PHUSE Analysis & Display White Papers Project Team, Analysis and displays associated with adverse events: Focus on adverse events in phase 2-4 clinical trials and integrated summary documents (Doc ID: WP-003) Version 1.0. 2017.

Julie Gubitz, MS, MPH, JD, Senior Biostatistician, has six years of experience providing statistical support for Phase 1-3 clinical trials. Ms. Gubitz’s experience includes supporting NDA submissions, consulting on protocol development, writing detailed statistical analysis plans, preparing CDISC-compliant specifications for analysis (ADaM) databases, creating CDISC-compliant submissions packages, specifying and performing statistical analyses, preparing displays and reports, and providing safety evaluations for data monitoring committees.  Ms. Gubitz has supported both federally funded projects and commercial projects with biotech/pharmaceutical companies. She has provided statistical support in multiple therapeutic areas, with a focus on pain and autoimmune disorders.