How to Conduct Sensitivity Analyses for Missing Data Assumptions in Clinical Trials

Missing data in clinical trials introduces uncertainty that can threaten the reliability of results. While primary analyses often assume missing at random (MAR), real-world data may violate this assumption. Sensitivity analyses are therefore essential to evaluate how robust your conclusions are under different missing data mechanisms, particularly Missing Not at Random (MNAR).

This tutorial explores the methods used for sensitivity analyses, including delta-adjusted multiple imputation, tipping point analysis, and pattern-mixture models. We’ll also touch on regulatory expectations and best practices to ensure your study meets standards set by agencies like the USFDA and EMA.

Why Sensitivity Analyses Are Critical

Primary imputation methods (e.g., MMRM, multiple imputation) often rely on MAR. But if data are Missing Not at Random (MNAR), these methods may yield biased results. Sensitivity analyses explore alternative assumptions to assess:

• The robustness of the treatment effect
• The direction and magnitude of bias
• The clinical significance of different assumptions

These analyses should be pre-specified in the Statistical Analysis Plan (SAP) and reported in the Clinical Study Report (CSR), as emphasized in GMP documentation.

Common Sensitivity Analysis Methods for Missing Data

1. Delta-Adjusted Multiple Imputation

This approach modifies imputed values by applying a delta shift, simulating different degrees of missing data bias. It allows trialists to explore the impact of worse (or better) outcomes among those with missing data.

How It Works:

• Standard multiple imputation is performed
• A delta value is added (or subtracted) from imputed outcomes
• Analysis is repeated to observe impact on treatment effect

Example: In a depression trial, if missing values are suspected to come from patients with worse outcomes, a delta of -2 is applied to imputed depression scores.

2. Tipping Point Analysis

This technique identifies the point at which the trial conclusion would change (i.e., lose statistical significance) under worsening assumptions for missing data.

Steps:

1. Systematically vary imputed values for missing data
2. Recalculate treatment effects across scenarios
3. Identify the “tipping point” where the conclusion shifts

This method is especially valuable in regulatory discussions where reviewers request a range of plausible scenarios before accepting efficacy claims.

3. Pattern-Mixture Models (PMM)

PMMs group data by missing data patterns (e.g., completers, early dropouts) and model each separately. They allow for explicit modeling of MNAR mechanisms by assigning different outcome distributions to different patterns.

Advantages:

• Can accommodate both MAR and MNAR scenarios
• Provides flexibility in modeling dropout effects
• Supported by regulators when assumptions are transparently defined

4. Selection Models

These models jointly model the outcome and the missingness mechanism. They require strong assumptions about how dropout depends on unobserved data.

Limitations:

• Complex to implement
• Highly sensitive to model misspecification

Though powerful, selection models are often used in conjunction with simpler methods like delta-adjusted MI to provide a full spectrum of analyses.

When and How to Apply Sensitivity Analyses

When:

• When primary analysis assumes MAR but MNAR is plausible
• When dropout rates exceed 10% and relate to outcome severity
• When regulators request additional robustness evidence

How:

1. Specify methods and rationale in the SAP
2. Use validated tools (e.g., SAS, R) for multiple imputation with delta shifts
3. Present results with confidence intervals and direction of change
4. Document any model assumptions clearly

These practices are outlined in clinical trial SOPs and should align with ICH E9(R1) guidelines on estimands and intercurrent events.

Regulatory Perspectives on Sensitivity Analyses

Agencies like the EMA and CDSCO recommend the inclusion of sensitivity analyses under different assumptions. These analyses:

• Strengthen confidence in trial conclusions
• Demonstrate robustness of efficacy or safety findings
• Support labeling decisions in case of high attrition

Regulators particularly value tipping point analysis for its transparency in evaluating how results depend on missing data assumptions.

Best Practices for Sensitivity Analyses

• Plan analyses during study design—not post hoc
• Use multiple methods to triangulate findings
• Report both adjusted and unadjusted results
• Involve biostatisticians early in protocol development
• Interpret findings with both statistical and clinical context

Practical Example

In a diabetes trial with 15% dropout, primary analysis used MMRM under MAR. Sensitivity analysis using delta-adjusted MI applied values from -0.5 to -2.5 mmol/L for missing HbA1c values. At a delta of -1.5, the treatment effect remained statistically significant. At -2.0, the p-value crossed 0.05. The tipping point was thus delta = -2.0, which was deemed unlikely based on observed dropout characteristics.

This demonstrated that conclusions were robust under realistic assumptions, a crucial component of the sponsor’s submission dossier.

Conclusion

Sensitivity analyses for missing data are no longer optional—they are essential for regulatory acceptance and scientific credibility. By exploring alternative assumptions through techniques like delta adjustment, tipping point analysis, and pattern-mixture models, researchers can demonstrate the reliability of their conclusions despite missing data. A well-planned sensitivity analysis strategy ensures that your clinical trial meets modern regulatory expectations and supports confident decision-making in drug development.

How to Use LOCF, MMRM, and Multiple Imputation in Clinical Trials

Handling missing data in clinical trials is a critical challenge that can significantly affect the integrity and reliability of study results. Patient dropouts, missed visits, and unrecorded outcomes are common, and how we address these gaps can influence regulatory decisions. To ensure robustness and minimize bias, biostatisticians use various imputation methods to estimate missing values based on observed data patterns.

Among the most widely used methods are Last Observation Carried Forward (LOCF), Mixed Models for Repeated Measures (MMRM), and Multiple Imputation (MI). Each technique has strengths and limitations, and their selection must align with the type of missing data—whether it’s Missing Completely at Random (MCAR), Missing at Random (MAR), or Missing Not at Random (MNAR).

This article offers a practical guide for selecting and applying imputation strategies in clinical trial analysis. It also reflects regulatory expectations from the USFDA and EMA, ensuring compliance with ICH guidelines and audit-readiness of your results.

1. Last Observation Carried Forward (LOCF)

What It Is:

LOCF replaces missing values with the last available observed value for that subject. It is simple and has historically been popular, especially in longitudinal studies measuring repeated outcomes such as symptom scores.

How It Works:

Suppose a subject completed Week 4 but missed Week 6 and 8 visits. LOCF will use their Week 4 value to fill in the missing timepoints.

Advantages:

• Simple to implement in most software (R, SAS, SPSS)
• Maintains the original sample size
• Helpful in sensitivity analyses

Limitations:

• Assumes no change after last observation (often unrealistic)
• Can underestimate variability and bias treatment effects
• Discouraged by regulators as a primary analysis method

Despite limitations, LOCF can still be included in pharma SOPs as a supplementary method during sensitivity analysis.

2. Mixed Models for Repeated Measures (MMRM)

What It Is:

MMRM uses all available observed data points and models the outcome over time. It assumes missing data are MAR and incorporates time as a fixed effect and subjects as random effects. Unlike LOCF, it doesn’t impute values explicitly but estimates them via maximum likelihood.

How It Works:

Each subject’s data trajectory contributes to the overall likelihood function. MMRM adjusts for baseline covariates and can accommodate unequally spaced visits and dropout patterns.

Advantages:

• Preferred by regulators when MAR assumption holds
• Statistically efficient and unbiased under MAR
• Handles unbalanced data without needing imputation

Limitations:

• Complex to implement and interpret
• Assumes missingness depends only on observed data
• Inappropriate for MNAR data

MMRM is frequently used in pivotal trials involving longitudinal measurements, such as HbA1c in diabetes or depression scores in CNS studies. It is a key strategy outlined in GMP documentation and SAPs for confirmatory trials.

3. Multiple Imputation (MI)

What It Is:

MI fills in missing data by creating several plausible values based on observed data patterns. These multiple datasets are analyzed separately, and results are pooled using Rubin’s rules to account for imputation uncertainty.

How It Works:

1. Create multiple complete datasets using random draws from a predictive distribution
2. Analyze each dataset using the same statistical model
3. Combine estimates and standard errors across datasets

Advantages:

• Accounts for uncertainty and variability in imputed values
• Applicable under MAR, flexible with data types
• Recommended by EMA and FDA when LOCF or complete-case analysis is inappropriate

Limitations:

• Requires expert statistical knowledge to implement correctly
• Subject to model misspecification risks
• Computationally intensive for large datasets

MI is a robust method often included in primary or secondary analyses of stability studies and efficacy endpoints, especially when data collection spans long periods.

Comparison of Imputation Methods

Regulatory Perspective

Regulators like EMA and CDSCO expect sponsors to:

• Specify primary and sensitivity imputation methods in the Statistical Analysis Plan
• Justify the choice of method based on the assumed missing data mechanism
• Conduct multiple imputation when data is MAR and analyze different patterns
• Perform sensitivity analyses to assess robustness of results

Inadequate handling of missing data can jeopardize trial approval, particularly when survival or patient-reported outcomes are endpoints.

Best Practices for Implementing Imputation

1. Define your imputation strategy in the trial protocol and SAP
2. Use validated software (e.g., SAS PROC MI, R mice package, SPSS missing values module)
3. Avoid relying solely on LOCF for primary analyses
4. Run multiple imputation diagnostics (convergence, plausibility)
5. Include assumptions and imputation details in Clinical Study Reports

Conclusion

Effective handling of missing data through LOCF, MMRM, or Multiple Imputation is essential for unbiased, credible, and regulatory-compliant clinical trial results. While LOCF is simple, it carries assumptions that may not reflect real-world progression. MMRM offers model-based strength for longitudinal designs, and Multiple Imputation provides a statistically sound approach under MAR assumptions. Selection of the right method should be data-driven, pre-specified, and backed by best practices from the fields of pharma validation and biostatistics. In the ever-evolving landscape of drug development, a thoughtful imputation strategy can mean the difference between success and setback.