A Comprehensive Guide to Reviewing a Statistical Analysis Plan in Clinical Research

Introduction: Why SAP Review Matters

The Statistical Analysis Plan (SAP) is a critical document in clinical research that outlines the planned analyses for a clinical trial. Reviewing this document ensures that statistical methods align with the protocol and that the study results will be credible, reproducible, and compliant with regulatory standards. The review of an SAP is a collaborative effort involving biostatisticians, clinical researchers, data managers, and regulatory personnel. Errors or oversights in the SAP can lead to data misinterpretation, trial delays, or even regulatory rejection.

The ICH E9 guideline provides the backbone for SAP development, and reviewing the SAP is part of a Good Clinical Practice (GCP)-compliant workflow. This tutorial provides a practical, detailed approach to reviewing SAPs for entry-level and experienced professionals alike.

Understanding the Structure of an SAP

Before diving into a review, it’s essential to understand the SAP’s structure. Most Statistical Analysis Plans follow a standard format:

• Title Page and Approval Signatures
• Version History and Amendments
• Study Objectives and Endpoints
• Population Definitions (e.g., ITT, PP, Safety)
• Statistical Hypotheses
• Analysis Sets
• Handling of Missing Data
• Derivation Rules for Variables
• Statistical Methods (Primary, Secondary, Exploratory)
• Interim Analysis (if applicable)
• Table, Listing, and Figure (TLF) Shells

Each section must be reviewed for scientific correctness, protocol consistency, clarity, and adherence to regulatory guidance. A mismatch between the SAP and the protocol is a common audit finding noted by agencies such as the FDA.

Key Steps in Reviewing the SAP

1. Cross-Check Against Protocol

Ensure that study objectives, endpoints, and analysis sets in the SAP match the approved protocol. Any discrepancies must be justified with a version history or amendment section.

2. Validate Statistical Hypotheses

Confirm that null and alternative hypotheses are clearly stated and logically aligned with the study design. For example, in a non-inferiority trial, the non-inferiority margin must be justified and statistically sound.

3. Confirm Population Definitions

Check the criteria for Intent-to-Treat (ITT), Per Protocol (PP), and Safety populations. Inconsistencies here can result in data integrity issues. Ensure that inclusion/exclusion criteria are respected in population derivation.

4. Evaluate Handling of Missing Data

Review the imputation strategy. Is LOCF (Last Observation Carried Forward) used inappropriately? Is the missingness mechanism (MAR, MCAR, MNAR) discussed? Sensitivity analyses should be included to test robustness.

5. Analyze the Statistical Methods Section

This is the heart of the SAP. Check whether the methods for primary and secondary endpoints are justified, valid, and reproducible. Confirm that multiplicity adjustments are specified (e.g., Bonferroni, Holm).

Example: If a primary endpoint is a time-to-event variable, is Cox proportional hazards modeling used? Is the proportionality assumption verified?

6. Derivation Logic Review

Ensure derived variables (e.g., “Responder Status”, “Time to Event”) have documented logic. Include dummy data tables or diagrams wherever possible. If derived using SAS macros or R scripts, reference the macro version and location in the code library.

7. Review of Tables, Listings, and Figures (TLFs)

Verify that mock shells (TLF templates) are present and align with SAP-defined endpoints. Ensure column headers are labeled, footnotes are clear, and statistical output is properly formatted.

Example:

8. Assess Documentation Quality and Version Control

All SAPs should have a version history log with date, author initials, and changes made. A signed approval page with dates from statisticians, clinical leads, and QA is essential. Audit trails should track changes for GxP compliance.

Check for proper referencing of external documents such as:

• FDA’s guidance on SAP submission
• PharmaSOP templates for statistical sections

9. Regulatory Expectations and Red Flags

Regulatory bodies like EMA and FDA often issue inspection findings for unclear endpoints, improper multiplicity control, or missing data plans. Ensure that the SAP pre-specifies all analysis elements and avoids “data-driven” modifications.

Red Flags:

• Endpoints defined differently than protocol
• No imputation plan for missing data
• Exploratory analyses not labeled clearly
• Inconsistent or vague derivation rules

10. Checklist Before SAP Sign-Off

• ✅ Does the SAP align with the final protocol version?
• ✅ Are all objectives, hypotheses, and endpoints clearly described?
• ✅ Are TLF shells included and formatted consistently?
• ✅ Are imputation strategies and sensitivity analyses provided?
• ✅ Has the SAP been reviewed by clinical, statistical, and QA teams?

Make sure all reviewers document their observations, and any changes post-review must be version-controlled with audit trails.

Conclusion

Reviewing a Statistical Analysis Plan is not just a formality—it is a regulatory safeguard. A properly reviewed SAP ensures clarity, alignment with the protocol, reproducibility of results, and compliance with international guidelines. Biostatisticians and reviewers must collaborate to ensure quality, mitigate regulatory risk, and uphold the scientific credibility of the trial.

References:

• FDA Guidance on Statistical Aspects of Clinical Trials
• ICH E9 Statistical Principles
• WHO GCP Handbook