Choosing Between Passive and Active Surveillance in Post-Marketing Vaccine Safety

Passive vs Active Surveillance—What They Are and When to Use Each

Passive surveillance collects Individual Case Safety Reports (ICSRs) from clinicians, patients, and manufacturers via national systems (e.g., VAERS/EudraVigilance analogs). It excels at early pattern recognition because it listens broadly: new Preferred Terms, atypical narratives, or demographic clustering can flag emerging issues quickly. Strengths include speed of intake, rich free-text, and relatively low cost. Limitations are well known: no direct denominators, susceptibility to under- or stimulated reporting, duplicate submissions during media spikes, and variable case quality. In passive streams, you will rely on disproportionality statistics (PRR, ROR, EBGM) to identify unusual vaccine–event reporting patterns that merit clinical review.

Active surveillance uses linked healthcare data (EHR/claims/registries, sometimes laboratory feeds) to construct cohorts with person-time denominators. It supports observed-versus-expected (O/E) checks, rapid cycle analysis (RCA) with MaxSPRT boundaries, and confirmatory designs such as self-controlled case series (SCCS) or matched cohorts. Strengths include stable denominators, control of confounding, and ability to estimate incidence rates and relative risks over calendar time. Limitations include access/agreements, data harmonization, lag, and the need for robust governance and validation packs (Part 11/Annex 11 controls, audit trails, and change control). In practice, sponsors rarely choose one or the other: passive detects, active quantifies, and targeted follow-up adjudicates. To align terminology and SOP structure with regulators, many teams adapt practical PV templates from PharmaRegulatory.in, and mirror public expectations summarized by the U.S. FDA.

Comparative Design Considerations: Data, Methods, and Compliance

Surveillance strategy is as much about design and documentation as it is about databases. Passive streams must prove clean inputs: MedDRA version control, explicit Preferred Term selection rules, ICSR de-duplication criteria (e.g., age/sex/onset/lot match), and translation QA for non-English narratives. Active streams must show traceable ETL pipelines, linkage logic, and privacy safeguards. Both must demonstrate ALCOA (attributable, legible, contemporaneous, original, accurate) and computerized system controls: role-based access, validated audit trails, and time synchronization. Pre-declare decision thresholds in your signal management SOP: what PRR/ROR/EBGM constitutes a “screen hit,” what O/E ratio prompts escalation, which risk windows apply by AESI, and when SCCS/cohort studies begin. Link these rules to your Risk Management Plan (RMP) and Statistical Analysis Plan (SAP) so clinical, safety, and biostatistics use the same vocabulary when evidence evolves.

Operationally, success comes from hand-offs. Write a responsibility matrix: safety scientists review screen hits weekly; epidemiology runs O/E; biostatistics maintains RCA/SCCS code; clinical adjudicates with Brighton criteria; QA reviews audit trails; regulatory owns labels and communications. Keep this map in the PSMF and TMF, with links to datasets and code hashes, so an inspector can trace the path from intake to decision without guesswork.

Analytics That Bridge Both: From PRR to O/E, SCCS, and RCA (with Numbers)

Pre-declare screens and thresholds to avoid hindsight bias. In passive data, a common rule is PRR ≥2 with χ² ≥4 and n≥3; ROR with 95% CI excluding 1; EBGM lower bound (e.g., EB05) >2. Combine these with clinical triage: age/sex clustering, time-to-onset after dose, and mechanistic plausibility. In active data, compute O/E using stratified background rates and biologically plausible windows. Example (dummy): Week W, 1,200,000 second doses to males 12–29; background myocarditis 2.1/100,000 person-years → expected in 7 days ≈ 1,200,000 × (7/365) × (2.1/100,000) ≈ 0.48. Observed 6 adjudicated cases → O/E ≈ 12.5 → escalate. Run RCA weekly with MaxSPRT; if the boundary is crossed, initiate SCCS. A typical SCCS result might show IRR 4.6 (95% CI 2.9–7.1) for Days 0–7, IRR 1.8 (1.1–3.0) for Days 8–21.

Where laboratory markers define cases, declare method capability so inclusion is transparent: high-sensitivity troponin I LOD 1.2 ng/L and LOQ 3.8 ng/L (illustrative) for myocarditis adjudication; platelet factor 4 (PF4) ELISA performance for thrombotic syndromes. Keep quality context close to safety: representative PDE 3 mg/day for a residual solvent and cleaning MACO 1.0–1.2 µg/25 cm² reassure reviewers that non-biological explanations (contamination, carryover) are unlikely. For a plain-language overview of signal expectations and pharmacovigilance vocabulary, the WHO library provides accessible references at who.int/publications.

Designing a Hybrid Surveillance Program: A Step-by-Step Playbook

Step 1 — Define AESIs and windows. Pre-register adverse events of special interest (AESIs) by platform (e.g., myocarditis for mRNA, TTS for vector vaccines) with Brighton definitions and risk windows (0–7, 8–21 days, etc.). Step 2 — Map data flows. Draw a single diagram linking ICSRs → coding/deduplication → screen queue; and registries/EHR/labs → ETL → O/E/RCA/SCCS pipelines. Step 3 — Write thresholds. Document PRR/ROR/EBGM cut-offs, O/E escalation rules, RCA boundary settings, and SCCS triggers. Step 4 — Validate systems. For passive, validate ICSR intake (E2B R3), MedDRA versioning, translation QA, and audit trails. For active, validate linkage logic, ETL checkpoints, time sync, and back-ups under Part 11/Annex 11; containerize analytics and lock code hashes. Step 5 — Staff governance. Run a weekly multi-disciplinary signal review (safety, clinical, epidemiology, biostatistics, quality, regulatory) with minutes, owners, and due dates. Step 6 — Pre-write communications. Draft label/FAQ templates so confirmed signals can be communicated with denominators and plain language quickly.

Keep a one-page crosswalk in the TMF: SOP → dataset → code → output → decision → label. If a screen hit escalates, an inspector should be able to start at the decision memo and walk back to the raw ICSR and the database cut that produced the O/E.

Case Study (Hypothetical): Turning Noisy Signals into Decisions

Week 1–2 (Passive): 20 myocarditis ICSRs in males 12–29 after dose 2; PRR 3.0 (χ² 9.2), EB05 2.2. Narratives cite chest pain and elevated troponin (above assay LOQ 3.8 ng/L). Week 3 (Active O/E): 1.2 M doses administered; background 2.1/100,000 person-years; expected 0.48; observed 6 adjudicated Brighton Level 1–2 → O/E 12.5. Week 4 (RCA): MaxSPRT boundary crossed in Days 0–7; geographies consistent. Week 5–6 (SCCS): IRR 4.6 (2.9–7.1) for Days 0–7; IRR 1.8 (1.1–3.0) for Days 8–21. Decision: add myocarditis to important identified risks; update label/HCP guidance with absolute risks (“~12 per million second doses in young males within 7 days”). Quality check: lots in shelf life; cold chain in range; representative PDE 3 mg/day and MACO 1.0–1.2 µg/25 cm² unchanged—reducing concern for non-biological drivers.

The full package—ICSRs, coding rules, O/E worksheets, RCA configs, SCCS code/outputs, adjudication minutes, and quality context—goes into the TMF and supports rapid, defensible labeling.

KPIs, Governance, and Inspection Readiness: Keeping the System Alive

Measure both surveillance performance and decision speed. Surveillance KPIs: % valid ICSRs triaged ≤24 h, screen hits reviewed per SOP cadence, median days from screen to O/E, RCA boundary checks on schedule, % adjudications completed within SLA. Quality KPIs: audit-trail review completion, ETL error rate, linkage success, reproducibility checks (code hash matches), and completeness scores for ICSRs. Decision KPIs: time to label update, time to DHPC release, and % of decisions backed by confirmatory analytics.

Inspection readiness is narrative clarity plus evidence. Keep a “read me first” note in the TMF that maps SOPs → data cuts → code → outputs → decisions. Store all public communications (FAQs, HCP letters) with the analytics that support them. For method calibration, run periodic negative-control screens so your system demonstrates specificity, not just sensitivity.

How to Detect Safety Signals After Vaccine Licensure

What “Signal Detection” Means—and the Architecture You Need

After licensure, millions of doses transform rare safety events from theoretical risks into observable data. A signal is a hypothesis—a statistically and clinically plausible association between a vaccine and an adverse event that warrants verification. Detecting it reliably requires a layered architecture: (1) passive spontaneous reports (e.g., national ICSRs) for early pattern recognition, (2) active denominated data (claims/EHR networks) for rate estimation, and (3) targeted follow-up for clinical adjudication. The system must connect methods to governance: a PV System Master File (PSMF), SOPs for coding/triage/escalation, and a standing multidisciplinary review (safety clinicians, epidemiologists, statisticians, quality). Documentation lives in the TMF with ALCOA discipline—attributable, legible, contemporaneous, original, accurate—so an inspector can trace any decision back to raw data and time-stamped actions.

Your design question is not “which method is best?” but “how do we make weak evidence in one stream corroborate in another?” Typical flow: disproportionality screens (PRR, ROR, EBGM) flag vaccine–event pairs in spontaneous reports; observed-versus-expected (O/E) analyses check whether counts in a short, biologically relevant window exceed background; sequential monitoring (e.g., MaxSPRT) controls false positives while watching weekly; and confirmatory designs—self-controlled case series (SCCS) or cohorts—quantify risk. Around the analytics, you must enforce clean inputs: MedDRA version control, ICSR de-duplication, stable case definitions (Brighton Collaboration), and causality recording (WHO-UMC). Finally, keep manufacturing/handling context visible so non-biological drivers are excluded: representative PDE (e.g., 3 mg/day residual solvent) and cleaning MACO (e.g., 1.0–1.2 µg/25 cm²) examples help demonstrate state-of-control while safety is assessed.

Disproportionality 101: PRR, ROR, and Empirical Bayes (EBGM)

Spontaneous reporting systems are rich in narratives but poor in denominators. To screen for unusual reporting patterns, use disproportionality statistics. The Proportional Reporting Ratio (PRR) compares the proportion of a specific Preferred Term among reports for your vaccine versus all others; a typical screen is PRR ≥2 with χ² ≥4 and at least 3 cases. The Reporting Odds Ratio (ROR) offers similar insight with confidence intervals; a 95% CI excluding 1 suggests elevation. Empirical Bayes approaches (e.g., EBGM) shrink noisy estimates toward the overall mean, stabilizing small counts; focus on the lower bound (e.g., EB05 >2) to avoid chasing noise. Statistics do not make a signal by themselves—apply clinical triage: time-to-onset, demographic clustering, and mechanistic plausibility. Document versioned data cuts, coding conventions, and deduplication rules in the TMF.

Build your SOP so screen hits trigger a multi-disciplinary review within a fixed cadence (e.g., weekly). Ensure narratives are adjudicated to Brighton levels where applicable (e.g., myocarditis, anaphylaxis). If diagnostics contribute to “rule-in,” declare their performance so decisions are transparent (e.g., high-sensitivity troponin I LOD 1.2 ng/L; LOQ 3.8 ng/L). For adaptable SOP templates and validation checklists that align with GDP/CSV expectations, see PharmaSOP.in. For public regulator terminology and safety expectations you should mirror in submissions, consult the European Medicines Agency.

Observed vs Expected (O/E): Getting Denominators and Windows Right

O/E asks whether the number of events observed after vaccination exceeds what would be expected from background incidence, given the person-time at risk. Build background rates by age, sex, geography, and calendar time from pre-campaign years; adjust for seasonality (splines or month fixed effects). Choose biologically plausible risk windows (e.g., anaphylaxis Day 0–1; myocarditis Days 0–7 and 8–21). Example calculation (dummy): 1,200,000 doses administered to males 12–29 in one week; background myocarditis 2.1 per 100,000 person-years; expected in 7 days ≈ 1,200,000 × (7/365) × (2.1/100,000) ≈ 0.48. If six adjudicated Level 1–2 cases are observed, O/E ≈ 12.5—an elevation that justifies confirmatory analytics. File the worksheet with assumptions, rate sources, and sensitivity analyses (alternative backgrounds, different lags) to your TMF.

Pre-specify how to handle boosters, dose intervals, prior infection, and competing risks. Keep lot/handling context close at hand. If an excursion or shelf-life question arises, cite representative PDE and MACO controls to show the product remained within manufacturing hygiene expectations while you evaluate temporal patterns.

Sequential Monitoring & Rapid Cycle Analysis: Watching Week by Week

When vaccines roll out rapidly, you need near-real-time surveillance that controls false positives. Rapid Cycle Analysis (RCA) applies repeated looks at accumulating data with statistical boundaries (e.g., MaxSPRT) that preserve overall type I error. Choose cadence (weekly), risk windows, and comparators (historical vs concurrent). Simulate operating characteristics before launch so stakeholders understand power and expected time-to-signal under plausible relative risks (e.g., RR 1.5, 2.0, 4.0). Define “stop/go” criteria in the protocol—e.g., cross the boundary for myocarditis in males 12–29 during Days 0–7, then initiate SCCS and clinical adjudication. Document software versions, parameter files, and outputs with checksums; inspectors will ask how boundaries were set and whether the code that ran matches the code in your validation pack.

RCA does not replace clinical review. Every boundary crossing should trigger case-level adjudication (Brighton levels), causality assessment (WHO-UMC), and a check for data or process artifacts (coding changes, batch updates). Keep a signal log with timestamps, decisions, and owners; file minutes from review boards. Align terminology and escalation thresholds with your Risk Management Plan and labeling sections to avoid inconsistent messaging.

Confirmatory Designs: SCCS and Cohorts That Survive Audit

Self-Controlled Case Series (SCCS) compares incidence in post-vaccination risk windows with control windows within the same individuals, controlling for fixed confounders by design. Specify pre-exposure periods to avoid bias (healthcare-seeking before vaccination), adjust for seasonality, and handle time-varying confounders (infection waves). Cohort studies (vaccinated vs concurrent/historical comparators) are intuitive but demand rigorous confounding control: high-dimensional propensity scores, negative controls, and sensitivity to unmeasured confounding. Pre-state primary endpoints, analysis sets, and missing-data rules; register code and lock it under change control. Example (dummy SCCS output): IRR 4.6 (95% CI 2.9–7.1) for myocarditis Days 0–7 and 1.8 (1.1–3.0) for Days 8–21, with an absolute risk difference 3.4 per 100,000 second doses in males 12–29—clinically relevant even if absolute risk remains low.

Be explicit about how confirmatory results drive decisions: label updates, RMP changes, targeted studies, or additional monitoring. Keep quality context tight—confirm that lots remained in shelf-life and within hygiene controls (PDE and MACO examples) so reviewers do not attribute patterns to manufacturing or cross-contamination. Where diagnostics define cases, include laboratory method performance (e.g., cardiac troponin LOD 1.2 ng/L; LOQ 3.8 ng/L) and chain-of-custody.

Case Study (Hypothetical): From Screen to Confirmed Signal in Six Weeks

Week 1–2: Screen. Passive reports show 18 myocarditis cases clustered in males 12–29 after dose 2; PRR 3.1 (χ² 9.8), EB05 2.4. Week 3: O/E. 1.2 M doses administered to males 12–29; expected in 7-day window ≈0.48; observed 6 adjudicated cases → O/E 12.5. Week 4–5: RCA boundary crossed. MaxSPRT triggers for Days 0–7; immediate clinical adjudication confirms Brighton Level 1–2 in most cases. Week 6: SCCS. IRR 4.6 (2.9–7.1) Days 0–7; IRR 1.8 (1.1–3.0) Days 8–21. Action. Update labeling and RMP, issue HCP guidance, and launch a registry. Quality cross-check. Lots were in specification; monitoring shows cold-chain in range; representative PDE and MACO controls unchanged—supporting a biological, not handling, explanation.

Documentation & Submission: Making ALCOA Obvious

Inspection readiness depends on traceability. Keep a crosswalk that links SOPs → data cuts → code → outputs → decisions. Archive: (1) spontaneous-report screen definitions and deduplication rules; (2) background-rate sources and O/E worksheets; (3) RCA simulation and configuration files; (4) SCCS/cohort protocols, code, and outputs; (5) adjudication minutes with case definitions; (6) quality context (shelf-life, cold-chain, representative PDE/MACO evidence). For the eCTD, place analytic reports in Module 5 and the integrated safety summary in Module 2.7.4/2.5, cross-referencing the RMP. Keep terminology consistent across SOPs, dashboards, and labeling to avoid inspector confusion.

Key Takeaways

Signals are hypotheses, not verdicts. Use a layered approach—disproportionality to sense, O/E to anchor, sequential monitoring to watch, and SCCS/cohorts to confirm. Surround analytics with clinical adjudication, causality assessment, and manufacturing/handling context (PDE, MACO, and assay LOD/LOQ where relevant). Document everything with ALCOA discipline. Done well, your signal detection system protects patients, preserves trust, and accelerates clear, defensible decisions.