How to Handle Missing or Incomplete Chart Data in Retrospective Studies

Retrospective chart reviews serve as a valuable methodology in real-world evidence (RWE) research. However, one recurring challenge is dealing with missing or incomplete data within electronic health records (EHRs) or paper charts. Incomplete data can introduce bias, threaten the validity of results, and raise concerns with regulatory authorities. This tutorial walks clinical trial and pharma professionals through practical, compliant methods for managing missing chart data effectively in retrospective observational studies.

Why Missing Data Is a Critical Problem

Unlike prospective trials where data collection is planned and monitored, retrospective studies depend on existing records not designed for research. As a result, data may be:

• Incomplete (e.g., vital signs recorded sporadically)
• Missing entirely (e.g., no lab values)
• Illegible or inconsistent (e.g., handwritten notes)
• Discrepant across visits or providers

If not handled properly, missing data can cause:

• Loss of statistical power
• Non-representative results
• Skewed conclusions or increased variance
• Regulatory rejection or audit findings

To ensure quality and compliance, it’s essential to implement structured strategies that align with GMP documentation and real-world data standards.

Step 1: Identify Types and Patterns of Missing Data

Before taking action, understand the nature of the missing data. Classify it into:

1. Missing Completely at Random (MCAR): No pattern or link to patient characteristics.
2. Missing at Random (MAR): Missingness related to other observed data (e.g., labs missing more often in elderly).
3. Not Missing at Random (NMAR): Missingness is related to unobserved data (e.g., side effects omitted due to stigma).

Use summary statistics, cross-tabulations, or data visualization tools to explore patterns. Document findings in your validation master plan.

Step 2: Define Acceptable Missing Data Thresholds

Pre-specify acceptable levels of missingness in your study protocol. For example:

• No more than 10% of baseline lab data missing
• At least 75% of medication dosing records available
• Outcome variables must be complete in ≥90% of charts

These thresholds help assess study feasibility and ensure stability indicating methods are interpretable over time. Report compliance with these thresholds in the study results section.

Step 3: Develop SOPs for Handling Missing Data

Create standardized procedures to ensure consistency across data abstractors:

• Use “NA” or predefined codes to label missing fields
• Document reasons for missing data where possible
• Flag any values that require clinical interpretation or review
• Maintain an audit trail of all changes

Refer to Pharma SOP checklist templates to build compliant procedures that cover real-time annotations and backtracking.

Step 4: Attempt Data Retrieval from Alternate Sources

Before labeling data as missing, explore secondary data sources:

• Pharmacy logs for drug details
• Radiology or lab portals for missing reports
• Referral letters and discharge summaries
• Insurance claims data

If using EHRs, search both structured fields and physician notes. Always record the source of retrieved data for traceability as per pharma regulatory compliance.

Step 5: Use Imputation Techniques When Justified

In some cases, statistical imputation can restore dataset usability:

• Mean/Median Substitution: For continuous variables
• Hot Deck Imputation: Replace with value from similar patient
• Multiple Imputation: Generate multiple datasets and aggregate results
• Last Observation Carried Forward (LOCF): For longitudinal data

Imputation should only be used when MAR or MCAR is confirmed. Always describe imputation in your statistical analysis plan (SAP).

Step 6: Track and Report Missingness Transparently

Reporting standards such as STROBE and CONSORT recommend transparent handling of missing data:

• Include flowchart showing records screened, excluded, and analyzed
• List variables with missing data and proportions
• Provide rationale for exclusions and imputation
• Include sensitivity analysis to assess robustness

These practices ensure your study is acceptable to agencies like CDSCO or EMA.

Step 7: Train Abstractors to Minimize Data Loss

Abstractor-related errors can result in apparent missing data. Avoid this by:

• Training on form completion and source navigation
• Defining each variable and acceptable formats
• Running inter-rater reliability checks
• Using dummy charts for practice abstraction

Include missing data protocol in SOP training pharma sessions to reinforce accountability.

Step 8: Implement Quality Checks and Data Audits

Build quality checks into your data workflow:

• Run automated queries for blank or null fields
• Perform double-data entry for high-risk fields
• Flag inconsistencies across related variables
• Conduct regular chart audits for compliance

Record all findings in a deviation log and issue CAPAs as needed to preserve process validation integrity.

Best Practices to Maintain Data Integrity:

1. Never fabricate data — label as “missing” with justification
2. Document every step taken to retrieve or verify information
3. Use SOPs and guidelines to standardize processes
4. Consult biostatisticians when imputing data
5. Prepare a detailed data integrity report before final analysis

Conclusion:

Managing missing or incomplete data in retrospective chart reviews is a nuanced but critical process. By identifying data gaps, applying structured methods, retrieving alternate data, and maintaining transparency, pharma professionals can protect study integrity and uphold regulatory expectations. A disciplined approach not only ensures accurate findings but also enhances the credibility of real-world evidence used in product development, labeling, or safety monitoring.