Using Real‑World Evidence to Strengthen Diagnostic Submissions

What Real‑World Evidence Means for Diagnostics (and How Regulators Use It)

Real‑world evidence (RWE) refers to clinical insights generated from data collected outside tightly controlled trials—such as electronic health records (EHRs), laboratory information systems (LIS), claims, registries, biobanks, and pragmatic or decentralized studies. For companion diagnostics (CDx) and other IVDs, RWE can confirm performance in diverse practice settings, characterize rare variants or phenotypes, and demonstrate that an assay’s real‑world use supports the same medical decisions described in its labeling. Regulators increasingly accept well‑designed RWE to complement clinical performance studies, justify label expansions (e.g., new tumor types or specimen matrices), or support bridging when a trial‑stage assay differs from the marketed configuration. Crucially, RWE is not a shortcut; agencies expect traceable provenance, pre‑specified analysis plans, and bias‑mitigation strategies that elevate observational data to decision‑grade evidence.

Two misconceptions commonly slow teams down. First, “RWE equals post‑market only.” In fact, prospective observational cohorts and pragmatic studies can run in parallel with pivotal trials to anticipate post‑market questions and accelerate submissions. Second, “Any big dataset is good enough.” Regulators weigh fitness for purpose—does the dataset reliably capture the analyte, the testing process, and the clinical outcomes tied to test‑guided therapy? For CDx, this means the record should include specimen type (e.g., FFPE vs plasma), platform/version, run controls, and the treatment actually administered based on the result.

Designing Decision‑Grade RWE: Cohorts, Comparators, and Confounding Control

Strong RWE starts with a protocolized plan and a clear question. Are you showing consistent clinical validity (e.g., biomarker‑positive patients benefit more than biomarker‑negative) or confirming analytical performance in routine practice (e.g., lot‑to‑lot precision, invalid rates, limit of detection behavior at the medical cut‑off)? Define your target trial emulation: eligibility, index date (specimen collection), exposure (test result and therapy), and outcomes (ORR, PFS, OS, or response categories relevant to the label). Choose a comparator strategy—concurrent biomarker‑negative patients, a historical external control aligned on line of therapy, or instrument‑to‑instrument comparisons at decentralized sites. Then pre‑specify confounding control: propensity scores, inverse probability weighting, stratification by line of therapy, and sensitivity analyses (e.g., E‑value for unmeasured confounding).

For qualitative CDx (positive/negative), include reclassification at the decision threshold and agreement with an orthogonal method captured in routine care. For quantitative markers (e.g., TMB), define allowable total error at the clinical cut‑off and evaluate bias across sites with mixed models. A practical acceptance framework might set PPA ≥95% and NPA ≥97% with lower 95% CI bounds ≥90%/94% respectively, weighted kappa ≥0.80 for categorical assays, and mean bias ≤10% at the threshold for quantitative results. While these are illustrative, keep criteria anchored to clinical risk—missing a true positive that withholds life‑saving therapy carries more weight than a small numeric bias far from the decision boundary. For process quality, track invalid rate (<3%), turnaround time (median ≤72 h), and repeat‑test frequency (<5%).

Data Architecture and Provenance: From EHR/LIS to Submission‑Ready Tables

Regulatory‑grade RWE depends on traceable data lineage. Start with a data dictionary covering analyte codes, platform versions, lot numbers, and key pre‑analytical fields (fixative, tumor content, time‑to‑freeze). Build an extract‑transform‑load (ETL) pipeline that preserves audit trails, and implement data quality rules (range checks for Ct or read depth, duplicate suppression, specimen‑ID concordance). Where multiple labs contribute data, harmonize units and reference ranges, and map local terminology to controlled vocabularies. For outcomes, link to pharmacy/claims and mortality sources using privacy‑preserving record linkage. Pre‑specify missing‑data strategies (multiple imputation vs complete‑case) and document them in the Statistical Analysis Plan.

Dummy RWE Data Quality Table:

When using decentralized testing, add inter‑site reproducibility (random‑effects models) and cross‑platform concordance. If the marketed assay differs from the trial assay, collect a bridging subset inside the RWE cohort (paired retesting on the new kit) to anchor comparability. For templates and SOP checklists that operationalize these controls, see PharmaValidation.in. For broad principles on quality and submissions, consult FDA’s device resources at fda.gov.

Building the RWE Package for Regulators: Analyses, Sensitivity Checks, and Narratives

Regulators review RWE with two questions in mind: “Are the results unbiased and robust?” and “Are they clinically meaningful for the labeled decision?” Present a hierarchy of analyses: primary (pre‑specified cohort, main endpoint, principal confounding adjustment), key sensitivity (alternative propensity models, negative control outcomes, site‑exclusion stress tests), and supportive (subgroups, time‑varying exposure). For agreement endpoints, include PPA/NPA/OPA with exact CIs and category agreement (kappa). For quantitative assays, add Deming regression, Bland–Altman bias/limits, and reclassification tables at the clinical cut‑off with two‑sided 95% CIs. Provide graphical diagnostics—love plots for covariate balance, funnel plots for site effects, and density overlays around the threshold.

Case Study (Illustrative): A PD‑L1 IHC CDx sought a tissue‑type expansion using a registry linking LIS results and EHR therapies across 42 centers. After pre‑specified propensity weighting, first‑line immunotherapy in PD‑L1‑high patients showed improved real‑world PFS (HR 0.68; 95% CI 0.61–0.76) versus chemo. Inter‑reader category agreement from routine practice yielded weighted kappa 0.83 (95% CI 0.80–0.86), with reclassification at TPS 50% of 3.5% (95% CI 2.7–4.4). Invalid rates and TAT met process targets. The dossier paired these RWE results with a small bridging concordance study on the marketed autostainer, enabling approval of the tissue‑type expansion without a new randomized trial.

Using RWE Under EU IVDR and Beyond: Performance Evaluation and PMS Synergy

Under the EU IVDR, RWE fits naturally into the Performance Evaluation Report (PER) across scientific validity, analytical performance, and clinical performance. Pre‑market observational evidence can prime the PER, while post‑market RWE feeds Post‑Market Surveillance (PMS) and Post‑Market Performance Follow‑up (PMPF). To streamline reviews, structure your PER with pre‑specified questions, robust methods, and traceable data sources; link each claim in the IFU to specific analyses and confidence intervals. Where a Notified Body and EMA must both opine (for CDx per Article 48(3)), highlight the drug‑diagnostic interface—how real‑world testing patterns map to labeled use, and how misclassification risk is monitored and minimized in practice. Practical IVDR insights and consultation mechanisms are available via EMA.

For global alignment, keep a cross‑walk that maps FDA RWE elements to IVDR PMS/PMPF and to Japan PMDA’s expectations for local applicability. When expanding into new regions, an RWE bridging cohort with local samples can reduce the need for large prospective trials if concordance and clinical outcomes mirror the reference population. Always pre‑agree success criteria with agencies and keep statistical code and curation logs audit‑ready.

Operational Playbook: Governance, Ethics, and Data Privacy

Ethical and privacy frameworks are central to RWE. Establish governance that covers data rights, site agreements, de‑identification or pseudonymization, and the legal basis for linkage. Ensure IRB/ethics approvals for observational use, especially where outcomes are abstracted from charts. Build a data monitoring process that tracks QC drift (e.g., weekly invalid rate >5% triggers corrective action), lot changes, and site outliers. For patient safety, define and trend real‑world failure modes (e.g., false negatives at low analyte levels). Provide a CAPA loop so issues detected in PMS translate into updated training, cut‑off clarifications, or software fixes. This continuous loop is what ultimately convinces reviewers that your assay is reliable in messy, real‑life settings—not just at a single center of excellence.

Sample RWE Governance Table:

Numbers That Matter: Sample RWE Performance Snapshot

To make RWE concrete, summarize decision‑critical metrics with targets that reflect clinical risk:

These values are illustrative but align with risk‑based expectations used in many submissions. Always defend targets with clinical reasoning and, where applicable, prior PMA or PER benchmarks.

Conclusion: Make RWE Work Like a Trial—Only Bigger, Broader, and Faster

RWE can accelerate diagnostic approvals and label expansions when it is planned like a trial, curated with audit‑ready provenance, and analyzed with methods that neutralize bias. For CDx especially, pair real‑world concordance and outcomes with tight process controls and a transparent narrative linking test behavior to treatment decisions. Combine this with early agency dialogue and you’ll turn routine practice data into compelling, review‑ready evidence that advances precision medicine at scale.

How to Address Data Quality and Completeness Issues in EHR-Based Research

Electronic Health Records (EHRs) offer rich datasets for real-world evidence (RWE) generation, but they are not without limitations. Pharma professionals and clinical researchers often face hurdles in the form of missing, inconsistent, or poorly structured data. If unaddressed, these issues can compromise patient safety insights, treatment outcome evaluations, and even regulatory acceptance of study findings.

This guide will walk you through practical strategies to ensure data quality and completeness in EHR-based research for robust, reproducible, and regulatory-compliant outcomes.

Understanding the Core Data Quality Challenges:

Several recurring problems can affect the reliability of EHR data in clinical trial planning and RWE generation:

• Missing or incomplete fields: Unrecorded vitals, demographics, or outcomes reduce analytical power.
• Data inconsistencies: Different physicians may document the same diagnosis differently.
• Unstructured data: Clinician notes and scanned PDFs are hard to analyze without NLP tools.
• Coding variations: Use of outdated or localized ICD/SNOMED codes affects interoperability.
• Delayed data entry: Time lags reduce the value of real-time surveillance.

As per EMA guidelines, RWE studies must clearly document how data quality was verified and managed prior to inclusion in study results.

Step-by-Step Solutions to Improve EHR Data Quality:

1. Assess Data Completeness Before Study Start:

   Run exploratory data analysis to calculate the percentage of missing values across critical fields such as age, diagnosis, medication, and lab values. Set thresholds for acceptable completeness (e.g., ≥90%).

2. Use Common Data Models (CDMs):

   Adopt models like OMOP or Sentinel to standardize variables and facilitate mapping across systems. This minimizes ambiguity and improves cross-site comparisons.

3. Implement Automated Validation Rules:

   Use algorithms to detect outliers, duplicates, or biologically implausible values (e.g., systolic BP = 20 mmHg). These automated flags are part of effective GMP documentation practices for informatics tools.

4. Audit Structured vs Unstructured Data:

   Conduct manual chart reviews to estimate the proportion of usable data captured in structured fields vs free text. Invest in NLP only if the unstructured portion is significant and relevant.

5. Clarify Time Stamps and Event Sequencing:

   Ensure every clinical event (admission, lab test, discharge) has accurate and machine-readable timestamps. Inconsistent timing can skew temporal analyses, especially in outcomes research.

6. Apply Data Provenance Tags:

   Track the origin and transformation of each data point—from source system to final analytical variable. This traceability supports GCP and regulatory compliance.

Tools and Technologies for EHR Data Validation:

Several tools can automate data validation, improve completeness, and clean EHR data:

• REDCap: Widely used for collecting structured data and verifying EHR imports.
• OHDSI’s Achilles: Performs automated data quality checks on OMOP CDM databases.
• SAS DataFlux: Enterprise-grade tool for cleaning and standardizing datasets.
• Python & Pandas: Popular scripting tools to apply custom data validation logic.

When implementing these tools, ensure the audit trails are in place, aligning with Pharma SOP examples for electronic data integrity.

Real-World Case Study: Improving Diabetes Dataset Quality

In a real-world study on Type 2 Diabetes, researchers faced 35% missing HbA1c values. A root cause analysis revealed these were entered in physician notes, not structured lab fields. By deploying an NLP engine and retraining staff, completeness rose to 92%—enhancing statistical power and regulatory acceptance.

This emphasizes that StabilityStudies.in methodology applies not only to chemical data but also to digital health records.

Monitoring and Continuous Improvement:

1. Set Data Quality KPIs: Monitor missingness rates, inconsistency ratios, and time-to-entry metrics.
2. Establish Feedback Loops: Share data quality dashboards with clinical data entry teams.
3. Run Quarterly Audits: Sample records for manual review and validate against source documents.
4. Document Corrections: Keep a detailed log of cleaning steps, transformations, and imputation methods.

Continuous monitoring aligns with pharmaceutical validation practices and supports future inspections or publications.

Ethical Considerations in Data Management:

• Ensure de-identified patient data remains anonymous through the entire quality pipeline.
• Communicate data quality limitations transparently in study publications and reports.
• Respect data access boundaries set by institutional review boards and consent protocols.

As per Health Canada, incomplete datasets used in drug safety evaluations may result in regulatory warnings or rejections. Therefore, proactive quality control is critical.

Conclusion: Make Data Quality a Strategic Asset

In the era of data-driven decision-making, the integrity and completeness of your EHR datasets are paramount. By implementing robust validation protocols, leveraging automated tools, and maintaining regulatory transparency, clinical and RWE studies can stand up to scrutiny and deliver trustworthy insights.

Pharma professionals must treat EHR data quality not as a bottleneck, but as a strategic pillar of evidence generation—essential for the credibility of findings and patient safety alike.

How to Handle Missing or Incomplete Chart Data in Retrospective Studies

Retrospective chart reviews serve as a valuable methodology in real-world evidence (RWE) research. However, one recurring challenge is dealing with missing or incomplete data within electronic health records (EHRs) or paper charts. Incomplete data can introduce bias, threaten the validity of results, and raise concerns with regulatory authorities. This tutorial walks clinical trial and pharma professionals through practical, compliant methods for managing missing chart data effectively in retrospective observational studies.

Why Missing Data Is a Critical Problem

Unlike prospective trials where data collection is planned and monitored, retrospective studies depend on existing records not designed for research. As a result, data may be:

• Incomplete (e.g., vital signs recorded sporadically)
• Missing entirely (e.g., no lab values)
• Illegible or inconsistent (e.g., handwritten notes)
• Discrepant across visits or providers

If not handled properly, missing data can cause:

• Loss of statistical power
• Non-representative results
• Skewed conclusions or increased variance
• Regulatory rejection or audit findings

To ensure quality and compliance, it’s essential to implement structured strategies that align with GMP documentation and real-world data standards.

Step 1: Identify Types and Patterns of Missing Data

Before taking action, understand the nature of the missing data. Classify it into:

1. Missing Completely at Random (MCAR): No pattern or link to patient characteristics.
2. Missing at Random (MAR): Missingness related to other observed data (e.g., labs missing more often in elderly).
3. Not Missing at Random (NMAR): Missingness is related to unobserved data (e.g., side effects omitted due to stigma).

Use summary statistics, cross-tabulations, or data visualization tools to explore patterns. Document findings in your validation master plan.

Step 2: Define Acceptable Missing Data Thresholds

Pre-specify acceptable levels of missingness in your study protocol. For example:

• No more than 10% of baseline lab data missing
• At least 75% of medication dosing records available
• Outcome variables must be complete in ≥90% of charts

These thresholds help assess study feasibility and ensure stability indicating methods are interpretable over time. Report compliance with these thresholds in the study results section.

Step 3: Develop SOPs for Handling Missing Data

Create standardized procedures to ensure consistency across data abstractors:

• Use “NA” or predefined codes to label missing fields
• Document reasons for missing data where possible
• Flag any values that require clinical interpretation or review
• Maintain an audit trail of all changes

Refer to Pharma SOP checklist templates to build compliant procedures that cover real-time annotations and backtracking.

Step 4: Attempt Data Retrieval from Alternate Sources

Before labeling data as missing, explore secondary data sources:

• Pharmacy logs for drug details
• Radiology or lab portals for missing reports
• Referral letters and discharge summaries
• Insurance claims data

If using EHRs, search both structured fields and physician notes. Always record the source of retrieved data for traceability as per pharma regulatory compliance.

Step 5: Use Imputation Techniques When Justified

In some cases, statistical imputation can restore dataset usability:

• Mean/Median Substitution: For continuous variables
• Hot Deck Imputation: Replace with value from similar patient
• Multiple Imputation: Generate multiple datasets and aggregate results
• Last Observation Carried Forward (LOCF): For longitudinal data

Imputation should only be used when MAR or MCAR is confirmed. Always describe imputation in your statistical analysis plan (SAP).

Step 6: Track and Report Missingness Transparently

Reporting standards such as STROBE and CONSORT recommend transparent handling of missing data:

• Include flowchart showing records screened, excluded, and analyzed
• List variables with missing data and proportions
• Provide rationale for exclusions and imputation
• Include sensitivity analysis to assess robustness

These practices ensure your study is acceptable to agencies like CDSCO or EMA.

Step 7: Train Abstractors to Minimize Data Loss

Abstractor-related errors can result in apparent missing data. Avoid this by:

• Training on form completion and source navigation
• Defining each variable and acceptable formats
• Running inter-rater reliability checks
• Using dummy charts for practice abstraction

Include missing data protocol in SOP training pharma sessions to reinforce accountability.

Step 8: Implement Quality Checks and Data Audits

Build quality checks into your data workflow:

• Run automated queries for blank or null fields
• Perform double-data entry for high-risk fields
• Flag inconsistencies across related variables
• Conduct regular chart audits for compliance

Record all findings in a deviation log and issue CAPAs as needed to preserve process validation integrity.

Best Practices to Maintain Data Integrity:

1. Never fabricate data — label as “missing” with justification
2. Document every step taken to retrieve or verify information
3. Use SOPs and guidelines to standardize processes
4. Consult biostatisticians when imputing data
5. Prepare a detailed data integrity report before final analysis

Conclusion:

Managing missing or incomplete data in retrospective chart reviews is a nuanced but critical process. By identifying data gaps, applying structured methods, retrieving alternate data, and maintaining transparency, pharma professionals can protect study integrity and uphold regulatory expectations. A disciplined approach not only ensures accurate findings but also enhances the credibility of real-world evidence used in product development, labeling, or safety monitoring.