Beyond Passing-Bablok: Advanced Method Comparison for In-Vitro Diagnostics Under CLSI EP09 and IVDR

Series: Advanced Biostatistics for MedTech: Bridging Clinical Evaluation and Engineering

The Regulatory Demand for Analytical Rigour in IVDs

The transition from the In Vitro Diagnostic Directive (IVDD 98/79/EC) to the In Vitro Diagnostic Regulation (IVDR 2017/746) has fundamentally altered the analytical performance expectations for medical devices in the EU/EEA (and Northern Ireland under the Windsor Framework), with Great Britain accepting IVDR-compliant CE-marked devices transitionally. Under IVDR Annex I (General Safety and Performance Requirements), manufacturers must demonstrably justify the analytical performance of their devices, including trueness (bias), precision (imprecision), accuracy, and measurement uncertainty.

For In Vitro Diagnostics (IVDs) and continuous monitoring devices (such as blood glucose meters (BGMs), continuous glucose monitors (CGMs), and high-sensitivity immunoassays) demonstrating analytical performance necessitates a method comparison study against a reference measurement procedure or a validated comparator. The regulatory standard for designing and analysing these studies is the Clinical and Laboratory Standards Institute (CLSI) guideline EP09c, Measurement Procedure Comparison and Bias Estimation Using Patient Samples.

Pardo (2023) introduces the foundational linear regression concepts for comparing an evaluation method to a comparator, initially presenting the simple case where the evaluation method has variability while the comparator method’s variability is assumed negligible. However, applying standard Ordinary Least Squares (OLS) regression to IVD method comparison data often violates this simplifying assumption. Relying on OLS in regulatory submissions for IVDs can lead to biased (attenuated) slope estimates and immediate technical deficiencies during Notified Body reviews. The correlation coefficient often cited in its support is itself a poor measure of agreement. It is inflated by the width of the concentration range, not by true concordance.

This article dissects the advanced statistical methodologies required for IVD method comparison under IVDR, focusing on the transition from OLS to Deming and Passing-Bablok regression, the handling of heteroscedasticity, and the deployment of Bayesian hierarchical models for multi-site analytical validation.

The Statistical Inadequacy of Ordinary Least Squares in Method Comparison

The fundamental premise of a method comparison study is to evaluate the agreement between two measurement methods: the reference method (X) and the evaluation method (Y) across a range of analyte concentrations. The standard linear model is:

In OLS regression, the minimisation objective is the sum of squared residuals, performed exclusively in the Y direction:

The critical assumption underlying OLS is that the independent variable (X) is measured without error, and all random error is contained within the dependent variable (Y). In IVD method comparison, both the reference method and the evaluation method possess measurement error. If X is measured with error, the observed values are X_{obs} = X_{true} + \delta, where \delta represents the random analytical error of the comparator.

When OLS is applied to data where the independent variable contains error, it results in regression dilution (attenuation bias). The OLS estimator for the slope is biased toward zero:

Where \sigma_{true}^2 is the variance of the true analyte concentration and \sigma_{\delta}^2 is the variance of the comparator’s measurement error. In high-precision IVD systems where the analytical measurement range (AMR) is narrow relative to the measurement error, this attenuation bias becomes substantial. Submitting an OLS-derived slope and intercept to a Notified Body under IVDR is statistically indefensible, as it attenuates the slope toward zero, distorting the proportional-bias estimate in a direction that depends on the true slope, while systematically biasing the intercept upward.

Deming Regression: Minimising Weighted Distances

To address the error-in-variables problem, CLSI EP09c recommends regression methods that account for error in both variables, with Deming regression as the default choice. Unlike OLS, Deming regression accounts for measurement error in both X and Y by minimising the weighted sum of squared residuals in both directions.

The Deming minimisation objective is:

Where \sigma_{\epsilon}^2 is the variance of the evaluation method’s measurement error, and \sigma_{\delta}^2 is the variance of the comparator method’s measurement error. This is often parameterised using the variance ratio \lambda = \sigma_{\epsilon}^2 / \sigma_{\delta}^2. The resulting slope estimator (\hat{\beta}_{1,Deming}) is consistent, provided that the variance ratio \lambda is correctly specified (or, in large samples, consistently estimated).

The estimation of \lambda requires a separate precision validation study, typically conducted in accordance with CLSI EP05 (Evaluation of Precision of Quantitative Measurement Procedures). If the variance ratio is assumed to be 1 (i.e., the error variances of both methods are equal), the technique is referred to as orthogonal regression. Assuming \lambda = 1 without empirical justification is a frequent pitfall in regulatory submissions. For IVDs, the reference method (e.g., isotope dilution mass spectrometry) typically exhibits significantly lower variance than the evaluation method (e.g., an enzymatic point-of-care test). A defensible IVDR submission must derive \lambda from replicates measured across the AMR.

Passing-Bablok Regression: Robustness Against Non-Normality and Outliers

While Deming regression resolves the error-in-variables bias, it remains a parametric technique dependent on the assumption that the measurement errors are normally distributed and free of gross outliers. Clinical samples frequently contain outliers due to haemolysis, lipaemia, or interfering substances.

Passing-Bablok regression is a non-parametric alternative recommended by CLSI EP09c when the data distribution is non-normal or when outliers are present. The method estimates the slope (\beta_1) by taking the median of all possible pairwise slopes of the lines connecting points in the dataset, adjusted to be robust to the exchange of X and Y axes.

For n data points, there are N = n(n-1)/2 pairwise slopes S_{ij}:

The Passing-Bablok slope estimator is:

Specifically, it is the median of the shifted pairwise slopes, with a correction for those slopes where the sign flips if X and Y are swapped. The intercept is subsequently derived as:

The primary advantage of Passing-Bablok in the medtech regulatory context is its robustness. A single outlier that would severely distort an OLS or even a Deming regression estimate has effectively negligible influence on the Passing-Bablok median. The estimator tolerates a substantial minority of outliers, providing robustness where standard parametric methods fail. Confidence intervals for the slope and intercept are constructed using the method’s own analytic, rank-based procedure, derived by Passing and Bablok from the distribution of the ranked pairwise slopes. Bootstrap resampling is an available alternative to this when the analytic assumptions are questionable.

The trade-off is statistical power. If the errors are truly normal and homoscedastic, Passing-Bablok is less efficient than Deming regression. The biostatistician must conduct preliminary diagnostics (normality tests, residual plots, and outlier detection (e.g., Cook’s distance)) to justify the selection of Deming versus Passing-Bablok in the Statistical Analysis Plan (SAP) prior to data unblinding.

Handling Heteroscedasticity: The Achilles Heel of Bland-Altman

Bland-Altman analysis (difference plots) is a staple of method comparison, focusing on the agreement between two methods rather than the mathematical relationship. The method plots the difference between methods (Y_i - X_i) against their average (\frac{Y_i + X_i}{2}).

The fundamental limitation of standard Bland-Altman analysis is its assumption of homoscedasticity, that the variance of the differences is constant across the entire measurement range. For the vast majority of IVDs, particularly continuous biosensors and immunoassays, this assumption is violently violated. Analytical imprecision typically scales with analyte concentration.

Consider a continuous glucose monitor evaluated against a laboratory hexokinase reference method. At hypoglycaemic concentrations (e.g., 2.0 \text{ mmol/L}), the absolute difference between methods may be \pm 0.3 \text{ mmol/L}. At hyperglycaemic concentrations (e.g., 20.0 \text{ mmol/L}), the absolute difference may be \pm 3.0 \text{ mmol/L}. If standard Bland-Altman limits of agreement are applied, the limits will be excessively wide at the low end (overestimating clinically acceptable error) and excessively narrow at the high end (masking clinically significant deviation).

Variance-Stabilising Transformations
The traditional regulatory approach to heteroscedastic method comparison data is to apply a variance-stabilising transformation, such as the logarithmic transformation. If the coefficient of variation (CV) is constant, the variance of the differences scales with the square of the mean. A log transformation converts proportional error into additive error.

Logarithmic transformations complicate clinical interpretation.

A Notified Body reviewer assessing a BGM against ISO 15197:2013 criteria needs to evaluate error in clinically meaningful units (mg/dL or mmol/L), not in log-transformed space. Back-transforming Bland-Altman limits from log space yields geometric means and ratios, which are frequently misinterpreted by clinical reviewers.

Weighted Deming Regression and Percentage Error
For heteroscedastic data where transformation is undesirable, Weighted Deming Regression is the mathematically rigorous solution provided a reliable variance function is available. Instead of a constant variance ratio \lambda, the weights are allowed to vary across the concentration range. The variance of the measurement error is modelled as a function of the analyte concentration.

A common model for the variance function in IVDs is a power function:

Where c determines the relationship between concentration and variance. By estimating this function (often via replicates at specific concentration levels), the Deming regression can be weighted inversely proportional to the variance at each point. This ensures that high-concentration points with inherently larger absolute variance do not unduly influence the estimation of the slope and intercept.

Regulatory standards like ISO 15197 for blood glucose monitoring systems explicitly define acceptable error as a combination of absolute and relative criteria (e.g., \pm 15 \text{ mg/dL} for glucose < 100 \text{ mg/dL}, and \pm 15\% for glucose \ge 100 \text{ mg/dL}). The statistical evaluation of method comparison data against such criteria requires the calculation of Parkes Error Grids or Surveillance Error Grids, which map the clinical risk of the observed differences. The biostatistician must integrate the outputs of the method comparison regression (bias and imprecision estimates) into these clinical risk frameworks to satisfy IVDR requirements for analytical performance.

The Total Analytical Error (TAE) Framework

Under IVDR Annex I, Section 9.1(a), manufacturers must specify and justify analytical performance characteristics – including trueness (bias), precision (imprecision), and accuracy. Separately, Section 9.3 establishes metrological traceability requirements, which lean toward the ISO/GUM measurement-uncertainty approach. While the regulation does not prescribe a single consolidated metric, in clinical-chemistry practice, these components are frequently combined into the Total Analytical Error (TAE) framework (formalised in CLSI EP21), which reflects the maximum expected error of a single measurement. While CLSI EP21 does not prescribe this as the only calculation method, a common approximation used in clinical chemistry is the one-sided Westgard model. It is worth noting that TAE coexists with, and is often contrasted against, the measurement-uncertainty approach.

The mathematical formulation of this TAE approximation is:

Or, utilising the coefficient of variation:

The factor 1.65 corresponds to the 95th percentile of a one-sided normal distribution. This reflects the regulatory expectation that 95% of all individual measurements will fall within this bound above (or below) the target value; a symmetric two-sided bound is a different (and separately debated) construction.

While the formula is simple, its application in method comparison is nuanced. The bias is not a single number; it is derived from the method comparison regression. If Deming regression yields a slope \hat{\beta}_1 = 1.05 and an intercept \hat{\beta}_0 = 2.0 \text{ mg/dL}, the bias is concentration-dependent:

Similarly, the imprecision (SD or CV) is derived from the precision study (CLSI EP05) and is typically heteroscedastic. Therefore, the TAE must be calculated and plotted across the entire AMR. A Notified Body will reject a submission that presents a single TAE value if the device exhibits proportional bias or concentration-dependent imprecision. The biostatistician must generate a TAE plot, overlaying the calculated TAE against the manufacturer’s claimed performance specifications, demonstrating that the upper bound of TAE does not exceed the allowable total error (ATE) at any point in the clinical reportable range.

Advanced Application: Bayesian Hierarchical Models for Multi-Site Validation

While IVDR does not explicitly mandate a specific number of testing sites, analytical performance is expected to capture inter-laboratory variability. The convention of using three distinct clinical laboratories stems from general regulatory practice and Notified Body expectations, while device‑specific standards such as ISO 15197 add concrete requirements (three reagent system lots and three sites) without altering the higher‑level IVDR expectation that inter‑laboratory variability be captured. The traditional frequentist approach involves calculating TAE at each site independently and either pooling the data or taking the worst-case scenario.

Pooling the data assumes that inter-site variability is negligible. If inter-site variability is significant (e.g., due to differences in reagent handling, altitude, or operator training), pooling biases the variance estimates. Taking the worst-case site is overly conservative and reduces the statistical power of the validation.

A statistically superior approach—well-supported by methodological best practices—is the application of Bayesian Hierarchical Models (also known as multilevel models). This framework allows for the simultaneous estimation of site-specific performance parameters while borrowing strength across sites to stabilise estimates.

Consider a method comparison study conducted across J sites, with n_j samples per site. The hierarchical model for the method comparison relationship can be formulated as:

Level 1 (Within-Site):

Level 2 (Between-Site):
\alpha_j \sim N(\mu_\alpha, \tau_\alpha^2)

Where:

• Y_{ij} is the evaluation method result for sample i at site j.
• X_{ij} is the comparator method result.
• \alpha_j and \beta_j are the site-specific intercept and slope.
• \mu_\alpha and \mu_\beta are the global (population-level) intercept and slope.
• \tau_\alpha^2 and \tau_\beta^2 represent the variance of the intercepts and slopes across sites.

While this independent univariate normal formulation is the standard teaching model, site slope and intercept are usually correlated. A fully specified model therefore employs a bivariate normal distribution with a covariance term to account for this joint relationship.

Using Markov Chain Monte Carlo (MCMC) methods, as described by Pardo (2023) in the context of Bayesian inference, we can sample from the posterior distributions of these parameters.

The advantage of this approach under IVDR is twofold. First, it provides a formal statistical test for inter-site variability. If the posterior distributions of \tau_\alpha^2 and \tau_\beta^2 are tightly clustered around zero, the manufacturer can statistically justify pooling the data. If they are significantly greater than zero, the hierarchical model naturally accounts for this variability without resorting to worst-case arithmetic. Second, the hierarchical model yields a posterior distribution for the global TAE, incorporating both the within-site analytical uncertainty and the between-site variance. This provides a far more robust and defensible estimate of analytical performance than traditional frequentist aggregation.

Conclusion
Method comparison for IVDs under IVDR is not a matter of plotting Y against X and fitting a trendline in Excel. The regulatory expectations demand rigorous treatment of measurement error, heteroscedasticity, and multi-site variability. The ability to defend a Deming regression slope or a hierarchical TAE estimate before a Notified Body reviewer is what differentiates a regulatory asset from a procedural bottleneck.

The selection of Deming versus Passing-Bablok regression must be statistically justified based on error distributions and outlier diagnostics. Heteroscedasticity must be addressed, either through variance-stabilising transformations or weighted regression techniques, to ensure that Bland-Altman limits of agreement are clinically meaningful across the analytical measurement range. The integration of method comparison data into the Total Analytical Error framework and its extension via Bayesian hierarchical modelling for multi-site studies represents current best practices.

References:

• Pardo, S. A. (2023). Statistical Methods and Analyses for Medical Devices. Springer.
• European Parliament and Council. (2017). Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR). Official Journal of the European Union.
• Clinical and Laboratory Standards Institute. (2018). EP09c: Measurement Procedure Comparison and Bias Estimation Using Patient Samples (3rd ed.).
• Clinical and Laboratory Standards Institute. (2014). EP05-A3: Evaluation of Precision of Quantitative Measurement Procedures.
• International Organization for Standardization. (2013). ISO 15197:2013 – In vitro diagnostic test systems – Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus.