Equivalence and Non-Inferiority Margins in 510(k) and CE Mark Submissions

Series: Advanced Biostatistics for MedTech: Bridging Clinical Evaluation and Engineering

The Regulatory Trap of “Substantial Equivalence”

Regulatory pathways for medical devices rely heavily on the concept of equivalence. Under the US FDA 510(k) framework, a manufacturer must demonstrate that a new device is “substantially equivalent” to a legally marketed predicate. In Europe, the EU Medical Device Regulation (MDR 2017/745) Article 61 requires clinical evaluation data to demonstrate equivalent safety and performance when relying on predicate device data. As the MDR legacy-device transition deadlines approach (31 December 2027 for higher-risk devices (Class III and Class IIb implantable) and 31 December 2028 for lower-risk classes under Regulation (EU) 2023/607) Notified Bodies are applying unprecedented scrutiny to these equivalence claims. Equivalence demonstrations effectively require comparable statistical standards to pivotal clinical investigations.

That said, the degree of statistical evidence required for a particular device submission is proportional to the risk classification of the device and the degree of reliance on predicate data. Class I or low-risk IIa devices with a well-established predicate may suffice with descriptive comparisons and basic confidence intervals. Class IIb, III, and implantable devices, or any submissions introducing novel technological changes, increasingly require the formal TOST or non-inferiority frameworks outlined below.

Historically, substantial equivalence was sometimes asserted retrospectively by leveraging a non-significant p-value from a conventional two-sided superiority test:

H_0: \mu_1 = \mu_2 \quad \text{vs.} \quad H_1: \mu_1 \neq \mu_2

Failing to reject H_0 merely indicates insufficient power to detect a difference, not proof of sameness. The institutional inertia to treat non-significant results as equivalence remains a persistent hurdle in device validation. If a clinical evaluation report (CER) submitted to a Notified Body relies on this absence-of-evidence fallacy, it constitutes a fatal statistical flaw that will trigger a Major Non-Conformity.

To satisfy regulatory expectations under FDA guidance and MDR requirements, the biostatistician must enforce a paradigm shift in hypothesis formulation. This requires mandating formal equivalence and non-inferiority testing frameworks, including the pre-specification of clinically justified margins and the application of specialised power analyses.

The Two One-Sided Tests (TOST) Framework

The correct approach to demonstrating equivalence is the Two One-Sided Tests (TOST) procedure, originally formalised by Schuirmann (1987). Instead of testing for a difference, the TOST framework tests whether the true difference falls within a pre-defined equivalence interval bounded by -\Delta and +\Delta.

The hypotheses are structured as:

H_0: |\mu_1 - \mu_2| > \Delta \quad \text{vs.} \quad H_1: |\mu_1 - \mu_2| \leq \Delta

Where \Delta is the equivalence margin. To test this, the procedure decomposes the hypothesis into two separate one-sided tests:

Test 1 (Lower Bound):

H_{0a}: \mu_1 - \mu_2 \leq -\Delta \quad \text{vs.} \quad H_{1a}: \mu_1 - \mu_2 > -\Delta

Test 2 (Upper Bound):

H_{0b}: \mu_1 - \mu_2 \geq +\Delta \quad \text{vs.} \quad H_{1b}: \mu_1 - \mu_2 < +\Delta

Both null hypotheses must be rejected to conclude equivalence. A critical property of the TOST procedure is its handling of the Type I error rate (\alpha). The TOST is an application of the Intersection-Union Test (IUT) principle. In an IUT, the overall null hypothesis is the union of individual null hypotheses, and the alternative is the intersection. The overall Type I error rate is bounded by the maximum of the individual test error rates and thus no multiplicity adjustment (such as Bonferroni) is required. Each one-sided test is evaluated at the 100(1-\alpha)th percentile, not the 100(1-\alpha/2)th percentile used in conventional two-sided testing.

This amounts to constructing a 100(1-2\alpha)\% two-sided confidence interval around the observed mean difference and verifying that the entire interval falls within the equivalence bounds [-\Delta, +\Delta]. For a standard 95% confidence level (implying \alpha = 0.05), a 90% two-sided confidence interval is calculated and compared against the margins.

The Justification of the Equivalence Margin (\Delta) in a Medical Device Context

Perhaps the most contentious aspect of equivalence testing in medical device submissions is the justification of the margin, \Delta. In pharmaceutical bioequivalence studies, the margins (80% to 125% for pharmacokinetic parameters) are codified in regulation. No such codified margins exist for medical devices.

The limits of equivalence or non-inferiority for medical devices are determined based on functionality and clinical context, not by arbitrary statistical thresholds. An engineer cannot set \Delta based on manufacturing tolerances, nor can a statistician set it based on standard deviations. The margin must represent the largest difference that is clinically acceptable within the specific context, such that a difference larger than \Delta would impact patient safety or clinical performance (Pardo, 2023).

Margin Derivation Strategies

Under MDR, clinical equivalence must be demonstrated across clinical, biological, and technical parameters. For a CER or 510(k) submission, the derivation of \Delta must be documented using a multi-faceted approach:

1. Clinical Significance Thresholds: \Delta may be anchored to the smallest change in a clinical endpoint that a patient or clinician would consider meaningful.
2. State of the Art (SOTA) Benchmarking: Under MDR, \Delta can be derived from the observed performance variance of the SOTA. If the SOTA devices on the market exhibit a standard deviation (σ_SOTA) in a specific metric, a clinically justified fraction of this observed variability may be proposed as the equivalence margin. For example, a manufacturer might argue that a difference not exceeding, say, 25% of the SOTA’s inherent variability is clinically negligible. This fraction must be explicitly anchored to patient-relevant outcomes and clinical expert consensus.
3. Analytical Performance Goals: For In Vitro Diagnostics, \Delta is often tied to Total Analytical Error (TAE) specifications. If a clinical laboratory allows a maximum TAE of 10%, the equivalence margin for bias in a method comparison study might be set at \pm 5\% to leave adequate room for imprecision.

If the margin is set too wide, the Notified Body will reject the claim as clinically meaningless. If set too narrow, the required sample size will become prohibitive. Margin derivation must be clinically justified and cannot be based solely on statistical convenience. For example, in orthopaedic implants, a difference in Range of Motion (ROM) of less than 5 degrees might be deemed clinically irrelevant, establishing Δ=5. Alternatively, Δ may be anchored to clinical significance thresholds or the observed performance variance of the SOTA. Any fraction of SOTA variance proposed must be explicitly justified in the Statistical Analysis Plan (SAP) with clinical and engineering rationale.

The Danger of Biocreep

A specific regulatory concern in equivalence testing is “biocreep.” If a manufacturer selects a slightly inferior predicate and sets a wide equivalence margin, the new device may be statistically equivalent to the predicate but clinically inferior to the original standard of care. Over successive iterations of 510(k) submissions, the performance of devices on the market can progressively degrade. Notified Bodies and the FDA are acutely aware of biocreep. To defend against this accusation, the SAP must demonstrate that the chosen margin \Delta is clinically relevant in the context of the SOTA, not merely in the context of the chosen predicate.

Power and Sample Size for Equivalence Testing

Power calculations for equivalence tests diverge significantly from superiority tests. In a superiority trial, power is at a minimum when the true difference is zero, and increases as the true difference moves away from zero. In an equivalence trial, power is maximised when the true difference is zero. A the true difference approaches the equivalence margin \Delta, the power drops precipitously toward the Type I error rate (\alpha).

To illustrate: Δ = 5°, n = 100 per arm, SD = 10°, α = 0.05

True difference	Superiority power	Equivalence power
0°	0.050	0.940
1°	0.108	0.875
2°	0.291	0.680
3°	0.560	0.407
5° (= Δ)	0.940	0.050
7°	0.998	0.001

The sample size for a two-group equivalence test (assuming equal variances and equal sample sizes) is derived using the non-central t-distribution. The non-centrality parameter (\delta_{ncp}) is a function of the assumed true difference (\Delta_a), the margin (\Delta_0), the standard deviation (\sigma), and the sample size per group (n):

\delta_{ncp} = \frac{|\Delta_a - \Delta_0|}{\sigma \sqrt{2/n}}

Assuming the true difference is zero (\Delta_a = 0), the formula simplifies. Regulatory bodies frequently require sponsors to power the study assuming a conservative non-zero true difference (e.g., assuming the true difference is 20% of the margin). The power (1 - \beta) is the probability that the test statistic exceeds the critical value under the alternative hypothesis.

A large-sample normal approximation for the sample size per group (assuming the true difference is zero, \Delta_a = 0) is:
n \approx \frac{2\sigma^2 (z_{1-\alpha} + z_{1-\beta/2})^2}{\Delta_0^2}
Where z_{1-\alpha} and z_{1-\beta/2} are the standard normal quantiles. Note that z_{1-\beta/2} is used because power is distributed across both tails of the rejection region when the true difference is zero. This is an approximate formula. Exact calculations using the t‑distribution (as implemented in PowerTOST) will give slightly larger required n.

A critical error in submission preparation is using standard superiority sample size calculators for equivalence designs. Doing so routinely underpowers the study which leads to failed verification and validation (V&V) activities.

The TOST procedure can be inherently conservative. Test statistic relies on the non-central t-distribution which means the actual Type I error rate is \leq \alpha. At the boundary the test statistic follows a central t‑distribution (non‑centrality = 0), giving exactly α. Away from the boundary it is conservative. While statistically safe, it means that standard sample size formulas (which rely on normal approximations) may slightly underestimate the required sample size. Exact power calculations using purpose-built equivalence tools are critical for regulatory submissions. These can be achieved with the PowerTOST or TOSTER packages in R (which implement the non-central t solution for TOST), or dedicated software such as PASS or SAS PROC POWER. General superiority-power functions such as base R’s power.prop.test or the pwr package’s pwr.t.test are not designed for TOST and will misstate the required sample size.

Non-Inferiority: The One-Sided Alternative

In many medical device regulatory applications, particularly those involving design modifications intended to reduce cost, improve usability, or minimise invasiveness, the goal is not to prove exact equivalence, but to prove that the new device is “no worse” than the predicate. This is the domain of non-inferiority testing.

Non-inferiority tests are one-sided. The hypotheses are:

H_0: \mu_1 - \mu_2 \leq -\Delta \quad \text{vs.} \quad H_1: \mu_1 - \mu_2 > -\Delta

Here, \Delta is the non-inferiority margin. If higher outcome values are better (e.g., tensile strength, sensitivity), the new device is non-inferior if its mean is not more than \Delta units below the predicate’s mean. The statistical validation of this claim requires constructing a 100(1-\alpha)\% one-sided confidence interval and verifying that the lower bound does not cross -\Delta.

Non-inferiority testing is particularly relevant under MDR for justifying the use of historical data. If a manufacturer modifies an existing device (e.g., changing a polymer supplier), the clinical evaluation can leverage the historical safety data of the predicate, provided a non-inferiority test on the critical quality attributes (CQAs) is statistically valid.

The Constancy Assumption in Medical Device Validation

A key statistical requirement for non-inferiority is the constancy assumption, borrowed from pharmaceutical context (ICH E10). This assumption requires that the historical predicate’s performance, established in previous studies, would remain unchanged if those studies were conducted today. In medical device development, where surgical techniques, imaging technologies, and patient management protocols evolve rapidly, the constancy assumption is often violated. If the standard of care has improved since the predicate was launched, a non-inferiority margin based on the predicate’s historical performance may be clinically obsolete. A biostatistician must assess whether the historical data supporting the margin is still valid in the current clinical context.

Equivalence and Non-Inferiority for Binary Outcomes

While continuous outcomes are relatively straightforward, binary outcomes (e.g., pass/fail rates, clinical success rates, sensitivity) present a more complex challenge. The sampling distributions of differences in proportions (p_1 - p_2) and odds ratios (OR) are more difficult to approximate reliably in small samples, see Pardo (2023).

For binary outcomes, the TOST framework is applied to the difference in proportions. The equivalence hypotheses are:

H_0: |p_1 - p_2| > \Delta \quad \text{vs.} \quad H_1: |p_1 - p_2| \leq \Delta

To compute confidence intervals for the difference in proportions or the odds ratio, standard normal approximations (e.g., the Wald interval) have poor coverage probability when sample sizes are small or when proportions are near 0 or 1. Methods like the Farrington-Manning score test or exact unconditional tests are standard regulatory choices, however as Pardo notes, there is a lack of a simple closed-form sampling distribution for these metrics. Bootstrap resampling is a robust, assumption-light alternative.

Bootstrap Resampling for Equivalence Intervals

The bootstrap procedure for an equivalence test of an odds ratio involves:

1. Computing the observed sample proportions \hat{p}_1 and \hat{p}_2, and the observed odds ratio \widehat{OR}.
2. For N repetitions (e.g., N = 10,000), drawing a random sample of size n_1 with replacement from group 1, and a random sample of size n_2 with replacement from group 2.
3. Calculating the resampled odds ratio OR^*_i for each iteration i.
4. Determining the 100(\alpha)th and 100(1-\alpha)th percentiles of the resulting OR^* distribution to form the two-sided 100(1-2\alpha)\% confidence interval.
5. Verifying that this percentile interval falls entirely within the pre-specified equivalence bounds for the odds ratio.

When submitting binary outcome data for regulatory review the percentile bootstrap method is a robust, assumption-light way of validating equivalence.

Variance Non-Inferiority: The Statistical Evaluation of Imprecision

In medical device validation there is a regulatory requirement to control imprecision. Where a new device’s repeatability, reliability, or overall dispersion is a critical determinant of safety or clinical performance, Notified Bodies and the FDA require a formal statistical justification of variance to ensure the new device is not unacceptably more variable than the comparator. The frequency of this requirement depends on the device type and the specific performance characteristics under evaluation.

In medical device validation the objective is often to demonstrate that the evaluation device’s standard deviation (\sigma_e) is “no worse” than the comparator’s (\sigma_c) by more than a pre-specified factor k_0. This is a non-inferiority test on standard deviations.

The hypotheses are:

H_0: \sigma_e > k_0 \sigma_c \quad \text{vs.} \quad H_1: \sigma_e \leq k_0 \sigma_c

Where k_0 \geq 1. If k_0 = 1.2, the assertion is that the new device’s standard deviation is no more than 20% larger than the predicate’s.

The test statistic utilises the F-distribution. Under the null hypothesis, the ratio of the evaluation variance to the scaled comparator variance is distributed as F with (n_e - 1) and (n_c - 1) degrees of freedom:

F = \frac{S_e^2}{k_0^2 S_c^2}

Where S_e and S_c are the sample standard deviations. The null hypothesis is rejected if F < F_{\alpha, n_e-1, n_c-1}, where F_{\alpha} is the \alpha-quantile of the F-distribution.

The power of this test depends on the true ratio of variances. Let the true ratio of standard deviations be k_1 = \sigma_e / \sigma_c. We define the variance ratio parameter as \delta_V = (k_1 / k_0)^2. As \delta_V decreases (i.e., the evaluation device is actually more precise than the comparator), the probability of rejecting the null hypothesis increases. As Pardo (2023) details, the power curve for this non-inferiority test is asymmetric, and sample size justification must be derived iteratively. The power is formally calculated as the probability that the test statistic falls below the critical value:

P \left( F_{n_e-1, n_c-1} < \frac{F_{\alpha, n_e-1, n_c-1}}{\delta_V} \right) , where δ_V = (k₁ / k₀)²

This is typically evaluated via simulation or the F‑distribution with degrees of freedom (n_e-1, n_c-1) and scale factor δ_V to provide adequate power at the assumed true variance ratio.

Documenting the Statistical Evicence for Regulators

To survive a Notified Body audit or an FDA 510(k) review, the statistical justification of equivalence or non-inferiority must be documented exhaustively. A sound submission includes:

1. Pre-Specification: The SAP must specify the TOST or non-inferiority framework, the margin (\Delta or k_0), the \alpha level, and the power calculation prior to data collection. Post-hoc margin adjustment is strictly prohibited.
2. Margin Justification: A dedicated section of the CER must justify \Delta using clinical, engineering, and SOTA data. The justification must explicitly state why a difference larger than \Delta would be unsafe or clinically unacceptable.
3. Assumption Verification: The submission must include diagnostic plots verifying the assumptions of the test (e.g., normality of residuals, homoscedasticity). If assumptions are violated, the submission must detail the use of robust alternatives (e.g., Passing-Bablok, bootstrap).
4. Intent-to-Treat vs. Per-Protocol: For clinical investigations, equivalence analyses are typically evaluated on both Per-Protocol (PP) and Intent-to-Treat (ITT) populations as co-primary analyses. In superiority trials, ITT is the conservative choice because protocol deviations tend to dilute the treatment effect. In equivalence and non-inferiority trials, protocol deviations can artificially narrow the difference between devices, making ITT anti-conservative. Conversely, PP can be biased if dropouts are related to treatment failure. Current FDA and EMA guidance therefore expects both ITT and PP to be analysed and the reasons for the differences in population, if any, must be explored and documented.

Summary

To demonstrate substantial equivalence or non-inferiority is a regulatory cornerstone of the medical device industry. It cannot be achieved through the misapplication of standard superiority testing. Application of the TOST framework is essential for regulatory success, combined with clinically justified margins and appropriate power analyses using the non-central t-distribution. For binary outcomes and complex variance comparisons, bootstrap resampling and non-central F-distributions provide the robust statistical defence required to satisfy Notified Body expectations.

References:

1. Pardo, S. A. (2023). Statistical Methods and Analyses for Medical Devices. Springer.
2. European Parliament and Council. (2017). Regulation (EU) 2017/745 on medical devices (MDR), Article 61 and Annex XIV. Official Journal of the European Union.
3. US Food and Drug Administration. (2019). The 510(k) Program: Evaluating Substantial Equivalence in Premarket Notifications [510(k)].
4. Schuirmann, D. J. (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15(6), 657-680.
5. International Council for Harmonisation. (2000). ICH E10: Choice of Control Group and Related Issues in Clinical Trials.
6. US Food and Drug Administration. (2016). Non-Inferiority Clinical Trials to Establish Effectiveness: Guidance for Industry.
7. European Medicines Agency. (2005). Guideline on the Choice of the Non-Inferiority Margin.