Introduction

Under the EU Medical Device Regulation (MDR) and the In Vitro Diagnostic Regulation (IVDR), the scrutiny applied to clinical evidence has fundamentally shifted. Notified Bodies and the UK MHRA are no longer simply checking for the existence of clinical data; they are interrogating the methodological soundness of how that data is generated, analysed, and interpreted.

At the centre of this regulatory attention is the Statistical Analysis Plan (SAP). The SAP is a binding, operational execution manual that translates clinical hypotheses into evidence generating methodology. Rather than a static theoretical blueprint, it is an active document that forms the methodological foundation of the Clinical Evaluation Report (CER) or Performance Evaluation Report (PER). 

For study Statisticians and Statistical Programmers, the SAP has a critical operational function: It serves as a formal workflow document that standardises how the statistician and programming team process study data from database lock to final reporting. It defines the sequential steps for data handling, outlines the precise logic for analysis, and sets the technical specifications for producing final tables and listings. By establishing this structured workflow before unblinding, it ensures the entire evaluation process is objective, consistent, repeatable and fully auditable.

What is needed to draft a SAP?

A robust SAP cannot be drafted in a vacuum. A statistician requires a comprehensive stack of finalised, or near-finalised, source documents. Without them, the SAP will inevitably contain contradictions, miss critical derivations, or misdefine analysis cohorts.

It is equally important to understand the direction of the clinical workflow. Certain documents are prerequisites that feed into the SAP, while other critical operational documents are downstream outputs driven by the SAP.

Before SAP drafting begins, the author must integrate the following inputs:

1. The final study protocol (QC’d by the clinical team, minus the statistical sections)

The clinical sections of the study protocol are non-negotiable prerequisites. The statistician requires the finalised clinical objectives, the exact definitions of the endpoints, and the inclusion and exclusion criteria. The clinical team must QC and lock these clinical elements before the SAP can be drafted. The statistical sections within the study protocol itself are often high-level and are frequently refined concurrently with, or driven directly by, the granular specifications of the SAP. Attempting to write a SAP while the clinical narrative of the study protocol is still shifting guarantees endless rewrites.

2. The annotated (electronic) Case Report Form (eCRF) and completion guidelines

A blank Case Report Form marked up with the exact database variable names, question text, and valid data ranges for every data point collected at the clinical sites.

The eCRF is the absolute blueprint of how data is captured. A statistician cannot define derivations without knowing the exact structure of the incoming data. If a SAP specifies an analysis of “change from baseline in vessel diameter,” but the eCRF only captures categorical data, the SAP is fundamentally flawed. The eCRF grounds the statistical methodology in the reality of data collection and enables the code for analysis and TLFs to be developed.

3. The official Data Management Plan (DMP)

The DMP outlines how data will be cleaned, coded, and handled. The statistician needs to understand the rules for data cleaning, as this impacts how missing, partial, or implausible data will be treated in the analysis. It also defines important derivations, such as how adverse events are coded using standard dictionaries like MedDRA. 

4. Instructions for use (IFU) and reference standard algorithm

For medical devices, the IFU defines the intended use population, which dictates the primary analysis cohort. For IVDs under IVDR, the algorithm defining how the “truth” is adjudicated is critical. The statistical analysis of sensitivity and specificity is entirely dependent on the integrity of this algorithm.

5. Data standardisation frameworks (if applicable) 

If the study is utilising CDISC standards (such as SDTM and ADaM), which many global MedTech companies adopt for data harmonisation or to support concurrent FDA submissions, the statistician requires these implementation guides as inputs. This ensures the SAP derivations map correctly to standard regulatory data structures. If CDISC is not being used, the SAP cannot simply ignore dataset architecture. In the absence of external standards, the SAP must explicitly define the structure of the analysis datasets, the naming conventions for variables, and the metadata. Whether using CDISC or a proprietary structure, the SAP must provide enough detail to guarantee that the resulting datasets are reproducible and auditable by a Notified Body.

Conversely, certain critical operational documents cannot be finalised until the SAP is designed, as they are direct results of the statistical methodology:

• Mock tables, listings, and figures (TLF) shells: The SAP dictates the exact analysis populations, derivations, and statistical methods required. The mock TLF shells, which define the precise layout and data content of the final outputs, must be built directly from these SAP specifications. Attempting to finalise mock shells before the SAP is locked is a fundamental workflow error.

• IRT specifications and randomisation schedule: While the study protocol dictates the high-level design, the exact stratification factors, allocation ratios, and randomisation methodologies are operationally finalised in the SAP. The interactive response technology system is subsequently built to these SAP specifications, and the randomisation schedule is generated as a direct result.

• Statistical sections of the Clinical Study Protocol: These sections will be derived from and consist of a highly abridged version of the final SAP draft based on the final pre- stats version of the protocol.

Designing the document for the persons doing the work

The primary functions of a SAP are to ensure scientific integrity, prevent data-driven inference, and guarantee reproducibility. However, from the perspective of the statistician and programmer executing the analysis, the document must also serve a highly practical function: it must facilitate an efficient, error-minimising workflow.

In theoretical document design, there is a general rule to avoid redundancy – the “Don’t Repeat Yourself” principle. The idea is that a global definition should be stated once, and subsequent sections should simply cross-reference it. When applied to a complex, multi-page SAP, this creates a significant operational problem.

If a statistical programmer is writing code for a specific analysis, but the SAP forces them to hold a complex rule in their head, scroll up forty pages to check the definition of the target population, scroll to another section to check the covariates, and scroll back to find their place, the document has failed them. This constant context-switching causes cognitive fatigue and is a primary driver of errors.

The alternative is to use local context. Rather than relying on distant cross-references, a well-designed SAP makes each analysis section self-contained.

Instead of writing: “Analysis of Primary Safety will be performed as specified in Section 4.2 using the population defined in Section 3.1,” the document explicitly restates the applicable rules within the local section:

“Analysis of Primary Safety: The Safety Analysis Set will be used, defined as all subjects who underwent the index procedure with the investigational device. The primary composite safety endpoint at 30 days will be analysed using the Kaplan‑Meier method; the survival proportion at 30 days and its 95% confidence interval will be estimated using the log‑log transformation to ensure the interval remains within [0, 1]. Subjects lost to follow‑up prior to day 30 will be censored at their last known alive date.”

By intentionally restating the applicable rules within the context of each specific analysis, the document offloads the burden from the programmer’s working memory. The theoretical risk of having to update text in multiple places if a global definition changes is minor compared to the very real risk of errors caused by cognitive overload. 

This self-contained structure is not solely for the benefit of writing code. When the statistician is conducting the analysis, interpreting the outputs, and drafting the clinical or performance evaluation report, they need to instantly recall the exact methodological rules applied to that specific endpoint – such as how missing data were handled or which covariates were included. A document that forces the author to hunt through distant sections to reconstruct the analytical context disrupts the flow of interpretation just as severely as it disrupts coding. A SAP should be optimised for the humans executing the work and writing the narrative, not for the theoretical elegance of the document architect. What appears like a repetative or overly reduncant document is oftentimes set-up that way to acheve workflow efficiency.

The necessity of senior statistical review

Because the SAP serves as the execution manual, its quality control (QC) process cannot be treated as a simple editorial review. A junior statistician may draft a grammatically flawless SAP, but a senior statistician is required to perform methodological QC.

This is particularly important in the EU/UK MedTech space. Sample sizes are often smaller than in pharmaceutical trials, and data structures are inherently more complex, such as clustered data from multiple lesions per subject, or repeated measurements over time from a single sensor. A senior statistician has the experiential foresight to identify methodological traps before any code is written.

When reviewing a device or diagnostic SAP, a senior statistician will review the document as a whole in addition to probing several specific areas (not an exhaustive list):

• Model feasibility: Will a proposed model converge if the device trial has a small sample size with an average cluster size of only 1.2 lesions per subject?

• Diagnostic accuracy: For IVDs under IVDR, does the proposed analysis for sensitivity and specificity adequately account for subjects with indeterminate or missing reference standard results? Are the confidence intervals for the Receiver Operating Characteristic (ROC) curve calculated using the appropriate methodology (e.g., DeLong, Obuchowski) for the study design?

• Handling of missing data: Beyond the high-level estimand framework, are the primary and sensitivity analyses robust against the expected missing data patterns? Does the plan appropriately differentiate between Missing at Random (MAR) and Missing Not at Random (MNAR) assumptions, and are the sensitivity models (e.g., multiple imputation, tipping point analysis) specified correctly?

• Bayesian and adaptive methodologies: If the device trial leverages historical data or adaptive designs, is the selection of prior distributions statistically defensible and clearly pre-specified? Are the adaptive decision boundaries calculated in a way that controls the Type I error?
• Interim analysis integrity: If planned interim looks are specified, are the stopping boundaries and alpha-spending functions correctly defined to control the overall Type I error rate without introducing operational bias?

• Regulatory alignment: Does the handling of intercurrent events, such as device malfunctions or procedure deviations, align with the estimand framework outlined in ICH E9(R1)?

• Notified Body scrutiny: Will the proposed multiplicity strategy withstand the statistical review of a Notified Body operating under MDR Annex XIV?

• Analysis set definitions: Are the precise rules for allocating subjects to the Intention-to-Treat, Per-Protocol, and Safety sets unambiguously defined? For instance, is it explicitly clear how to handle a subject who was randomised but underwent a different procedure, or a subject with no post-baseline data?

• Visit windowing and derivations: Are the derivation rules for calculating baseline, endpoints, and change-from-baseline explicitly clear? Do the defined visit windows (e.g., Day 30 ± 7 days) align logically with the data collection schedule, and are the rules for selecting data points within overlapping windows (e.g., “closest to the target day,” “worst case”) specified?

• Output specifications and denominators: Do the mock table shells perfectly align with the SAP text? Are the exact rules for calculating percentages and N-counts explicitly defined (e.g., “N = number of subjects in the Safety Set; n = number of subjects with the event”) to prevent programming ambiguity?

• Safety event counting rules: How are recurrent adverse events handled: by subject or by event? How are MedDRA coded terms grouped (e.g., System Organ Class vs. Preferred Term) for presentation, and how are pre-existing conditions differentiated from new-onset events?

• Continuous vs. categorical data: If a continuous variable is being dichotomised (e.g., “success” if vessel diameter > 2.0 mm), is the threshold clearly defined, and is the handling of missing or indeterminate continuous values specified before the categorical transformation is applied?

Discovering a statistical flaw during the drafting of a clinical evaluation report is highly disruptive, often requiring a complete re-analysis of the dataset, delayed regulatory submissions, and significant financial cost. Senior methodological review acts as the necessary safeguard to prevent these flaws from entering the execution phase.

The boundaries of non-statistical review

To streamline operations, clinical sponsors occasionally route the SAP for primary QC to a clinical project manager, medical monitor, or clinical affairs lead. This cross-functional review is highly valuable. Non-statisticians play a crucial role in verifying clinical alignment, ensuring that endpoint definitions perfectly match the study protocol and that the clinical narrative is accurately reflected. However, the boundaries of methodological review must be clearly delineated to protect the integrity of the document.

When a clinical reviewer evaluates the mathematical parameters of a SAP, there is a risk of a “false pass.” A complex sentence specifying a mixed-effects model for repeated measures may read grammatically well and appear clinically reasonable. However, without deep statistical specialisation, it is difficult to assess whether that model is over-parameterised for the expected data structure and will ultimately fail to converge.

The deliberate repetition of rules throughout sections- designed to prevent context-switching during programming – may appear to a non-statistician as an editorial oversight. This can lead to reviewers wasting valuable time commenting on and attempting to “correct” this structural redundancy, pulling focus from the clinical alignment they are actually there to verify.

This crossover between clinical and statistical expertise can inadvertently trigger the Dunning-Kruger effect. When operating outside our core domain, human nature often prevents us from seeing the methodological complexities we do not understand. Consequently, well-intentioned clinical reviewers might suggest methodological changes that are clinically logical but technically or mathematically unsound.This is not a reflection of clinical capability, but a natural consequence of crossing disciplinary boundaries.

When a “false pass” or a methodologically compromised plan proceeds to the programming phase, the code will often execute without generating a technical error. The true detriment emerges later: the analysis will reliably produce statistically invalid, biased, or uninterpretable outputs that look like genuine results. The rework required at this late stage- revisiting the SAP, reprogramming the datasets, and rewriting the clinical evaluation report- is exponentially more expensive and time-consuming than catching the methodological error during the review phase.

Evaluating AI in statistical QC

As artificial intelligence (AI) and large language models become integrated into clinical operations, there is a temptation to use AI for SAP QC. While AI can be helpful, it must be approached with caution. AI functions as a syntactic engine, not a mathematical authority. It can perform administrative QC, but it is incapable of nuanced methodological QC. In certain ways the “intelligence” of artificial intelligence is a misnomer or at least greatly overestimated.

On the administrative side, AI is highly valuable. It can cross-reference the SAP against the study protocol to flag missing endpoints, detect internal semantic contradictions, identify missing standard sections, and standardise terminology. For a first-pass cleanup, AI is a useful tool.

However, AI often fails when asked to evaluate methodology. It cannot assess whether a proposed statistical model will converge given the expected sample size and data sparsity of a specific clinical trial. It reads the text and recognises a valid statistical method, but it cannot project the mathematical reality of the data. AI also may lack access to the nuanced, unpublished precedents of EU Notified Bodies or the MHRA. It provides textbook statistical answers, which are frequently misaligned with the specific regulatory strategy required for a successful MDR or IVDR submission. When presented with complex adaptive designs, AI is prone to confidently “correcting” the text based on standard principles, thereby corrupting the methodology.

There is also the matter of cost. In many cases, the cost of a locally run LLM – needed to ensure the protection of intellectual and commercial property – can often outpace the hourly rate of an experienced statistician. This is because human oversight is required to prompt the QC process, QC the QC and iterate when things inevitably fall short due to inherent limitations. Ironically leading to a task that is completed in more time, not less.

Summary

In the highly regulated EU and UK medical device landscape, the SAP is a critical component of clinical evidence. It is an operational manual that must balance regulatory rigour with practical efficiency. A well-constructed SAP protects the statistician’s working memory through local context, supporting both the coding and the clinical writing phases. It relies on the methodological foresight of a senior statistician to navigate the mathematical complexities of device and diagnostic data. It recognises the boundaries of non-statistical review and the limitations of artificial intelligence. Finally, it is built upon a solid, integrated foundation of prerequisite clinical documentation, while driving the operational outputs that follow. Treating the SAP with the operational and strategic respect it deserves is essential for a smooth regulatory submission and successful market access.