MAMS vs. Response-Adaptive Randomisation: A Practical Guide for Medical Device Clinical Investigations

Medical device clinical investigations, as formally regulated under FDA Investigational Device Exemption (IDE) rules or EU MDR/ISO 14155 guidelines, increasingly incorporate adaptive methodologies to improve efficiency and patient outcomes. Two primary approaches have emerged: Multi-Arm Multi-Stage (MAMS) designs, which evaluate arms at predefined milestones to drop inferior treatments or stop early for efficacy, typically maintaining a fixed allocation ratio among remaining arms; and Response-Adaptive Randomisation (RAR), which continuously shifts the randomisation probabilities of future patients toward better-performing treatments throughout the investigation.

The choice between these methods depends on investigation characteristics including endpoint timing, data infrastructure, regulatory requirements, and scientific objectives. This guide examines the mathematical foundations and operational considerations for each approach to help biostatisticians and sponsors select the most appropriate method.

Fundamental Differences Between MAMS and RAR The core distinction lies in the mechanism of adaptation. MAMS designs adapt by eliminating underperforming arms or stopping the investigation entirely at discrete interim analyses. Between these analyses, randomisation remains fixed (e.g., 1:1 among active arms).

Conversely, RAR designs adapt the allocation ratio itself. Rather than dropping arms, RAR continuously updates the probability of assignment, skewing enrolment toward the treatment arms demonstrating the best outcomes. This fundamental difference has profound implications for statistical methodology, operational complexity, and regulatory considerations.

Multi-Arm Multi-Stage (MAMS): Mathematical Framework and Implementation MAMS designs evaluate the big picture through formal interim analyses. The mathematics centres on group sequential testing, where investigators evaluate treatment effects and apply pre-specified stopping rules for efficacy or futility.

Let H_{0,i} denote the null hypothesis for treatment i at stage k, where k = 1, 2, \dots, K represents the interim analysis stages. The decision rule to drop an arm for futility or stop for efficacy relies on test statistics Z_{i,k} crossing predefined boundaries:

\text{If } Z_{i,k} \geq U_k, \text{ stop for efficacy and reject } H_{0,i}

\text{If } L_k < Z_{i,k} < U_k, \text{ continue enrolling } 1:1 \text{ in remaining arms}

MAMS implementations must strictly control the overall Type I error across multiple arms and stages. The Lan-DeMets approach allocates Type I error across K analyses using \alpha(t_k) = \alpha \times f(t_k), where t_k = I_k/I_{max} is the information fraction. Common spending functions include the O’Brien-Fleming boundary:

f(t) = 2 - 2\Phi\left(\frac{Z_{1-\alpha/2}}{\sqrt{t}}\right)

which applies stringent boundaries early in the investigation when data is sparse. If an arm is dropped, the investigation continues with fixed allocation among the remaining arms.

Worked Example: Consider a cardiac stent investigation testing three new drug-eluting stents (A, B, C) against standard care (D). After enrolling 200 patients (1:1:1:1 fixed allocation), the first interim analysis is conducted. Stent A shows a 15% reduction in target vessel revascularisation, while Stents B and C perform similarly to the control.

Using the O’Brien-Fleming spending function, the efficacy boundary U_1 might require a Z-score of 2.8, and the futility boundary L_1 might be set at 0.5.

• If Stent A crosses U_1, the investigation stops early for overwhelming efficacy.
• If Stents B and C fall below L_1, they are dropped.
• The next stage continues with fixed 1:1 allocation between Stent A and Standard Care.

This approach requires sophisticated upfront planning. Sponsors must specify exactly when interim analyses occur, the exact boundaries for dropping arms, and how Type I error is controlled. Regulatory agencies appreciate this level of pre-specification because it ensures investigation integrity and prevents ad-hoc decision-making.

Response-Adaptive Randomisation (RAR): Bayesian and Frequentist Approaches Response-adaptive randomisation operates at a much more granular level, updating beliefs about treatment effectiveness after each patient outcome. Rather than dropping arms, RAR shifts allocation probabilities. The mathematical foundation typically involves Bayesian updating, where the investigation maintains probability distributions representing current beliefs about each treatment’s efficacy.

Thompson sampling maintains posterior distributions for each treatment’s efficacy parameter. For binary outcomes, if treatment i has observed s_i successes in n_i trials, the posterior under a \text{Beta}(\alpha, \beta) prior becomes:

\theta_i | \text{data} \sim \text{Beta}(\alpha + s_i, \beta + n_i - s_i)

At each allocation, a value \tilde{\theta}_i is sampled from each posterior, and the next patient is assigned to the treatment with the highest sampled value. The allocation probability for treatment i converges to \pi_i = P(\theta_i = \max_j \theta_j | \text{data}).

Alternatively, allocation probabilities can be driven directly by the posterior mean. A frequentist alternative, the Randomised Play-the-Winner (RPW) rule, uses urn models. If treatments A and B have current success rates S_A and S_B, a simplified allocation rule might assign the next patient to treatment A with probability P(A) = S_A / (S_A + S_B). For example, when treatment A succeeds in 8 out of 10 patients while treatment B succeeds in 6 out of 10, the next patient has an 8/(8+6) \approx 57% chance of receiving treatment A.

Additional RAR rules include Covariate-Adjusted Response-Adaptive (CARA) randomisation, which models success probability as:

\text{logit}(P(Y=1|X,Z)) = X^T\beta + Z^T\gamma

where Z indicates treatment assignment. For continuous outcomes following N(\mu_i, \sigma^2), Thompson sampling updates \mu_i | \text{data} \sim N(\hat{\mu}_i, \sigma^2/n_i), where \hat{\mu}_i is the sample mean for treatment i.

Worked Example: AI Diagnostic System Investigation Consider a clinical investigation testing three approaches for detecting diabetic retinopathy: AI-only, traditional ophthalmologist review, and combined AI plus ophthalmologist verification.

Starting with uninformative priors \text{Beta}(1,1) for each approach, after 50 patients the data shows: AI-only correctly diagnosed 42 cases with 8 errors; traditional review got 38 right with 12 wrong; and the combined approach achieved 47 correct with only 3 errors. The Beta distributions become \text{Beta}(43,9), \text{Beta}(39,13), and \text{Beta}(48,4), respectively.

For the next patient allocation, the system samples from each distribution thousands of times and counts how often each approach produces the highest sample. The combined approach, with its dominant \text{Beta}(48,4) distribution, might win 80% of these samples, earning it an 80% allocation probability for the next patient. No arm is dropped, but the AI-only and traditional review arms receive significantly fewer future patients.

This approach naturally handles the delayed response problem that Di and Ivanova (2020) addressed in their Biometrics paper. When diagnostic results take weeks to confirm, beliefs cannot be immediately updated for patients enrolled yesterday. Their methodology maintains separate “pending pools” for each treatment, applying the adaptation rule only to data from patients whose outcomes are confirmed available by time t-d:

\pi_i(t) = \frac{\alpha_i + s_i(t-d)}{\alpha_i + \beta_i + n_i(t-d)}

where s_i(t-d) and n_i(t-d) represent successes and total allocations from patients enrolled by time t-d.

Device-Specific Statistical Considerations Medical devices introduce mathematical challenges that pharmaceuticals rarely face. Learning curves create a particularly complex problem because early poor outcomes might reflect operator inexperience rather than device inferiority.

As described by Berry and colleagues (2010), this can be modelled explicitly. If \theta_{ij} represents the success probability for surgeon i on case j, a hierarchical model might use:

\text{logit}(\theta_{ij}) = \alpha_i + \beta_i \times \log(j) + \gamma \times \text{treatment_effect}

where \alpha_i captures surgeon i’s baseline ability, \beta_i represents their learning rate, and \gamma is the true device effect. The logarithmic term captures the typical learning curve shape where improvement is rapid initially, then plateaus. This framework is highly compatible with RAR, as it prevents promising devices from being unfairly penalised during the skill acquisition phase.

Software updates during investigations present another mathematical puzzle. Instead of treating device versions as completely separate entities, hierarchical Bayesian methods model them as related. If version 1.0 has parameter vector \theta_1 and version 1.1 has \theta_2, the model can specify:

\theta_2 \sim \text{Normal}(\theta_1 + \delta, \Sigma)

where \delta represents expected improvement and \Sigma captures uncertainty about how modifications affect performance. This approach “borrows strength” from pre-update data while learning about post-update performance.

Statistical Challenges and Solutions Both MAMS and RAR amplify “garbage-in, garbage-out” problems because incorrect early decisions cascade through the entire investigation. To mitigate this, observations can be weighted by their reliability. Following data maturity frameworks, analysts can apply weights:

w_i = \min(1, \text{days_since_observation}_i / \text{required_follow_up_days})

using w_i \times \text{outcome}_i in adaptation calculations instead of raw outcomes.

Multiple comparisons present another challenge. More interim analyses mean more opportunities for false positives, but traditional Bonferroni corrections are too conservative for adaptive investigations. The Lan-DeMets \alpha-spending approach provides a more nuanced solution for MAMS designs, while RAR designs often rely on simulation-based calibration of the randomisation algorithm to ensure Type I error control.

Simulation Requirements and Operating Characteristics Before implementing any adaptive design, extensive simulation studies are required, typically 10,000 or more simulated investigations under different scenarios. Simulations must test the null hypothesis (where all treatments are equivalent), realistic alternative hypotheses with plausible effect sizes, and mixed scenarios where some treatments work while others do not. For each simulated investigation, patient outcomes are generated from appropriate probability distributions, adaptation rules are applied, and final sample sizes, allocation ratios, and conclusions are recorded.

Regulators require evidence that the Type I error remains strictly controlled (below 0.05) under the null hypothesis, that the investigation achieves adequate power (typically \geq 80%) to detect clinically meaningful differences, and that the design yields efficiency gains when clear winners exist (Pallmann et al., 2018).

Regulatory Considerations

Regulators globally have become increasingly supportive of adaptive designs, provided they are grounded in mathematical rigour. Regulatory frameworks and documentation requirements differ depending on the region.

In the United States, the FDA expects adaptation algorithms, whether MAMS stopping boundaries or RAR probability updates, to be detailed in the Statistical Analysis Plan (SAP). Adaptive clinical trials rely on pre-specified algorithms, thus the FDA expects the SAP to be finalised before the first interim analysis to ensure that adaptive features are truly prospective and algorithmic rather than subjective.

In the European Union, clinical investigations are governed by the EU Medical Device Regulation (MDR 2017/745) and ISO 14155:2021. Under the MDR, the study protocol is officially designated as the Clinical Investigation Plan (CIP). The CIP must outline the high-level adaptive methodology and justify how the design protects human subjects and minimises risk (Article 62). The exact statistical algorithms, simulation parameters, and

α-spending functions are then specified in a separate, detailed SAP. For EU Competent Authorities, the SAP must be finalised and locked before the initiation of any adaptive algorithms to ensure strict data traceability and prevent operational bias.

In the United Kingdom, the MHRA has been uniquely proactive in embracing innovative trial designs. The MHRA’s joint guidance with the HRA and NIHR on Adaptive Designs in Clinical Trials explicitly encourages the use of MAMS and RAR approaches. The MHRA requires robust evidence of Type I error control and often requests that sponsors provide the simulation code used to derive the operating characteristics so reviewers can independently validate the design outlined in the CIP and SAP.

Across all regions, pre-submission interactions (e.g., FDA Pre-Subs, MHRA Innovation Meetings, or Expert Consultations with EU Competent Authorities) should include the proposed CIP and draft SAP, simulation results demonstrating operating characteristics, clear rationale for methodological choices, and detailed plans for Independent Data Monitoring Committee (IDMC) involvement.

Implementation Requirements Implementing adaptive randomisation requires careful consideration of operational constraints. MAMS designs require \alpha-spending function calculations at discrete intervals. RAR designs demand much more: real-time Bayesian updating capabilities, Monte Carlo sampling for Thompson sampling (typically requiring 1,000+ samples per allocation), and automated allocation probability updates.

The system architecture for RAR flows from data entry through real-time databases to statistical engines that feed the randomisation system. RAR investigations inherently require greater statistical analysis plan complexity, increased programming effort for the creation and validation of simulation code, and heightened data management complexity to ensure real-time data cleaning and availability.

Decision Framework for Method Selection

Choose Multi-Arm Multi-Stage (MAMS) Designs when:

• There are well-defined interim analysis timepoints (e.g., safety run-ins, planned efficacy looks)
• Primary endpoints require substantial follow-up time
• The investigation features multiple treatment arms where dropping inferior arms is preferred over simply skewing allocation
• There is a regulatory preference for pre-specified, discrete adaptation rules and fixed randomisation between looks
• Real-time data processing capabilities are limited

Choose Response-Adaptive Randomisation (RAR) when:

• Rapid endpoint assessment is possible (minutes to days)
• There is a strong ethical imperative to minimise exposure to inferior treatments without dropping them entirely
• The patient population is homogeneous with consistent response patterns
• Robust, real-time data systems are available
• The primary focus is on optimising patient allocation efficiency rather than definitive early stopping

Consider hybrid approaches when:

• Different endpoints have different assessment timelines
• Both early safety signals (MAMS) and longer-term allocation efficiency (RAR) matter
• Regulatory discussions suggest openness to novel designs

The key is matching the adaptive mechanism to the investigation’s operational realities and scientific objectives. RAR’s patient-level adaptation offers maximum efficiency but demands flawless data systems. MAMS provides more operational control and traditional regulatory familiarity but may miss opportunities for real-time optimisation. Both approaches require extensive simulation studies to demonstrate operating characteristics under realistic scenarios. The choice between them should be driven by which method best serves the specific combination of scientific questions, operational constraints, and regulatory pathway.

References

1. Berry, S. M., Carlin, B. P., Lee, J. J., & Muller, P. (2010). Bayesian Adaptive Methods for Clinical Trials. Chapman and Hall/CRC.
2. Di, J., & Ivanova, A. (2020). Response-Adaptive Randomization for Clinical Trials with Delayed Responses. Biometrics, 76(3), 895-903.
3. Hu, F., & Rosenberger, W. F. (2006). The Mathematics of Adaptive Randomization in Clinical Trials. Springer.
4. International Council for Harmonisation (ICH). (2019). E9(R1) Statistical Principles for Clinical Trials: Estimands and Sensitivity Analysis.
5. International Organization for Standardization (ISO). (2021). ISO 14155:2021 – Clinical investigation of medical devices for human subjects — Good clinical practice.
6. Pallmann, P., Bedding, A. W., Choodari-Oskooei, B., et al. (2018). Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Medicine, 16, 63.
7. U.S. Food and Drug Administration (FDA). (2016). Adaptive Designs for Medical Device Clinical Investigations: Guidance for Industry and Food and Drug Administration Staff.
8. Wason, J. M. S., & Trippa, L. (2014). A comparison of Bayesian adaptive randomization and group sequential methods for binary outcomes. Statistics in Medicine, 33(17), 2949-2962.