Scenario

A med-tech company had invested in the research and development of a new polymer material for coronary artery stents. While the newly proposed material for use in stents was promising, the amount of quality data on its use in biological implants was limited. Extensive safety studies during R&D and clinical trials therefore had to be carried out.

At the R&D phase, biocompatibility testing data was collected in animals. The device was then tested to ensure minimal endovascular trauma, mechanical stability, and that the material was MRI-safe as well as suitable for fluoroscopic guidance to enable safe implantation by catheter. Advice was sought on the analysis of the R&D data as well as the design of an initial clinical study to compare their new stents with existing bare-metal and drug-eluting stents.

Solution

The biocompatibility data from R&D studies was analysed, and factors including biotoxicity, haemocompatibility and the potential for leachables were evaluated against their respective benchmarks. The company had planned to follow these R&D stages with clinical studies. A pivotal study was designed using Bayesian adaptive methods. A statistical analysis plan (SAP), sample-size report and randomisation schedule were produced for the study and used to inform the study protocol.

A parallel-groups equivalence design was chosen, based on a repeated-measures ANOVA design with some adjustments for multiplicity. Two insertion methods were included for each stent type in the study (angiography-guided versus without). Endpoints included the success rate of the initial procedure (resulting in improved blood flow and a widened artery lumen), restenosis rate, and mean time to restenosis. The Kaplan–Meier method was used to model time to adverse events, and a Cox proportional-hazards frailty model adjusting for relevant covariates was used for time to restenosis.

The sample size for the clinical trial was planned using Bayesian meta-analysis to derive the parameters necessary for the sample-size calculation, drawing on patient clinical data and the literature on stent safety and efficacy for comparable devices. A hierarchical random-effects model using MCMC methods accommodated the heterogeneity of the included studies, and informative prior distributions were used to simulate the sample size based on the study design.

An interim analysis at week 12 was planned to assess differences between the two insertion methods in terms of procedural success rate and restenosis rate. This data would be used for an updated prior for the final analysis, with a Bayesian sample-size readjustment performed if one insertion method showed a significantly lower initial success rate or a higher rate of restenosis. Following the pivotal trial, post-market surveillance studies were planned to monitor device effectiveness and long-term biocompatibility against bare-metal and drug-eluting stents. Adverse events for this were five-year all-cause mortality, cardiovascular death, spontaneous MI, procedural MI, stroke and repeat revascularisation.

The benefits of the Bayesian approach were the incorporation of external data; the use of data from intermediate outcomes in an interim analysis, with the ability to adjust the design if necessary; better management of missing endpoint data, important given the smaller sample size; and a cleaner way to manage and adjust for multiplicity.

Scenario

A small medtech company aimed to conduct a pilot comparison study evaluating a regenerative-medicine injectable adjunct therapy against the standard treatment for Achilles tendon injury. The primary objectives of the pilot study were to assess study feasibility and gather preliminary data to inform the design of a full-scale clinical trial.

Initially, the researchers proposed a non-inferiority study design with a non-inferiority margin of 15 points on the Achilles Tendon Total Rupture Score (ATRS). The design included five endpoints with repeated-measures data collected over four time points post-treatment. Based on clinical advice, the researchers decided on a sample size of five patients per treatment arm. The company requested an audit of the study design before finalising the statistical analysis plan and contributing to the statistical sections of the study protocol.

Audit

Upon auditing the study design in light of the study goals, several issues were identified:

• Sample size. Five patients per arm was insufficient. To estimate the standard deviation of the ATRS score with reasonable precision, 12–25 patients per arm would provide a more reliable estimate, crucial for calculating the sample size of the full-scale trial. Assessing feasibility (participation, compliance and dropout rates) also requires a larger sample to capture variability and potential issues.
• Non-inferiority margin. A margin of 15 points on the ATRS score was considered too wide and may not be clinically meaningful; it could deem the adjunct treatment non-inferior even if it is clinically worse. A more stringent margin better reflects the minimally important difference in ATRS scores and keeps the pilot data relevant to the full-scale trial. And while a pilot study is not powered to detect definitive treatment effects, it can still provide preliminary insight into the efficacy of the novel treatment, guiding whether to proceed with a full-scale trial and how to design it.
• Study design. The client initially preferred a non-inferiority design for its lower sample-size and power requirements. However, because the novel treatment is administered in addition to the standard treatment (an adjunct therapy), the goal is to demonstrate enhanced efficacy over the baseline treatment, which aligns with a superiority design rather than non-inferiority. A superiority design better assesses the potential benefit of the adjunct therapy and aligns with the needs of the full-scale trial.
• Multiple endpoints and time points. The design included multiple endpoints and time points, which could complicate the analysis and interpretation of results and needed to be carefully managed.

Solution & deliverables

To address these issues, the goals of the pilot study were redefined to estimate the standard deviation of the primary outcome (ATRS score) so the full-scale trial could be accurately powered; to estimate feasibility metrics (participation, compliance and dropout rates); and to collect preliminary data from initial patients that could be incorporated into the full-scale trial, maximising the use of resources. A sample size of 12–25 patients per arm was recommended, and a superiority design was adopted with a superiority margin corresponding to the minimally important difference in ATRS score.

Based on the revised design, the deliverables were a comprehensive audit report detailing the findings and recommendations specific to the study; a statistical analysis plan (SAP) outlining the statistical methods, plans for data analysis, handling of repeated measures and interim analyses; and statistical sections of the study protocol describing the design, objectives and analysis methods.

Scenario

A pharma start-up had invested heavily in the development of a compound using bio-simulation techniques. The related biomarkers were known to be present in several cancer types, based on pre-existing research. The identification of drug targets based on specific genomic biomarkers related to disease progression led to an immuno-therapeutic compound. Initial animal studies were promising and suggested the new antibody-based drug would be more effective than the existing in-class alternative for colorectal (CRC) and non-small cell lung (NSCLC) cancer.

The company had worked with a full-service CRO to conduct phase I and II studies in humans. The phase II results struggled to define an optimal effective dose, and a pilot phase III efficacy study using a crossover design had not been able to establish efficacy or equivalence in a small patient sample. The company was looking at abandoning the compound but came to us for further advice.

Solution

Upon reviewing the patient data, it was noted that roughly half of the patients in the clinical trial responded positively to the treatment while others did not show a sufficient response. The patients who did not respond also seemed to have worse side effects, which affected the overall efficacy of the therapeutic as determined by the study, as well as its side-effect profile.

Our biostatistics team analysed the clinical data of patients who showed a clinically meaningful response against those who did not, to see if there were any differences between the two groups. After comparing demographics, baseline disease and other characteristics, there did not appear to be a compelling difference. It was noted, however, that responders were slightly more likely to have been in the group that received the novel compound at stage 2 of the study (group 2), rather than stage 1.

Genomic data had also been collected from the patients, which we subjected to bioinformatic analysis. A few biomarkers of interest were identified, including a mutation in the KRAS pathway, a signalling pathway with a role in several key cell processes, including proliferation. One biomarker in particular was present only in patients with a limited response to the therapeutic.

The company decided to conduct a follow-up biomarker-guided clinical trial comparing the efficacy of the novel compound against an existing antibody-based treatment, with subjects restricted to those established in the genomic analysis to be likely responders. A parallel design was used, for two reasons. First, by restricting the sample to likely responders, there was no longer the same level of treatment risk that had necessitated the crossover design in the previous study. Second, the clustering of most responders in group 2, and the fact that some non-responders in group 1 lacked the biomarker associated with non-response, hinted at the possibility of paradoxical progression in some patients taking the novel treatment. If so, a crossover design would not be the optimal way to assess efficacy moving forward.