ISoP ISoP on Twitter | ACoP ACoP on Twitter | PAGE | WCoP | PAGANZ | PAGJa

Summary Statistics for Blinded Clinical Trial Results

Statistics and Pharmacometrics (SxP)
#1

I am often the unblinded individual in an otherwise blinded clinical trial. And, as part of my unblinded role, I am often asked to provide summary statistics for results when the summary statistics may unblind results either directly or indirectly. As a simple example of direct unblinding, when not all subjects have finished the trial, providing the number of subjects contributing to a summary statistic can unblind. So, I have a question: How do you provide summary statistics without unblinding anyone?

My current favored method is to provide the median as the main measure of central tendency with the following rules:

  1. If all subjects have completed the study or cohort with that treatment, provide the actual median.
  2. If not all subjects have completed the study or cohort with that treatment (e.g. if n_missing have not completed), take the median of the data with n_missing removed from the top, take the median of the data with n_missing removed from the bottom to establish the range of possible median values, and then provide as the summary statistic a uniformly distributed random number between those bounds.

I know that other methods include removing the last X values where X is the maximum number of subjects who could receive the current treatment if everyone were randomized to it, and then calculate the summary statistic based on that subset of the data.

Do you have another preferred method? Do you have tools that help you generate these statistics?

#2

Interesting post. I am not aware of best practices and or software that help in preventing premature unblinding of studies ( or PK/PD ) studies I have seen a 2018 paper from Janssen.

and this one:

but not sure where to find SOP / rules on how DSMB handle this on a routine basis

#3

Hi Bill,
Generally, providing summary statistics by treatment group, even if the treatment group labels aren’t disclosed, is considered unblinding and should be avoided in blinded trials. Sometimes it makes sense to monitor total number of events or other statistics pooled across the entire trial population. This is OK since it does not require access to treatment codes. Usually, unblinded or grouped data are only shared with data monitoring committees. The types of data and frequency of sharing are outlined in a charter or the protocol. You could also monitor concentrations as long as no other information regarding efficacy or safety was shared and you had a documented process for doing that while protecting trial integrity. For early access to data needed for PK/PD modeling, you could also develop a process that is acceptable to regulators. At a minimum, please check with the study statistician before providing grouped results.
Best Regards,
Matt

#4

@SamerMOUKSASSI, thanks to the pointers to those documents, they are very helpful!

@MattR, I’m curious why you think that providing summary statistics by treatment group is generally unblinding (when done with care)? Given the fact that you mention data monitoring committees, I wonder if your comment is more focused on late-phase or longer studies where DMCs are typically used. (Though, I recognize that DMCs are not solely used in those scenarios.) I can think of a myriad ways that it is not unblinding when done carefully, though I do agree there are ways that it can be unblinding either when not done carefully. Generally, if the blinded individuals have access to the raw data without treatment assignment, I agree that summary statistics cannot be provided as they could be deconvoluted by testing combinations and permutations of treatment assignments with the data in hand. And it would also be unblinding to provide primary or secondary endpoint results or anything tightly correlated to the primary or secondary endpoint results when they are relevant to trials where the endpoint has downstream importance such as a pivotal trial or corporate material information (though not unblinding to treatment individual assignment). But there are also many scenarios where providing summary statistics by treatment arm are required such as PK summaries in dose escalating studies with PK limits and adaptive dose-finding studies where the PK/PD by treatment group is required for decision making about the next dose level.

While I agree that it is important to be careful of the blind (ergo this post), it is also very often necessary to release treatment-level summaries.

#5

I assumed you were monitoring the PK and that you were being asked for summaries of other info. To the point of your message, it’s hard to maintain the blind even with simple PK summaries, especially in small studies.