The Meaning of Meaningful: What makes a research finding meaningful enough to report?

by Dr. Nileema Conlon Vaswani in

There are few things more depressing than learning that a drug that is intended to help you overcome depression might in fact make you feel worse and perhaps even suicidal.  Glaxo Smithkline (GSK) has come under heavy criticism for not revealing this information about its anti-depressant drug, Seroxat, in a timely manner.  It has been argued that GSK knew that Seroxat heighted the risk of suicide or suicidal thoughts in patients who were under eighteen but did not reveal this finding until 2003.  The reason why this discussion has now been re-opened is because the Medicines and Healthcare Products Regulatory Agency (MHRA) has decided not to file criminal proceedings against GSK because legislation on disclosure at the time of GSK knowing this data was not strong enough to require it.  Now there are proposals to tighten regulations surrounding the disclosure of data from clinical trials.  GSK has argued that they conducted nine trials on patients under eighteen and when the data from each was analysed individually, it was seen not to be clinically meaningful.  Only later, they have stated, when the data of all nine trials was analysed together, did they reveal that there was a heighted risk of suicide in patients under eighteen. GSK has also stated that Seroxat was not licensed or promoted as a drug for those below eighteen. The information on the drug packaging, as explained on their web site, also states that the drug is not recommended for use by those below eighteen.

One might argue that GSK was at fault for not analysing its collective data sooner or one might choose to blame the doctors who, despite the information on the Seroxat packaging, chose to prescribe the drug to patients under eighteen.  It is legal for doctors in the UK to do this at their discretion if they believe that the drug will be beneficial for the patient.  To engage in an argument that is intended to attribute blame to one party or another is not particularly productive.  Several factors made it possible for a drug that was essentially unsafe to be used by patients.  GSK could, perhaps, have reported their findings sooner if they had them, or our laws about reporting data could have been stronger.  Further, the idea of GP discretion could have been examined more closely. Pharmaceutical companies might be partly responsible in situations such as these but if GPs, despite warnings, are prescribing these drugs, the burden of responsibility, at a very minimum, is shared.  But regardless of GSK and Seroxat, what is both important and interesting is the question of what makes data meaningful?

In research terms, data is significant or meaningful if there is a pattern to it.  But incidents that are rare are still meaningful to those who experience them.  Say, for example, one patient experienced a serious side effect in a clinical trial but the other 999 did not experience this side effect.  For this patient, the side effect is meaningful and very real regardless of whether it has meaning in a research context and quite apart from the fact that the other trial participants did not experience it.  Similarly, data that is clinically insignificant for the purposes of reporting could be significant to patients who are interested in the data for their own medical conditions outside of a clinical trial.  This could also hold true for those who are participating in subsequent clinical trials. They may wish to be informed of data from preceding trials rather than just that which the researchers consider statistically significant.  

There are, of course, good reasons not to report data that is statistically insignificant.  One reason is that such data, given that it will not affect the majority of the population, is likely to be misleading.  If the majority of people to whom this data is unlikely to apply consider the data when deciding whether or not to take the drug, then they may well lose out from the benefits of taking the drug in question.  Another reason to arguably withhold such data is that publishing it might be alarming.  Publishing known, yet statistically insignificant, side effects of a drugs might cause unnecessary panic and, again, people who might safely stand to benefit from the drug might lose out by not taking it because they are worried about the effects that the drug might have on them.  If the general population is inadvertently discouraged from taking the drug or vaccine because the data that has been published is misleading or alarming then the purpose of the trials has been defeated.  

The very purpose of clinical trials is to determine whether or not a drug or vaccine is safe and effective for the general population.  Regardless of the findings of the clinical trial, the general population stands to benefit.  If it emerges that a certain drug or vaccine is safe and effective, then people can benefit from taking it; if not, then the general population benefits by knowing this information.  But what lies at the heart of disclosing information is the idea of accuracy.  Just as data that reveals that a drug is unsafe ought not to be withheld, data that indicates safety and effectiveness ought also not to be withheld.

If there are both pros and cons for disclosure of data, what ought to be done?  How should the pharmaceutical industry and other regulatory bodies decide whether certain data ought to be disclosed? And what makes data meaningful?

The most appropriate solution to this problem is to disclose all the findings of clinical trials, even those that are believed not to be clinically meaningful.  Once these findings are in the public domain, potential consumers can decide which data assumes meaning for them.  Once people have all the information before them, they can determine what is relevant to their medical conditions.  Fears about data being misleading or inaccurately worrying will be alleviated by the fact that all the data is available.  It is only by having access to all this information, can patients genuinely weigh up the pros and cons and make an informed decision as to whether or not to take the drug.