I’m back at last. I have spent the last few days helping to get my mother buried, in our hearts if not in fact, and only now have time to finish up.
In this post I’ll address the arguments made by David Felson as to the efficacy of glucosamine and chondroitin. Dr. Felson focused mostly on glucosamine for which there is more evidence. His argument was in three parts: he taught us about effect sizes as a way to compare studies; he talked about the GAIT study and issues of subgroup analysis; and he reviewed a meta-analysis done by him and his fellow, Steven Vlad.
Part 4a: Effect Sizes
First, he introduced the concept of the effect size. This is a way of standardizing the results of trials so that they are more directly comparable. It is defined as the difference between outcomes between to trial arms, divided by the standard deviation. It therefore gives the difference in outcome standardized to a normal distribution: a change of 1 represents an improvement of 1 standard deviation for treatment over placebo. This is a large effect of any therapy. An effect size of 0.2 is a small effect, 0.5 a modest effect, and 0.8 a large effect. For comparison, treatment of knee OA with NSAIDs gives an effect size on the order of 0.2-0.3. A total knee replacement – definitive therapy for knee OA – gives an effect size of 1 or greater. Glucosamine has been claimed to have an effect size on the order of 0.3-0.8 depending on the study you read; at least as good and maybe better than NSAIDs. Some studies have shown effect sizes larger than 1 – on the order of magnitude of a knee replacement. Dr. Felson finds this to be simply impossible to believe and I agree.
Part 4b: The GAIT Trial
The GAIT trial, as noted previously, has been used to support the idea that G/CS is effective in a subgroup of patients – those with moderate to severe knee pain – even though the overall results of the study are negative. The reason for this is that when the investigators looked at this subset of patients there was a ‘statistically significant’ result between G/CS and placebo even though the rest of the trial was negative.
Dr. Felson made a very important point: the findings of subgroup analyses (which is what this is) must be treated with great suspicion. This goes back to the ‘multiple comparisons’ problem which I mentioned previously. The more comparisons you make, the more likely it is that one of them will be ‘significant’ or ‘positive’ by chance.
In this case, subjects were ‘stratified’ by the severity of pain when they were randomized. That is, the investigators made sure that subjects with moderate to severe pain were equally represented in each study arm (placebo, celecoxib, glucosamine, chondroitin and the G/CS combination). This is a valid way to make sure that pain severity does not ‘confound’ the results.
The problem is that the investigators then looked at this group separately at the end of the study – this makes this analysis subject to the ‘multiple comparisons’ problem. Felson sited a number of papers that demonstrated that post-hoc analyses must be viewed with caution, and cited a number of methods by which this could be minimized. The GAIT trial subgroup analyses met none of the criteria he cited.
Therefore we should take this result with a large (huge! giagantic!) grain of salt. The main trial results are much more likely to represent the true results even in this subset. Conclusion: the evidence that G/CS is effective in this subgroup is poor.
He also pointed out that of the 66(!!) primary outcome analyses, not one was positive. It was only when the investigators looked at secondary outcomes and subgroup analyses that positive results were found. And this is true regardless of whether one uses the Bonferoni correction for subgroup analyses or not.
Part 4c: Meta-analysis
The final piece that Dr. Felson presented was a meta-analysis and meta-regression undertaken by his fellow, Steven Vlad.
As background, meta-analysis is a way of combing the results of many trials and finding a ‘summary’ that represents the true effect of a therapy. Be aware that meta-analyses are fraught with difficulties. The major one is that the trials that are meta-analysed must be sufficiently similar for the summary result to actually be a good indication of the effect of the therapy. If the trials are too dissimilar the summary result is likely to be biased, and biased in an unpredictable way.
There is a way to measure this dissimilarity. In statistics it is referred to as ‘heterogeneity’ and one of the ways to measure it is a value called the I-square. This represents the between-trial difference that cannot be attributed to chance. That is, there will always be some variation between trials, even if they are done in exactly the same way, that can be attributable to chance. Chance is to be expected. I-square measures the variability beyond chance. Thus, if a meta-analysis has an I-square value to 0.80, then 80% of the variation in the separate results of the trials is due to more than just chance. There are serious differences between trials.
I can’t go into the results of this study in too much detail (I wish I could) because it’s being prepared for submission to a journal (which means that the journal will have the copyright and the right to report the results), but the gist is that there is a lot of heterogeneity in glucosamine studies which makes the summary effect sizes reported for previous meta-analyses very suspect.
In addition, it looks like the reason for all the heterogeneity among glucosamine trials could be the result of bias by the studies’ supporters; i.e. the people who funded the studies. Independent studies tend to be negative, while those supported by drug/product manufacturers tend to be positive.
This effect has been shown in studies of other therapies unrelated to OA too. It is well known that trials supported by pharmaceutical companies are more likely to report positive results than those supported by an independent source.
The suggestion then is that the apparent effect of glucosamine may be due to the fact that the majority of the positive trials are industry-supported, while those that are negative are not. Which one would you trust more? I know where I’d put my money!
So that’s my summary of this particular ACR 2006 session. Personally, I think the hands-down winner was the anti-G/CS side. I found Dr. Theosodakis’s arguments unconvincing, and Dr. Felson’s to be quite well-reasoned.
I’ve already had a commenter on a previous post tell me that he uses G/CS and finds them to be very helpful. My suspicion is that this is a placebo effect, but I would never tell this person that he should stop using G/CS. As long as he has the money to spend and feels that he gets benefit from the products, he should continue to use them. The one thing everyone agrees on is that this combination of products is safe. So it’s hard to find a downside, except for the money wasted.
However, I think in light of the evidence that it is irresponsible for me as an individual practitioner, and irresponsible for any professional organization to recommend using these therapies. I cannot in good conscious recommend a placebo to my patients.