The Lancet paper by Shang et al.

The Lancet paper by Shang et al.
In August 2005, the Lancet published a study1 which was reported as having compared 110 similar trials on homeopathy and conventional medicine, and reached the conclusion that homeopathy is no better than placebo.
An accompanying editorial by The Lancet editor entitled ‘The end of homoeopathy’2 lead to widespread media attention.
However, the conclusions of this study were in fact based on only 8 of the 110 trials, none of which involved usual homeopathic care. Furthermore, if you switch just one of the 8 trials they picked from the 110 for a different one, the results are reversed, showing that homeopathy works beyond placebo.3
This demonstrates that the findings of this paper are completely unreliable.
The sequence of events surrounding this issue of The Lancet, the political controversy which followed and key flaws of the study itself, are summarised in ‘Homeopathy and the Lancet‘.3
A paper with continued global impact
The Shang et al. paper has become notorious because, despite such fundamental flaws, it continues to be mis-quoted to this day as definitive ‘proof’ that homeopathy doesn’t work e.g.:
“Over 150 trials have failed to show it works”
Dr Ellie Cannon – resident GP for the Mail on Sunday. ‘Health Comment’, Mail on Sunday, 14 September 2014

“We’ve had over 100 trials of homeopathy now. For any medical treatment, after 100 trials have failed to give good overall evidence of benefit, this is the point at which a sensible person, with no vested interests in that one particular treatment, would loudly and clearly state: “No more money should be spent researching this blind alley”.
Ben Goldacre – British physician, academic and science writer.‘Pharma Chameleon’, New Europe, 17 April 2011 – Issue 93.
Results of the Shang paper
Shang et al.’s findings as reported in the abstract of the paper are:
110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study size was 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional-medicine trials were of higher quality. In both groups, smaller trials and those of lower quality showed more beneficial treatment effects than larger and higher-quality trials. When the analysis was restricted to large trials of higher quality, the odds ratio was 0·88 (95% CI 0·65–1·19) for homoeopathy (eight trials) and 0·58 (0·39–0·85) for conventional medicine (six trials).
Note: These ‘odds ratios’ mean that, for these particular 8 trials of homeopathy, if you analyse them together, the results suggest that the effects seen are due to placebo rather than a real clinical effect.
How reliable are these results?
The role of the Shang paper in the homeopathy debate cannot be overstated, so the issue of quality and reliability of the paper is paramount.
Unfortunately, the poor scientific quality of the paper is obvious, having been described by experts in the field of homeopathy research3,4,5,6,7 as well as independent researcher Prof Hahn, who highlights the main problem:
To conclude that homeopathy lacks clinical effect, more than 90% of the available clinical trials had to be disregarded.8
 
More
Relevance of Shang’s 2005 paper in 2015
Reliability of the analysis is not the only problem with the Shang paper. As we take a fresh look at the evidence in 2015, we also need to consider how well this study reflects the entirety of today’s evidence base.
A rigorous study by Mathie et al. published in 2014, found that homeopathic medicines, when prescribed during individualised treatment, are 1.5- to 2.0-times more likely to have a beneficial effect than placebo.13
This study includes 151 placebo-controlled randomised trials – 41 more than Shang’s team identified in 2005, but which would have met their inclusion criteria if they had been available at the time.
This demonstrates the extent to which the 10 year-old Shang et al. paper, which now covers only 73% of the eligible trials, has been superseded.
Read HRI’s brief summary of Mathie et al.’s study or listen to Robert Mathie presenting his findings at the HRI Rome 2015 conference.
Shang et al. key facts

• Although 110 homoeopathy trials and 110 matched conventional medicine trials were included in the study, the results were based on only 8 homeopathy trials and 6 conventional medicine trials described as ‘larger higher quality trials’.
• If all 21 homeopathy trials they identified as higher quality had been analysed, they would have got the opposite result – that homeopathy has an effect beyond placebo.10 No justification has been given for discounting the other 13 high quality trials just because they were smaller.
• The study fails a ‘sensitivity analysis’ i.e. if you switch just one of the 8 homeopathy trials they chose from the 110, you get the opposite result i.e. that homeopathy works better than placebo.4
• None of the 8 homeopathy trials they used involved individualised homeopathic treatment – the form of homeopathy considered to be ‘usual care’ – instead they were testing non-individualised homeopathic medicines given without a consultation. The conclusions of this study therefore have no relevance to the evidence base ‘for or against’ usual homeopathic treatment.
• The 8 trials used were testing a range of different homeopathic medicines being used for different conditions. One would assume that, as with conventional medicine, some homeopathic treatments work and others don’t, so it is no surprise that some of these studies were negative whilst others were positive. Mixing the results together to decide whether they show ‘homeopathy’ works makes no scientific sense.
• In the original paper Shang et al. do not state which 8 homeopathy trials and which 6 conventional trials they used. Such lack of reporting transparency and precision is typically not tolerated in studies of conventional medicine.6 Following an outcry, the authors eventually provided this information.
• The decade-old Shang paper is out of date and has been superseded. Since 2005, 41 more placebo-controlled randomised trials have been published which would have been suitable for inclusion in Shang’s review. These have been included in the most recent meta-analysis by Mathie et al. published in 2014 which reached a positive conclusion.13

https://www.hri-research.org/resources/homeopathy-the-debate/the-lancet-paper-by-shang-et-al/