Homeopathy Papers

Australia’s National Health and Medical Research Council (NHMRC) Misled Scientists and the Public Over Homeopathy

dddcefcbbf

This article details how Australia’s National Health and Medical Research Council (NHMRC) totally contrived a “study” to discredit homeopathy. It shows how they claimed to examine1863 papers, but really drew their conclusions from only five papers! All positive evidence for homeopathy was intentionally hidden. That bias was present from the highest levels of the NHMRC is shown by the Chariman’s statement: “As Chairman of NHMRC I can also assure you that NHMRC does not support homeopathy.” This specious study was reported worldwide as factual, so it is important that it be thoroughly discredited. This paper sheds light on the true story.

Executive Summary

Extracted from the Submission to the Commonwealth Ombudsman by Complementary Medicines Australia, Australian Homoeopathic Association and Australian Traditional Medicine Society, submitted in August 2016. HRI provided the scientific analysis for this submission.

Between October 2010 and March 2015 the National Health and Medical Research Council (NHMRC) conducted an investigation into homeopathy, to inform the Australian community on the “effectiveness of homeopathy.” This investigation was an example of NHMRC’s function to ‘advise the community’ under section 7(1)(a) of the NHMRC Act 1992. It was instigated under the NHMRC Strategic Plan 2010-2012 “to examine the evidence underlying the alternative medicines most highly used” and culminated in a formal review of the evidence for effectiveness of homeopathy (the Homeopathy Review).

NHMRC’s findings were that, for the 61 health conditions covered by the Review:

‘…no good-quality, well-designed studies with enough participants for a meaningful result reported either that homeopathy caused greater health improvements than placebo, or caused health improvements equal to those of another treatment’.

Their overall conclusion, based on this assessment of the evidence was that, “…there are no health conditions for which there is reliable evidence that homeopathy is effective.

NHMRC’s media release announcing publication of the Review claimed that, “The conclusion is based on the findings of a rigorous assessment of more than 1800 papers” and was accompanied by a statement presenting their interpretation of the results as, “Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious.”

The potential impact of an evidence review by a respected institution such as NHMRC cannot be overstated – the general public, health practitioners, decision-makers and other researchers all rely on their findings. It is therefore essential that NHMRC reviews are free from bias, providing a fair and objective assessment of a given topic. Risk of bias is normally minimised by three key safeguards:

  1. Use of standardised and accepted scientific methods
  2. Internal policies and procedures g. adherence to NHMRC legislation, standards, guidelines and conflict of interest policy
  3. Transparency and accountability g. public disclosure of processes followed, meaningful public consultation and accurate communications to the public.

It is our contention that in the case of the NHMRC Homeopathy Review all three safeguards were breached, exposing the process to unacceptable levels of anti-homeopathy bias – evident in both its administrative and methodological aspects – which led directly to distortion of the Review results. Key examples of this bias are outlined below:

In December 2010 the NHMRC agreed to a draft position statement, describing homeopathy as ‘unethical’, ‘inefficacious’, ‘implausible’ and even ‘deceptive’. This statement was prepared without any scientific evaluation of homeopathy by NHMRC, being based solely on the findings of a single non-academic report prepared by a committee of Members of Parliament in the UK.  This anti-homeopathy position on the part of NHMRC was echoed in an article published around the same time by its CEO Professor Anderson who described homeopathy as “a retreat from reason” and “an alleged therapy”. In the wake of criticism relating to bias and lack of procedural and scientific rigour following a leak of this draft statement into the public domain, NHMRC initiated a formal investigation into homeopathy, instigated by Professor Anderson.

Professor Anderson appointed an expert overview committee for the Review that was compromised by conflicts of interest and bias from the outset. This Homeopathy Working Committee (HWC), which was directly involved in deciding how the evidence was analysed and interpreted, was initially chaired by Professor Peter Brooks who signed a Declaration of Interest (DOI) form, declaring he was not “affiliated or associated with any organisation whose interests are either aligned with or opposed to homeopathy,” despite being a member of the anti-homeopathy medico-political lobby group Friends of Science in Medicine (FSM). The CEO appointed Professor Brooks as Chair after being notified by letter of FSM’s attempt to influence NHMRC reviewers. After the conflict was exposed, although Professor Brooks stood down as Chair, the CEO/NHMRC supported his continuing presence as an active member of the HWC for the duration of the Review.

The HWC also failed to contain a single expert in either homeopathy or homeopathic research; exclusion of a topic expert is unprecedented in NHMRC processes of this kind and is in breach of NHMRC guidelines and policies informing appointments to its expert committees.

The HWC carried out the formal review of the evidence for effectiveness of homeopathy twice, under two different external contractors. NHMRC’s claim that the first review, which appears to have found positive evidence for the effectiveness of homeopathy, was rejected on grounds of poor quality, not its results; this is despite the review being conducted by a highly experienced and reputable reviewer who is co-author of NHMRC’s own Additional Levels of Evidence and Grades for Recommendations for Developers of Guidelines guidance document. The existence of this first review (final draft completed in August 2012) was hidden from the public and NHMRC continues to refuse to disclose details of this review and why it was terminated.

The method used for assessing the evidence the second time (resulting in the final published Homeopathy Review), was not a ‘standardised accepted method’; it was created specifically for this Review by NHMRC. The approach hinged on their unique definition of ‘reliable’ evidence i.e. for a trial to be ‘reliable’ it had to have more than 150 participants and meet an unusually high standard of trial quality (scoring 5 out of 5 on the ‘Jadad’ quality rating scale or equivalent on other scales).  All trials which fell below either of these thresholds were disregarded as being of ‘insufficient size and/or quality to warrant further consideration of their findings’ (Overview Report, p.38). NHMRC’s quality threshold for a ‘reliable’ trial is highly unusual, but their decision to set a minimum ‘sample size’ of 150 participants for trial reliability was completely arbitrary, unprecedented and cannot be justified scientifically.

When explaining their decision to classify trials with less than N=150 as being ‘small’ or ‘very small’, NHMRC refer to an article in the highly respected journal BMJ1. NHMRC said: “HWC considered the following study in the development of these thresholds: Influence of trial sample size on treatment effect estimates: meta-epidemiological study.” (Overview Report Appendices, p.274). This implies that their decision to dismiss trials smaller than N=150 as ‘unreliable’ is scientifically justified by this paper. It is not. NHMRC correctly describes most of the homeopathic trials they assessed as using ‘continuous outcomes’, yet the BMJ paper states categorically that its findings cannot be applied to trials of this kind.

NHMRC used this citation of the BMJ study against the N=150 threshold multiple times across the final report documents released to the public, who would not question that an expert body such as NHMRC would make such a fundamental error and/or intentionally publish misleading information.

In NHMRC’s 2015 media release they further misled the public by stating that “more than 1800” papers underwent “rigorous assessment”, implying that their conclusions were based on a thorough examination of this exhaustive body of evidence.  In fact, although 1863 papers on homeopathy were identified by NHMRC reviewers or submitted by external parties, NHMRC’s choice of inclusion criteria meant that only 267[1] studies were assessed in any detail, being considered to be possibly relevant to the Review; of those, only 176 trials were  finally assessed as suitable to be entered into the Review.

Applying NHMRC’s combined ‘reliability’ filter of 150 participants very high quality, then led to 171 out of 176 trials being dismissed as ‘unreliable’ and of “insufficient size and/or quality to warrant further investigation of their findings”.  Having reduced the evidence base to only 5 ‘reliable’ trials, none of which NHMRC considered to show homeopathy to be effective, the HWC found (unsurprisingly) that there is ‘no reliable evidence’ that homeopathy is effective.

The direct impact of this approach was to exclude good quality, positive trials showing homeopathy to be effective, thus distorting the results. This issue was identified in 2013 by NHMRC’s own expert independent reviewer with no connection to the homeopathy sector, who expressed concern about NHMRC’s concluding statements, saying, “If the intent is to provide general statements about the effectiveness of homeopathy, then ‘no reliable evidence’ may not adequately reflect the research. For example, when a substantial proportion of small (but good quality) studies show significant differences, […] ‘no reliable evidence’ does not seem an accurate reflection of the body of evidence.”2

NHMRC’s chosen scientific method for the Homeopathy Review was an ‘overview’ i.e. a ‘review of reviews’. This meant that instead of analysing the 176 individual studies themselves, they relied only on secondary data provided about those trials in other ‘systematic reviews’ (SRs) that summarise evidence. As much of the necessary information was inaccurate or missing, this inherent flaw reduces the credibility of the Review.

A further ‘limitation’ listed by NHMRC in using this approach is that they may have missed relevant single trials that were not described in the SRs, but misleadingly suggest that this ‘risk’ was ‘offset’ by inviting submissions from homeopathy interest groups and a via a formal public consultation. They inaccurately state that this externally submitted evidence was, “…assessed using a similar method to that applied in the overview” (Information Paper, p.8) but ‘… did not alter the overall findings of the assessment of the evidence (Information Paper, p.25).

In fact, external submissions were assessed entirely differently and separately from the rest of the evidence base, in a way which meant that it was never possible for any externally submitted evidence to alter the results of the Review: of 49 submitted trials that NHMRC considered suitable for their Review, 0 entered the Overview Report. This makes a sham of NHMRC’s apparent attempt at external co-operation and transparency.

NHMRC also failed to disclose that the external contractor who assessed submissions from the public consultation (40 studies, representing almost a quarter of the total evidence considered for the Review) has direct links to the anti-homeopathy lobby group FSM: the Australian Research Centre for Health of Women and Babies (ARCH), Robinson Research Institute (RRI), University of Adelaide employed supporters of FSM with direct links to FSM’s co-founder, Professor Alastair MacLennan. Prof MacLennan had already directly lobbied NHMRC (8 April 2014) on behalf of FSM to support NHMRC’s negative findings on homeopathy, urging that Australians not be “sold snake oil” – a phrase reiterated by the NHMRC CEO in his public orations on the subject3. Over 50 RRI staff, including its Director, are official FSM Supporters yet NHMRC did not report any conflicts of interest.

Anti-homeopathy conflicts of interest within NHMRC were not limited to the HWC and ARCH: during the Review, members of the NHMRC Health Care Committee (HCC) that the HWC directly reported to, as well as NHMRC Council itself, contained FSM supporters. These significant conflicts were also not reported. This reveals a culture where anti-homeopathy vested interests were present at the highest levels of the organisation. This is exemplified by the Chair of Council’s statement in July 2012 that he was no supporter of homeopathy” and that “as Chairman of NHMRC I can also assure you that NHMRC does not support homeopathy.”

NHMRC’s own documentation shows that significant modifications were made to the research protocol (the precise method used to assess and interpret the evidence), months into the review process. Furthermore NHMRC did not report that these changes included such key factors as introducing the N=150 threshold which underpinned their findings and overall conclusion. Agreeing to a protocol before a review process starts is a recognised safeguard against scientific bias, so making such significant post-hoc changes fundamentally undermines the credibility of the Review results.

In summary, NHMRC have misled the public by giving the impression that ‘no stone has been left unturned’ in the Homeopathy Review process – that the most rigorous, open and transparent methods were used to evaluate all available evidence concerning the effectiveness of homeopathy for any health condition – and they failed to find a single piece of valid scientific evidence that homeopathy works.

It is no wonder that the national and international media, having picked up NHMRC’s press release announcing the findings of the Review ran with such damning headlines as, “Homeopathy Doesn’t Work”4, “1800 studies later scientists conclude homeopathy doesn’t work”5 and “There is no scientific case for homeopathy: the debate is over”6.

Yet our investigation has revealed that multiple customary safeguards against bias were removed, facilitating a flawed process that appeared to be engineered to reach predetermined conclusions, sympathetic to the views of anti-homeopathy vested interests. The Submission shows how these interests were allowed undue influence in the Review, directly influencing the outcome, and how key NHMRC personnel – from the CEO down – publically endorsed (even iterated) the same anti-homeopathy views throughout NHMRC’s focus on homeopathy between 2010 and 2015.

Detailed analysis of the NHMRC Overview Report has identified at least 5 clinical conditions (diarrhoea in children, sinusitis, allergic rhinitis, URTIs and lower back pain) for which there is reliable evidence for the effectiveness of homeopathy; it is possible that there are more which we cannot identify due to the lack of accuracy and clarity of NHMRC’s data throughout the Review.

Thus NHMRC’s conclusion that “…there are no health conditions for which there is reliable evidence that homeopathy is effective” is inaccurate, highly misleading to the public and unjustly damaging to the credibility of the homeopathy sector. It is therefore essential that all published documents relating to the Homeopathy Review are rescinded in their entirety.

References

1  Dechartres, A., Trinquart, L., Boutron, I. & Ravaud, P. (2013) Influence of trial sample size on treatment effect estimates: meta-epidemiological study, BMJ, 346:f2304

2  2013-07-09 – Australasian Cochrane Centre Methodological Review – FOI 2015-16 008-13-Doc 13

3  2014-04-08 – FSM congratulatory open letter to NHMRC re. NHMRC draft Information Paper

4  Lupkin, S. (2015) Homeopathy Doesn’t Work, Major Australian Study Concludes – ABC News

5  Blakemore, E. (2015) 1,800 Studies Later, Scientists Conclude Homeopathy Doesn’t Work Smithsonian

6  Ernst, E. (2015) There is no scientific case for homeopathy: the debate is over The Guardian

Attachment – Executive Summary

https://tinyurl.com/ExecSummaryOmbudsman (available 19.00 BST, April 6)
Extracted from the Submission to the Commonwealth Ombudsman by Complementary Medicines Australia, Australian Homoeopathic Association and Australian Traditional Medicine Society, submitted in August 2016.  HRI provided the scientific analysis for this submission.

The Australian Report: an in depth analysis

(available 19.00 BST, April 6)
Ms Rachel Roberts, CEO Homeopathy Research Institute presents key facts from HRI’s in-depth scientific analysis of NHMRC’s Homeopathy Review, demonstrating that the public were misled by serious misreporting of the evidence.

Media

Carolyn Howgego, HRI Communications Manager

07596 833664/[email protected]

“Just One Drop” premiere enquiries only

Storm PR, Anneka Wahlhaus, Senior Account Manager

0207 240 2444/[email protected]

Available for interview

Rachel Roberts, HRI chief executive (please Carolyn Howgego)

Further information:

HRI

The Homeopathy Research Institute (HRI) is an innovative international charity created to address the need for high quality scientific research in homeopathy. The charity was founded by physicist, Dr Alexander Tournier.

NHMRC

The National Health and Medical Research Council (NHMRC) is Australia’s leading expert body promoting the development and maintenance of public and individual health standards.

AHA

The Australian Homœopathic Association (AHA) is the only national association representing professional homœopaths in Australia.

*AHA’s complaint to the Australian Ombudsman was submitted jointly with Complementary Medicines Australia (CMA) and the Australian Traditional Medicine Society (ATMS).

Commonwealth Ombudsman

The Commonwealth Ombudsman considers and investigates complaints from people who believe they have been treated unfairly or unreasonably by an Australian Government department or agency.

The Cochrane Collaboration

Cochrane is a global independent network of researchers, professionals, patients, carers, and people interested in health.  Their work is recognized as representing an international gold standard for high quality, trusted information.

References

  1. Reilly, D. T., Taylor, M. A., McSharry, C. & Aitchison, T. (1986) Is homoeopathy a placebo response? Controlled trial of homoeopathic potency, with pollen in hayfever as model. Lancet (London, England), 2:881-886
  2. Zabolotnyi, D. I. et al. (2007) Efficacy of a complex homeopathic medication (Sinfrontal) in patients with acute maxillary sinusitis: a prospective, randomized, double-blind, placebo-controlled, multicenter clinical trial. Explore (New York, N.Y.), 3:98-109
  3. Jacobs, J., Jonas, W. B., Jiménez-Pérez, M. & Crothers, D. (2003) Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials, The Pediatric Infectious Disease Journal, 22:229-234

[1] 267 studies were reviewed in full text, comprising 183 studies identified by Optum (a mixture of systematic reviews and individual trials), 25 trials submitted to NHMRC by stakeholders and 59 trials submitted through public consultation.

About the author

Carolyn Howgego

Carolyn Howgego

HRI is an innovative international charity created to address the need for high quality scientific research in homeopathy. The charity was founded by physicist, Dr Alexander Tournier, who previously worked as an independent researcher for Cancer Research UK, conducting interdisciplinary research at the boundaries between mathematics, physics and biology.

Leave a Comment