Research bias in ME/CFS

Research in every field deals with bias. Research into myalgic encephalomyelitis/chronic fatigue syndrome faces specific kinds of bias issues as it navigates significant gaps in scientific knowledge and several opposing viewpoints.

Emphasis on randomized controlled trials and systematic reviews
Following an Evidence-Based Medicine (EBM) model gives greater weight to systematic reviews and randomized controlled trials. However, some systematic reviews and a number of randomized controlled trials have been shown to have significant flaws, including a failure to follow pre-defined procedures or standards, and altering the primary outcome(s) part way through unblinded trials either without adequate justification or without a referring to the change, and failure to comply with prespecified trial plans, serious failings in ethics including false ethical approval statements in the publication of more than a dozen research outputs about children with CFS, and research incompatible with child protection and safeguarding, e.g. in the SMILE trial. The most influential examples of these are the Cochrane review of exercise therapy for CFS, which Cochrane initially planned to withdraw after admitting it did not meet the set standards, and the PACE trial, which is the largest treatment trial for CFS, and resulting in a lengthy legal battle for the release of the original primary outcome data including the results of the six-minute walk test.

PACE trial
Davenport et al. (2018) state the lack of randomized controlled trials in ME/CFS research mean that an evidence-based model can be unduly influenced by a very small number of studies, for example the PACE trial found that treating ME/CFS with graded exercise therapy or cognitive behavioral therapy did not result in clinically significant improvements - but this trial heavily influenced the clinical use of those treatments. The PACE trial was so problematic that it was used an the example for Prof. Bruce Levin's presentation How Not To Conduct A Randomized Clinical Trial, and The Lancet received open letters signed by over 100 people, including researchers and academics, calling for the trial to be retracted.

Cochrane systematic review
The highly influential Cochrane review of exercise therapy for chronic fatigue syndrome was also found to have significant flaws. Cochrane announced it would be withdrawn, but when the authors refused to agree Cochrane then changed its position and published repeated amendments and editorial notes instead. Cochrane allocated to reviews about chronic fatigue syndrome to its Common Mental Disorders group, despite the long standing classification of CFS as a neurological disease by the World Health Organization, and the absence of any psychological symptoms in any of the differing diagnostic criteria. Cochrane have withdrawn the individual patient data used in the exercise review.

Reliance on unpublished data and withholding outcomes
A number of journals have disregarded their data access policies or published studies that did not comply with them, resulting in editorial corrections or withdrawn data rather than retractions, for example in a PLoS PACE trial follow up publication and the Cochrane systematic review of exercise as a therapy for chronic fatigue syndrome. When outcome swapping has taken place during a trial, in some cases there has either been no mention of the change in the published trial results, or the change has been referred to but the data has been omitted, and researchers have refused to provide it to other academics, for example in the PACE trial which received £5 million in funding.

Clinicians' disbelief in unsuccessful trials
Some clinicians appear to have become so convinced that a treatment must work during a trial that a null or negative response to the treatment has been misattributed to a patient mistake. Examples include the FINE trial, which was the sister trial to the PACE trial but for severely ill patients, where a null result was published along with a comment that some nurses delivering the treatment felt that "the bastards [patients] just don't want to get better". Clear conflicts of interest with representing both insurance companies in dismissing claims for illness insurance, in private training companies run by clinician researchers, and in prior published popular self-help books for chronic fatigue or CFS commonly occur, and are often not disclosed in publication, including in all of the three PACE trial principle authors, at least one other PACE trial author and the majority of biopsychosocial treatment studies. A publication by Simon Wessely, one of the clinicians who delivered treatment in the PACE trial, described the management of the trial using a comparison of steering a ship to its pre-determined destination.

Bias and inadequate peer review
Many journals do not have open peer reviews, however some have been found to have clearly biased peer reviewers, or authors are invited to suggest "independent" peer reviewers but suggest biased ones. In one published peer review, the peer reviewer commented that they did not read past the abstract, but the research was published regardless. In another case, an author submitted an invited editorial and was asked to nominate independent peer reviewers but nominated only those with well-documented adherence to the same paradigm of ME/CFS. In this case, the journal rejected the publication, and the author then published it on a popular health website which did not require peer review.

A series of publications about chronic fatigue syndrome in UK school children were published using the same invalid ethics approval number without this being picked up in peer review. This lead to an investigation into pediatrician Esther Crawley by the University of Bristol, and published corrections for all articles - none of which had an ethics approval. The publications were reclassified as "service evaluations" which did not require ethics approval.

Editorial bias
A number of highly influential journals have published editorials either strongly in favor of controversial treatments, criticizing key publications which challenged the journals' previous ideas about ME/CFS, or harshly criticizing advocates, patients and even some researchers or clinicians who raised genuine scientific queries or made scientific criticisms of research; such criticism has continued for decades despite changes in editorship. Journals criticized for editorial, publication or other bias including the BMJ, the Lancet, which published the PACE trial; and Archives of Disease in Childhood, which published an extremely lengthy editorial correction rather than retract the SMILE trial, a pediatric trial of the controversial Lightning process, which is based on neurolinguistic programming.

Failure to retract, withdraw or correct flawed research
A number of journals plus Cochrane, have published research which failed to meet their own standards, including clinical treatment trials of graded exercise therapy and cognitive behavioral therapy for ME/CFS, and have then failed to retract some which could not be appropriately corrected in a way that met the journal standards. A significant number of editorial corrections, expressions of concern or corrections by authors have been published, merged data from a successful feasibility study into a treatment trial; the journal's editor described the correction as "extensive". Cochrane announced it would withdraw a systematic review of exercise as a therapy for CFS after a detailed complaint described its failure to meet Cochrane's own standards and incorrect use of the GRADE framework required by Cochrane - but then reversed this decision after the authors refused to agree to the withdrawal of the review.

The Lancet responded to multiple open letters requesting an investigation into the PACE trial and an independent re-analysis of the data by refusing those requests and continued to do so after the full PACE trial data was released which showed that patient's six-minute walk test results had barely improved, and the 13% of those taking part in the trial met at least one of the "recovery" outcomes at the start of the trial due to the definition for recovery being lowered part-way through the trial.

Bias recognition
Geraghty et al. (2019) looked at evidence behind the popular cognitive behavioral model of ME/CFS, and found that a number of research papers contained significant evidence that did not support the model combined with contradictory abstracts or conclusions at odds with the authors' results, and notable selective interpretation of evidence, such as ignoring any contradictory research.

Sonia Lee (PhD) assessed research waste in ME/CFS by comparing the PACE trial with two cellular trials, finding that "selective reporting", an indicator of research waste, was greater in the PACE trial; however this paper (2017) was not peer reviewed. Nacul et al. (2017) discussed the effects of selection bias, highlighting how psychosocial/biopsychosocial studies frequently used the broadest diagnostic criteria - the Oxford criteria - and did not break results down by subgroup. The use of the Oxford criteria lead to a 100-fold increase in disease prevalence compared to the Canadian Consensus Criteria (CCC); with 14 out of 15 patients meeting the Oxford criteria typically failing to meet the Canadian Consensus Criteria for ME/CFS.

Notable studies

 * 1982, Suffering and the Social Construction of Illness: The Delegitimation of Illness Experience in Chronic Fatigue Syndrome | (Abstract)
 * 2005, Chronic Fatigue Syndrome: Editorial Bias in The Lancet (Abstract) - Not peer-reviewed
 * 2011, Chronic Fatigue Syndrome: Editorial Bias in the British Medical Journal (Abstract)
 * 2017, Bias, misleading information and lack of respect for alternative views have distorted perceptions of myalgic encephalomyelitis/chronic fatigue syndrome and its treatment (Full Text)
 * 2017, How have selection bias and disease misclassification undermined the validity of myalgic encephalomyelitis/chronic fatigue syndrome studies? (Full text)
 * 2017, Research Waste in ME/CFS (Full text) - not peer reviewed
 * 2018, Checking our blind spots: current status of research evidence summaries in ME/CFS (Abstract) - Editorial
 * 2018, Are ME/CFS Patient Organizations "Militant"? : Patient Protest in a Medical Controversy. (Abstract)
 * 2019, The ‘cognitive behavioural model’ of chronic fatigue syndrome: Critique of a flawed model (Full text)

Articles and blogs

 * 2017, PACE team response shows a disregard for the principles of science - Jonathan Edwards


 * 2017, MEA Review: The SMILE trial - a lesson in how not to conduct clinical trials in people with ME/CFS (Full text) - by Dr Charles Shepherd, ME Association medical advisor