PACE trial: Difference between revisions

From MEpedia, a crowd-sourced encyclopedia of ME and CFS science and history
(fleshed out the controversy section a bit, think I covered all the impt points, feel free to be more fastidious with the citations.)
Line 5: Line 5:
==Controversy==
==Controversy==


The PACE Trial has been heavily criticised by patient groups and some researchers and science journalists for a number of methodological problems, most glaringly outcome switching and underreporting of harms.  Many critics have noted that the criteria for "recovery", by PACE's standards, were broader than the criteria for entry, meaning that one could be simultaneously sick with "chronic fatigue syndrome" and completely cured as per their sloppy definitions.<ref>[http://blogs.plos.org/mindthebrain/2015/10/29/uninterpretable-fatal-flaws-in-pace-chronic-fatigue-syndrome-follow-up-study/ "Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study"]</ref><ref>[https://www.youtube.com/watch?v=KSGtZgzkoRQ "James Coyne: A skeptical look at the PACE chronic fatigue trial (video talk at Edinburgh University 16 Nov 2015"]</ref><ref>[http://www.virology.ws/2015/10/21/trial-by-error-i/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study"]</ref><ref>[http://www.virology.ws/2015/10/22/trial-by-error-ii/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (second instalment)"]</ref><ref>[http://www.virology.ws/2015/10/23/trial-by-error-iii/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (final instalment)"]</ref><ref>[http://www.virology.ws/2015/10/30/david-tuller-responds-to-the-pace-investigators/ "David Tuller responds to the PACE Investigators"]</ref><ref>[http://www.virology.ws/2015/11/04/trial-by-error-continued-did-the-pace-study-really-adopt-a-strict-criterion-for-recovery/ "Trial By Error, Continued: Did the PACE Study Really Adopt a ‘Strict Criterion’ for Recovery?"]</ref><ref>[http://www.virology.ws/2015/11/09/trial-by-error-continued-why-has-the-pace-studys-sister-trial-been-disappeared-and-forgotten/ "Trial By Error, Continued: Why has the PACE Study’s Sister Trial been Disappeared and Forgotten?"]</ref><ref>[http://www.virology.ws/2015/11/13/an-open-letter-to-dr-richard-horton-and-the-lancet/ "An open letter to Dr. Richard Horton and The Lancet"]</ref><ref>[http://www.virology.ws/2015/11/17/trial-by-error-continued-pace-teams-work-for-insurance-companies-not-related-to-pace-really/ "Trial by error, Continued: PACE Team’s Work for Insurance Companies Is “Not Related” to PACE. Really?"]</ref><ref>[http://keithsneuroblog.blogspot.co.uk/2015/11/pace-thoughts-about-holes.html "PACE - Thoughts about Holes"]</ref><ref>[http://keithsneuroblog.blogspot.co.uk/2015/11/song-for-siren.html "Song for Siren"]</ref><ref>[http://www.slate.com/articles/health_and_science/medical_examiner/2015/11/chronic_fatigue_pace_trial_is_flawed_should_be_reanalyzed.html "Hope for Chronic Fatigue Syndrome"]</ref><ref>[http://www.slideshare.net/jamesccoyne/edinburgh-skeptics-in-the-pub-talk-on-pace-chronic-fatigue-trial "Edinburgh Skeptics in the Pub talk on PACE chronic fatigue trial"]</ref><ref>[http://www.meaction.net/2015/11/01/prof-jonathan-edwards-pace-trial-is-valueless/ "Prof. Jonathan Edwards: PACE Trial is valueless"]</ref><ref>[http://www.investinme.org/Documents/Lancet/Letter%20to%20Editor%20of%20the%20Lancet%20November%202015.pdf Invest in ME - To The Editor of the Lancet – The PACE Trial]</ref>  Furthermore, an informal survey conducted by the MEAssociation in 2012 showed that 80% of patients had their symptoms worsen after a course of GET. <ref>http://www.meassociation.org.uk/2015/05/23959/</ref>  Further analysis of the trial is difficult as the various institutions involved in the trial have been subject to numerous FOI requests, which they have repeatedly brushed asided on the charge that they were "vexatious".
The PACE Trial has been heavily criticised by patient groups and some researchers and science journalists for a number of methodological problems, most glaringly outcome switching and underreporting of harms.  Many critics have noted that the criteria for "recovery", by PACE's standards, were broader than the criteria for entry, meaning that one could be simultaneously sick with "chronic fatigue syndrome" and completely cured as per their sloppy definitions.<ref>[http://blogs.plos.org/mindthebrain/2015/10/29/uninterpretable-fatal-flaws-in-pace-chronic-fatigue-syndrome-follow-up-study/ "Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study"]</ref><ref>[https://www.youtube.com/watch?v=KSGtZgzkoRQ "James Coyne: A skeptical look at the PACE chronic fatigue trial (video talk at Edinburgh University 16 Nov 2015"]</ref><ref>[http://www.virology.ws/2015/10/21/trial-by-error-i/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study"]</ref><ref>[http://www.virology.ws/2015/10/22/trial-by-error-ii/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (second instalment)"]</ref><ref>[http://www.virology.ws/2015/10/23/trial-by-error-iii/ "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (final instalment)"]</ref><ref>[http://www.virology.ws/2015/10/30/david-tuller-responds-to-the-pace-investigators/ "David Tuller responds to the PACE Investigators"]</ref><ref>[http://www.virology.ws/2015/11/04/trial-by-error-continued-did-the-pace-study-really-adopt-a-strict-criterion-for-recovery/ "Trial By Error, Continued: Did the PACE Study Really Adopt a ‘Strict Criterion’ for Recovery?"]</ref><ref>[http://www.virology.ws/2015/11/09/trial-by-error-continued-why-has-the-pace-studys-sister-trial-been-disappeared-and-forgotten/ "Trial By Error, Continued: Why has the PACE Study’s Sister Trial been Disappeared and Forgotten?"]</ref><ref>[http://www.virology.ws/2015/11/13/an-open-letter-to-dr-richard-horton-and-the-lancet/ "An open letter to Dr. Richard Horton and The Lancet"]</ref><ref>[http://www.virology.ws/2015/11/17/trial-by-error-continued-pace-teams-work-for-insurance-companies-not-related-to-pace-really/ "Trial by error, Continued: PACE Team’s Work for Insurance Companies Is “Not Related” to PACE. Really?"]</ref><ref>[http://keithsneuroblog.blogspot.co.uk/2015/11/pace-thoughts-about-holes.html "PACE - Thoughts about Holes"]</ref><ref>[http://keithsneuroblog.blogspot.co.uk/2015/11/song-for-siren.html "Song for Siren"]</ref><ref>[http://www.slate.com/articles/health_and_science/medical_examiner/2015/11/chronic_fatigue_pace_trial_is_flawed_should_be_reanalyzed.html "Hope for Chronic Fatigue Syndrome"]</ref><ref>[http://www.slideshare.net/jamesccoyne/edinburgh-skeptics-in-the-pub-talk-on-pace-chronic-fatigue-trial "Edinburgh Skeptics in the Pub talk on PACE chronic fatigue trial"]</ref><ref>[http://www.meaction.net/2015/11/01/prof-jonathan-edwards-pace-trial-is-valueless/ "Prof. Jonathan Edwards: PACE Trial is valueless"]</ref><ref>[http://www.investinme.org/Documents/Lancet/Letter%20to%20Editor%20of%20the%20Lancet%20November%202015.pdf Invest in ME - To The Editor of the Lancet – The PACE Trial]</ref>  Moreover, an informal survey conducted by the ME Association in 2012 showed that 80% of patients had their symptoms worsen after a course of GET. <ref>http://www.meassociation.org.uk/2015/05/23959/</ref>  Further analysis of the statistical data is difficult as the various institutions involved in the trial have been subject to numerous FOI requests, which they have repeatedly brushed asided on the charge that they were "vexatious".


==Criticism==
==Criticism==

Revision as of 17:30, December 11, 2015

The PACE Trial was a large-scale trial to test and compare the effectiveness of four treatments for people suffering from chronic fatigue syndrome (CFS), also known as myalgic encephalomyelitis (ME). These are adaptive pacing therapy (APT), cognitive behaviour therapy (CBT), Graded Exercise Therapy (GET), and standardised Specialist Medical Care (SMC). The protocol paper was published in BMC Neurology in 2007.[1] The main study was published in The Lancet in 2011.[2].

Controversy[edit | edit source]

The PACE Trial has been heavily criticised by patient groups and some researchers and science journalists for a number of methodological problems, most glaringly outcome switching and underreporting of harms. Many critics have noted that the criteria for "recovery", by PACE's standards, were broader than the criteria for entry, meaning that one could be simultaneously sick with "chronic fatigue syndrome" and completely cured as per their sloppy definitions.[3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18] Moreover, an informal survey conducted by the ME Association in 2012 showed that 80% of patients had their symptoms worsen after a course of GET. [19] Further analysis of the statistical data is difficult as the various institutions involved in the trial have been subject to numerous FOI requests, which they have repeatedly brushed asided on the charge that they were "vexatious".

Criticism[edit | edit source]

Professor Ronald Davis, Stanford University[edit | edit source]

“I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.”

Dr. Bruce Levin, Columbia University[edit | edit source]

“To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”

Dr. Arthur Reingold, University of California, Berkeley[edit | edit source]

“Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”

Dr. Jonathan Edwards, University College London[edit | edit source]

“It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”

Dr. Leonard Jason, DePaul University[edit | edit source]

“The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”

Professor James Coyne, University Medical Center, Groningen[edit | edit source]

  • "The data presented are uninterpretable. We can temporarily suspend critical thinking and some basic rules for conducting randomized trials (RCTs), follow-up studies, and analyzing the subsequent data. Even if we do, we should reject some of the interpretations offered by the PACE investigators as unfairly spun to fit what has already a distorted positive interpretation of the results."
  • "The self-report measures do not necessarily capture subjective experience, only forced choice responses to a limited set of statements."
  • "One of the two outcome measures, the physical health scale of the SF-36 requires forced choice responses to a limited set of statements selected for general utility across all mental and physical conditions."
  • "Despite its wide use, the SF-36 suffers from problems in internal consistency and confounding with mental health variables. Anyone inclined to get excited about it should examine its items and response options closely. Ask yourself, do differences in scores reliably capture clinically and personally significant changes in the experience and functioning associated with the full range of symptoms of CHF?"
  • "The validity other primary outcome measure, the Chalder Fatigue Scale depends heavily on research conducted by this investigator group and has inadequate validation of its sensitivity to change in objective measures of functioning."
  • "Such self-report measures are inexorably confounded with morale and nonspecific mental health symptoms with large, unwanted correlation tendency to endorse negative self-statements that is not necessarily correlated with objective measures."

Professor Coyne spoke critically about the PACE trial at Edinburgh University in November 2015.[20][21][22][23][24][25]

Professor Coyne has questioned whether the PACE trial paper could ever have been properly peer-reviewed.[26]

David Tuller, Public health journalist, University of California, Berkeley[edit | edit source]

  • "The study included a bizarre paradox: participants’ baseline scores for the two primary outcomes of physical function and fatigue could qualify them simultaneously as disabled enough to get into the trial but already “recovered” on those indicators–even before any treatment. In fact, 13 percent of the study sample was already “recovered” on one of these two measures at the start of the study."
  • "In the middle of the study, the PACE team published a newsletter for participants that included glowing testimonials from earlier trial subjects about how much the “therapy” and “treatment” helped them. The newsletter also included an article informing participants that the two interventions pioneered by the investigators and being tested for efficacy in the trial, graded exercise therapy and cognitive behavior therapy, had been recommended as treatments by a U.K. government committee “based on the best available evidence.” The newsletter article did not mention that a key PACE investigator was also serving on the U.K. government committee that endorsed the PACE therapies."
  • "The PACE team changed all the methods outlined in its protocol for assessing the primary outcomes of physical function and fatigue, but did not take necessary steps to demonstrate that the revised methods and findings were robust, such as including sensitivity analyses. The researchers also relaxed all four of the criteria outlined in the protocol for defining “recovery.” They have rejected requests from patients for the findings as originally promised in the protocol as “vexatious.” "
  • "The PACE claims of successful treatment and “recovery” were based solely on subjective outcomes. All the objective measures from the trial—a walking test, a step test, and data on employment and the receipt of financial information—failed to provide any evidence to support such claims. Afterwards, the PACE authors dismissed their own main objective measures as non-objective, irrelevant, or unreliable."
  • "In seeking informed consent, the PACE authors violated their own protocol, which included an explicit commitment to tell prospective participants about any possible conflicts of interest. The main investigators have had longstanding financial and consulting ties with disability insurance companies, having advised them for years that cognitive behavior therapy and graded exercise therapy could get claimants off benefits and back to work. Yet prospective participants were not told about any insurance industry links and the information was not included on consent forms. The authors did include the information in the “conflicts of interest” sections of the published papers."

Angela Kennedy, a sociologist of science, health, disability and medicine[edit | edit source]

1. Serious risks to clinical patient safety caused by unsound claims made about the efficacy of CBT and GET following the PACE trial;

2. Gross discrepancies between research and clinical cohorts, and how clinical patients (and the physiological dysfunction associated with them) appear to have been actively excluded from PACE and other research by the research group involved in PACE, which has, ironically, caused serious resulting risks to clinical patient safety in the UK in particular;

3. Related to the above, gross discrepancies in how various sets of patient criteria were used (and/or rejected), including but not limited to a changing of the London criteria by PACE authors from its original state, a set of criteria which was already controversial and problematic to start with for a number of reasons;

4. Failure of the PACE trial authors to acknowledge the range and depth of scientific literature documenting serious physiological dysfunction in patients given diagnoses of ME or CFS, and how CBT and GET approaches may endanger patients in this context;

5. The inclusion of major mental illnesses in the research cohort;

6. The distortion by PACE trial researchers of 'pacing' from an autonomous flexible management strategy for patients into a therapist led Graded Activity approach;

7. The post hoc dismissal of adverse outcomes as irrelevant to the trial, in direct contradiction to what is scientifically known about the physiological dysfunctions of people given diagnoses of Myalgic Encephalomyelitis or Chronic Fatigue Syndrome ;

8. The instability of 'specialist medical care' as a treatment category, and the lack of any sound category of 'control' group.

Analysis by Patient "Citizen-Scientists"[edit | edit source]

Graham McPhee, Tom Kindlon and others collaborated to take the results of their analyses of the PACE trial and create a set of explanatory videos to communicate the flaws that they had found.

Graham McPhee and Janelle Wiley also collaborated with others to create a web site summarising their critical findings.

Tom Kindlon has also has also written a larger number of e-letters and comments in response to the published papers.

MEAction has published an overview of PACE and its flaws.[27]

Investigators[edit | edit source]

Subsequent Publications[edit | edit source]

2012 Cost-effectiveness Paper[edit | edit source]

A 2012 PACE paper was published in PLOS One on cost-effectiveness.[28]

2013 "Recovery" Paper[edit | edit source]

A PACE Trial "recovery" paper was published in Psychological Medicine in 2013.[29]

Graham McPhee and others created a video animation examining the results of the paper: How's that recovery?.

2013 Statistical Analysis Paper[edit | edit source]

This 2013 paper published in Trials Journal further examined the PACE data.[30]

2014 Adverse Effects Paper[edit | edit source]

This 2014 paper published in the Journal of Psychosomatic Research examined adverse effects and deterioration.[31]

2015 Secondary Mediation Analysis[edit | edit source]

This 2015 paper was published in The Lancet Psychiatry. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial (Trudie Chalder, Goldsmith KA, Peter White, Michael Sharpe, Pickles AR)

The paper has been criticised by Byron Hyde.[32]

2015 Long-term Follow-up Paper[edit | edit source]

A long-term follow-up paper was then published in The Lancet Psychiatry in October 2015.[33] The Science Media Centre published expert opinion on the paper.[34]

2015 Longitudinal Mediation Paper[edit | edit source]

This short paper published in Trials Journal on 16 November 2015 concludes "Approximately half of the effect of each of CBT and GET were on physical function was mediated through reducing avoidance of fearful situations".[35]

Future PACE Publications[edit | edit source]

Professor Simon Wessely stated in November 2015 that "there are also more trials in the pipeline".[36]

Freedom of Information Requests[edit | edit source]

There have been a number of Freedom of Information requests in relation to the PACE trial, some of which have requested the release of the trial data.[37]

A ruling by the UK government body the Information Commissioner's Office (ICO) on 27 October 2015 compels Queen Mary University of London (home of the PACE trial's lead investigator Peter White to release the trial data, subject to appeal within 28 days.[38] The freedom of information requests included in the ICO decision are:

James Coyne has publicly called for the trial data to be released and has made a request to the PACE authors to release the data related to a paper published in PLOS One (as required by the terms of publication).[39][40][41]

Richard Horton, editor of The Lancet, stated on 18 April 2011 in an Australian radio interview "The freedom of information requests and the legal fees that have been wracked up over the years because of these vexatious claims has added another £750,000 of taxpayers’ money to the conduct of this study".[42]

2015 Claim Retraction Petition[edit | edit source]

In 2015 a petition demanding retraction of specific incorrect claims made by the PACE investigators.[43]

References[edit | edit source]

  1. "Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy"
  2. "Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial"
  3. "Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study"
  4. "James Coyne: A skeptical look at the PACE chronic fatigue trial (video talk at Edinburgh University 16 Nov 2015"
  5. "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study"
  6. "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (second instalment)"
  7. "TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study (final instalment)"
  8. "David Tuller responds to the PACE Investigators"
  9. "Trial By Error, Continued: Did the PACE Study Really Adopt a ‘Strict Criterion’ for Recovery?"
  10. "Trial By Error, Continued: Why has the PACE Study’s Sister Trial been Disappeared and Forgotten?"
  11. "An open letter to Dr. Richard Horton and The Lancet"
  12. "Trial by error, Continued: PACE Team’s Work for Insurance Companies Is “Not Related” to PACE. Really?"
  13. "PACE - Thoughts about Holes"
  14. "Song for Siren"
  15. "Hope for Chronic Fatigue Syndrome"
  16. "Edinburgh Skeptics in the Pub talk on PACE chronic fatigue trial"
  17. "Prof. Jonathan Edwards: PACE Trial is valueless"
  18. Invest in ME - To The Editor of the Lancet – The PACE Trial
  19. http://www.meassociation.org.uk/2015/05/23959/
  20. "James Coyne: A skeptical look at the PACE chronic fatigue trial - Part 1"
  21. "James Coyne: A skeptical look at the PACE chronic fatigue trial - Part 2"
  22. "James Coyne: A skeptical look at the PACE chronic fatigue trial - Part 3"
  23. Complete talk transcript
  24. Summarised talk transcript
  25. Edited talk audio recording
  26. Was independent peer review of the PACE trial articles possible?
  27. PACE Trial Overview
  28. "Adaptive Pacing, Cognitive Behaviour Therapy, Graded Exercise, and Specialist Medical Care for Chronic Fatigue Syndrome: A Cost-Effectiveness Analysis"
  29. "Recovery from chronic fatigue syndrome after treatments given in the PACE trial"
  30. "A randomised trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome (PACE): statistical analysis plan"
  31. "http://www.sciencedirect.com/science/article/pii/S0022399914001883"
  32. Follow the money
  33. "Rehabilitative treatments for chronic fatigue syndrome: long-term follow-up from the PACE trial"
  34. expert reaction to long-term follow-up study from the PACE trial on rehabilitative treatments for CFS/ME, and accompanying comment piece
  35. "Longitudinal mediation in the PACE randomised clinical trial of rehabilitative treatments for chronic fatigue syndrome: modelling and design considerations"
  36. "The PACE Trial for chronic fatigue syndrome: choppy seas but a prosperous voyage"
  37. "Search results 'pace trial'"
  38. "Freedom of Information Act 2000 (FOIA) Decision notice"
  39. "Why the scientific community needs the PACE trial data to be released"
  40. What it takes for Queen Mary to declare a request for scientific data “vexatious”
  41. Update on my formal request for release of the PACE trial data
  42. Professor Malcolm Hooper’s Detailed Response to Professor Peter White’s letter to Dr Richard Horton about his complaint re: the PACE Trial articles published in The Lancet
  43. "MISLEADING PACE CLAIMS SHOULD BE RETRACTED"