Outcome switching: Difference between revisions

From MEpedia, a crowd-sourced encyclopedia of ME and CFS science and history
m (alt name, may or may not affect results)
m (Text replacement - " first = " to " first = ")
 
(32 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Outcome switching''' or ''outcome swapping'' is when authors of a clinical trial "move the goal posts" during a trial, which may be done to achieve the desired results.  
'''Outcome switching''' or '''outcome swapping''' is when authors of a clinical trial "move the goal posts" during a trial, which may be done to achieve the desired results or may accidentally affect the trial's findings.{{citation needed}}  For example, changing the primary outcome that is used to decide if a treatment is successful from the distance walked in 6 minutes to the percentage of patients who rated their symptoms as "better" or "much better" on a patient questionnaire.


The [[PACE trial]] authors, in a clinical trial of [[ME/CFS]] patients undergoing [[GET]]/[[CBT]], employed this tactic.
The [[PACE trial]] authors, in a clinical trial of [[ME/CFS]] patients undergoing [[GET]]/[[CBT]], employed this tactic.  
 
==Pre-specified outcomes ==
When a clinical trial has a protocol published in advance, it will typically include [[pre-specified outcome|pre-specified primary outcomes]], these are the main measures used to determine if an intervention or drug is effective. Secondary outcomes are considered to be less important. For example, change in average pain level measured with a  questionnaire, biological test results or physical measurements of ability.{{citation needed}}
 
Outcome swapping is the process of changing these primary outcomes during the trial or prior to publication.{{citation needed}} For example, a secondary outcome may be changed with a primary outcome, which may give then change the  results reported in the trial publication.
 
In the [[PACE trial]], outcome swapping led to results showing that both [[CBT]] and [[GET]] were moderately effective, but when a secondary analysis used the original pre-specified outcomes this showed the interventions were less effective than the previously published results.{{citation needed}}


== Articles on Outcome switching in clinical trials==
== Articles on Outcome switching in clinical trials==
*[https://www.psychologytoday.com/blog/side-effects/201604/how-outcome-switching-is-corrupting-medical-research How Outcome Switching is Corrupting Medical Research]<ref>{{Cite web|url=https://www.psychologytoday.com/intl/blog/side-effects/201604/how-outcome-switching-is-corrupting-medical-research|title=How Outcome Switching is Corrupting Medical Research|last=Lane|first=Christopher|date=Apr 30, 2016|website=Psychology Today|language=en|archive-url=|archive-date=|dead-url=|access-date=2018-08-11}}</ref>  
*[https://www.psychologytoday.com/blog/side-effects/201604/how-outcome-switching-is-corrupting-medical-research How Outcome Switching is Corrupting Medical Research]<ref>{{Cite web|url=https://www.psychologytoday.com/intl/blog/side-effects/201604/how-outcome-switching-is-corrupting-medical-research| title = How Outcome Switching is Corrupting Medical Research | last = Lane | first = Christopher | date = Apr 30, 2016 | website = Psychology Today|language=en|archive-url=|archive-date=|url-status=|access-date=2018-08-11}}</ref> - Psychology Today
*[http://www.economist.com/news/science-and-technology/21695381-too-many-medical-trials-move-their-goalposts-halfway-through-new-initiative For my next trick...] <ref>{{Cite news|url=https://www.economist.com/science-and-technology/2016/03/26/for-my-next-trick|title=For my next trick...|last=|first=|date=Mar 26, 2016|work=The Economist|access-date=2018-08-11|archive-url=|archive-date=|dead-url=|language=en}}</ref>
*[http://www.economist.com/news/science-and-technology/21695381-too-many-medical-trials-move-their-goalposts-halfway-through-new-initiative For my next trick...]<ref>{{Cite news | url=https://www.economist.com/science-and-technology/2016/03/26/for-my-next-trick| title = For my next trick... | last = | first = | date = Mar 26, 2016|work=The Economist|access-date=2018-08-11|archive-url=|archive-date=|url-status=|language=en}}</ref> - The Economist
*[http://www.clinicalleader.com/doc/is-outcome-switching-still-a-problem-in-clinical-trials-0001 Is Outcome Switching Still A Problem In Clinical Trials?]<ref>{{Cite web|url=https://www.clinicalleader.com/doc/is-outcome-switching-still-a-problem-in-clinical-trials-0001|title=Is Outcome Switching Still A Problem In Clinical Trials|last=Miseta|first=Ed|date=Mar 7, 2016|website=www.clinicalleader.com|language=en|archive-url=|archive-date=|dead-url=|access-date=2018-08-11}}</ref>
*[http://www.clinicalleader.com/doc/is-outcome-switching-still-a-problem-in-clinical-trials-0001 Is Outcome Switching Still A Problem In Clinical Trials?]<ref>{{Cite web|url=https://www.clinicalleader.com/doc/is-outcome-switching-still-a-problem-in-clinical-trials-0001| title = Is Outcome Switching Still A Problem In Clinical Trials | last = Miseta | first = Ed | date = Mar 7, 2016 | website = clinicalleader.com|language=en|archive-url=|archive-date=|url-status=|access-date=2018-08-11}}</ref> - clinicalleader.com
*[http://www.vox.com/2015/12/29/10654056/ben-goldacre-compare-trials How researchers dupe the public with a sneaky practice called "outcome switching"]<ref>{{Cite news|url=https://www.vox.com/2015/12/29/10654056/ben-goldacre-compare-trials|title=How researchers dupe the public with a sneaky practice called "outcome switching"|last=Belluz|first=Julia|date=Dec 29, 2015|work=Vox|access-date=2018-08-11|archive-url=|archive-date=|dead-url=}}</ref>  
*[http://www.vox.com/2015/12/29/10654056/ben-goldacre-compare-trials How researchers dupe the public with a sneaky practice called "outcome switching"]<ref>{{Cite news | url=https://www.vox.com/2015/12/29/10654056/ben-goldacre-compare-trials| title = How researchers dupe the public with a sneaky practice called "outcome switching" | last = Belluz | first = Julia | date = Dec 29, 2015 |work=Vox|access-date=2018-08-11|archive-url=|archive-date=|url-status=}}</ref> - Vox


'''Tracking outcome switching'''
'''Tracking outcome switching'''


*[http://compare-trials.org/ compare-trials.org]<ref>{{Cite web|url=http://compare-trials.org/|title=Tracking switched outcomes in clinical trials|website=COMPare|language=en-GB|access-date=2018-08-11}}</ref>
*[http://compare-trials.org/ compare-trials.org]<ref>{{Cite web|url=http://compare-trials.org/| title = Tracking switched outcomes in clinical trials|website=COMPare|language=en-GB|access-date=2018-08-11}}</ref>


== PACE trial ==
== PACE trial ==
*In 2016, [[Julie Rehmeyer]] gave the talk [http://www.slideshare.net/JulieRehmeyer/bad-statistics-bad-reporting-bad-impact-on-patients-the-story-of-the-pace-trial?utm_source=slideshow&utm_medium=ssemail&utm_campaign=post_upload_view_cta ''Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE Trial'' Slideshow given at the 2016 Joint Statistics Meetings]<ref>{{Cite web|url=https://www.slideshare.net/JulieRehmeyer/bad-statistics-bad-reporting-bad-impact-on-patients-the-story-of-the-pace-trial|title=Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE trial|last=Rehmeyer|first=Julie|date=Aug 1, 2016|website=www.slideshare.net|publisher=|language=en|via=SlideShare|archive-url=|archive-date=|dead-url=|access-date=2018-08-11|authorlink=Julie Rehmeyer|last2=|first2=|authorlink2=}}</ref>  (Notes may be be viewed by clicking the “Notes” icon just above the comments section in the slideshow link.)
*In 2016, [[Julie Rehmeyer]] gave the talk [http://www.slideshare.net/JulieRehmeyer/bad-statistics-bad-reporting-bad-impact-on-patients-the-story-of-the-pace-trial?utm_source=slideshow&utm_medium=ssemail&utm_campaign=post_upload_view_cta ''Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE Trial'' Slideshow given at the 2016 Joint Statistics Meetings]<ref>{{Cite web|url=https://www.slideshare.net/JulieRehmeyer/bad-statistics-bad-reporting-bad-impact-on-patients-the-story-of-the-pace-trial| title = Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE trial | last = Rehmeyer | first = Julie | date = Aug 1, 2016 | website = slideshare.net|publisher=|language=en|via=SlideShare|archive-url=|archive-date=|url-status=|access-date=2018-08-11 | authorlink = Julie Rehmeyer}}</ref>  (Notes may be be viewed by clicking the “Notes” icon just above the comments section in the slideshow link.)


*[[David Tuller]] spoke about and answered questions on the PACE trial and its flaws.
*[[David Tuller]] spoke about and answered questions on the PACE trial and its flaws.


:* Feb 28, 2016, David Tuller gave a [https://www.youtube.com/watch?v=1fS6Gzc52VI speech with Q&As].<ref>{{Cite web|url=https://www.youtube.com/watch?v=1fS6Gzc52VI|title=Lezing dr David Tuller met Nederlandse ondertiteling|last=Tuller|first=David|date=Mar 28, 2016|website=YouTube|publisher=ME/cvs Vereniging|archive-url=|archive-date=|dead-url=|access-date=|authorlink=David Tuller|last2=|first2=|authorlink2=}}</ref>
:* Feb 28, 2016, David Tuller gave a [https://www.youtube.com/watch?v=1fS6Gzc52VI speech with Q&As].<ref>{{Cite web|url=https://www.youtube.com/watch?v=1fS6Gzc52VI| title = Lezing dr David Tuller met Nederlandse ondertiteling | last = Tuller | first = David | date = Mar 28, 2016 | website = YouTube|publisher=ME/cvs Vereniging|archive-url=|archive-date=|url-status=|access-date= | authorlink = David Tuller}}</ref>
 
:* Feb 27, 2016, [https://www.youtube.com/watch?v=UN1bYhkDHA4&feature=youtu.be An Interview with David Tuller], held at The [[Forgotten Plague]] Conference, discussing the flaws of the [[Pace trial]].<ref>{{Cite web|url=https://www.youtube.com/watch?v=UN1bYhkDHA4&feature=youtu.be| title = Interview with David Tuller (Amsterdam, 27th February 2016) | last = Tuller | first = David | date = Mar 4, 2016 | website = YouTube|publisher=Frank Twisk|archive-url=|archive-date=|url-status=|access-date= | authorlink = David Tuller}}</ref>


:* Feb 27, 2016, [https://www.youtube.com/watch?v=UN1bYhkDHA4&feature=youtu.be An Interview with David Tuller], held at The [[Forgotten Plague]] Conference, discussing the flaws of the [[Pace trial]].<ref>{{Cite web|url=https://www.youtube.com/watch?v=UN1bYhkDHA4&feature=youtu.be|title=Interview with David Tuller (Amsterdam, 27th February 2016)|last=Tuller|first=David|date=Mar 4, 2016|website=YouTube|publisher=Frank Twisk|archive-url=|archive-date=|dead-url=|access-date=|authorlink=David Tuller|last2=|first2=|authorlink2=}}</ref>
*[http://me-pedia.org/wiki/PACE_trial#Release_of_Data PACE trial - Release of Data]
*Geraghty (2017) discussed outcome switching and other [[PACE trial]] flaws.<ref name="Geraghty, 2017">{{Cite journal | last1 = Geraghty | first1 = Keith | authorlink1 = Keith Geraghty| title  = Further Commentary on the PACE Trial: Biased Methods and Unreliable Outcomes| journal = Journal of Health Psychology | volume = 22 | issue = | page = 1209-1216 | date = May 2017 | doi = 10.1177/1359105317714486|url =https://www.researchgate.net/publication/317098481_Further_Commentary_on_the_PACE_Trial_Biased_Methods_and_Unreliable_OutcomesFurther}}</ref>


*[http://me-pedia.org/wiki/PACE_trial#Release_of_Data PACE trial - Release of Data]<ref>{{Cite web|url=https://me-pedia.org/wiki/PACE_trial#Release_of_Data|title=PACE trial - MEpedia|website=me-pedia.org|language=en|access-date=2018-08-11}}</ref>
==SMILE trial==
In the [[SMILE trial]], school attendance was swapped from a primary outcome to a secondary outcome during the trial, with all primary outcomes then based only on questionnaire results, despite the unblinded treatment which encouraged children to believe they would recover and no longer see themselves as ill.<ref name="SmileCorrection">{{Cite journal | last = Brown | first = Nick | authorlink = Nick Brown | date = Jul 2019| title = Editor's Note on Correction to Crawley et al 2018|url=https://adc.bmj.com/content/early/2019/07/11/archdischild-2017-313375ednote.full|journal=Archives of Disease in Childhood|volume=|issue=|pages=|quote=|via=}}</ref> The Archives of Disease in Childhood held an investigation but failed to retract the study, instead publishing an extensive editor's note.<ref name="SmileCorrection"/>


== Learn more ==
== Learn more ==
*[[Open data]]
*[[Open data]]
*[[Open access]]
*[[Open access]]
*[[Peer-review]]
*[[Peer review]]


== See also ==
== See also ==
Line 33: Line 44:


== References ==
== References ==
<references />
{{Reflist}}


[[Category: Clinical trial terminology]]
[[Category:Clinical trial terminology]]
[[Category:Research terminology]]

Latest revision as of 14:10, July 25, 2023

Outcome switching or outcome swapping is when authors of a clinical trial "move the goal posts" during a trial, which may be done to achieve the desired results or may accidentally affect the trial's findings.[citation needed] For example, changing the primary outcome that is used to decide if a treatment is successful from the distance walked in 6 minutes to the percentage of patients who rated their symptoms as "better" or "much better" on a patient questionnaire.

The PACE trial authors, in a clinical trial of ME/CFS patients undergoing GET/CBT, employed this tactic.

Pre-specified outcomes[edit | edit source]

When a clinical trial has a protocol published in advance, it will typically include pre-specified primary outcomes, these are the main measures used to determine if an intervention or drug is effective. Secondary outcomes are considered to be less important. For example, change in average pain level measured with a questionnaire, biological test results or physical measurements of ability.[citation needed]

Outcome swapping is the process of changing these primary outcomes during the trial or prior to publication.[citation needed] For example, a secondary outcome may be changed with a primary outcome, which may give then change the results reported in the trial publication.

In the PACE trial, outcome swapping led to results showing that both CBT and GET were moderately effective, but when a secondary analysis used the original pre-specified outcomes this showed the interventions were less effective than the previously published results.[citation needed]

Articles on Outcome switching in clinical trials[edit | edit source]

Tracking outcome switching

PACE trial[edit | edit source]

  • David Tuller spoke about and answered questions on the PACE trial and its flaws.

SMILE trial[edit | edit source]

In the SMILE trial, school attendance was swapped from a primary outcome to a secondary outcome during the trial, with all primary outcomes then based only on questionnaire results, despite the unblinded treatment which encouraged children to believe they would recover and no longer see themselves as ill.[10] The Archives of Disease in Childhood held an investigation but failed to retract the study, instead publishing an extensive editor's note.[10]

Learn more[edit | edit source]

See also[edit | edit source]

  • Alem Matthees won a Freedom of Information Act tribunal for the release of the PACE trial data.

References[edit | edit source]

  1. Lane, Christopher (April 30, 2016). "How Outcome Switching is Corrupting Medical Research". Psychology Today. Retrieved August 11, 2018.
  2. "For my next trick..." The Economist. March 26, 2016. Retrieved August 11, 2018.
  3. Miseta, Ed (March 7, 2016). "Is Outcome Switching Still A Problem In Clinical Trials". clinicalleader.com. Retrieved August 11, 2018.
  4. Belluz, Julia (December 29, 2015). "How researchers dupe the public with a sneaky practice called "outcome switching"". Vox. Retrieved August 11, 2018.
  5. "Tracking switched outcomes in clinical trials". COMPare. Retrieved August 11, 2018.
  6. Rehmeyer, Julie (August 1, 2016). "Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE trial". slideshare.net. Retrieved August 11, 2018 – via SlideShare.
  7. Tuller, David (March 28, 2016). "Lezing dr David Tuller met Nederlandse ondertiteling". YouTube. ME/cvs Vereniging.
  8. Tuller, David (March 4, 2016). "Interview with David Tuller (Amsterdam, 27th February 2016)". YouTube. Frank Twisk.
  9. Geraghty, Keith (May 2017). "Further Commentary on the PACE Trial: Biased Methods and Unreliable Outcomes". Journal of Health Psychology. 22 (9): 1209-1216. doi:10.1177/1359105317714486.
  10. 10.0 10.1 Brown, Nick (July 2019). "Editor's Note on Correction to Crawley et al 2018". Archives of Disease in Childhood.