MEpedia talk:Science guidelines

Old ideas

Some possible ideas to consider:

1) Quality of references: should be peer-reviewed journal articles or reputable textbooks

2) In the absence of good research, anecdotes can be cited under certain conditions (what conditions?) so long as in the text of the article, the information is not passed off as an established fact and the source of the information is clear. For example, "some clinicians have observed...." or "many patients have reported...." There still needs to be an external citation.

3) Avoid where possible citing health blogs or other third party sources for information that is considered to be scientific fact, especially where a first party source (i.e., a journal article) is available. Blogs, forums, etc. *can* on a case by case basis be reasonable sources for anecdotal or observational information.

4) Where available, always report both negative and positive results. If you do a search for, say, muscle biopses in ME patients, report both the studies that found evidence of abnormalities and studies that did not.

5) Where possible, report information about the design of a specific study or series of studies if they help the reader to grasp the quality and reliability of the findings. For example you might point out features of the design that decrease our confidence in the outcome: small sample sizes, lack of controls, open label. You might also point out information about the design that increase our confidence: randomized, double blind, large N, proper controls, finding replicated. This does not need to be done in great detail but you might say, "Several small n studies have found...." Or "A large, randomized trial found......"

--Meaction (talk) 13:41, 21 November 2015 (PST)

content hidden

Volunteer Slack? Under the "Less is more" subsection, it says "go to the Volunteer Slack". What is a "Volunteer Slack"??? Pyrrhus (talk) 21:31, 14 March 2019 (EDT) Pyrrhus, Slack is a technology that's basically a very fancy chatroom service; you can make a Slack for a specific group and then it offers the possibility of having different, easily searchable threads for different topics. I take it there is an MEpedia Slack, although I'm not part of it and don't know the Slack's name or who to ask for an invite, so eventually we should probably try to find out how users who wish to can join (personally I don't love using it; I find it overstimulating and would rather discuss with editors here, but it's good to offer different options as different people will undoubtedly have a variety of different preferences and accessibility requirements.) Meanwhile if you want, you can check out the tech at Slack.com though; the basic service is free, you only pay for upgrades (like storage if your group has more than 10,000 messages, etc.) Poking around, I do see an MEAction Volunteers Slack (meaction.slack.com) and you can sign into it if you have an meaction.net email address. But I don't know if MEpedia has a separate one or if MEpedia is a "channel" in that one; also don't know if you have to get an meaction.net email or if possibly existing members (or especially, moderators) can add send invites to others to join the Slack, if that is the right one. So many questions! Sorry I'm probably giving more questions than answers here! Canele (talk) 19:29, 19 March 2019 (EDT) Thank you so much Canele! That is very very informative. I prefer discussing things here on MEpedia too, but we desperately need to enable communication between contributors any way we can. I'll ask around to see how I could join. Now, whom do I ask? That would be a good question for the Slack group! ;) Pyrrhus (talk) 21:12, 19 March 2019 (EDT) The best bet as far as I know would be User:JenB, or maybe she can suggest who best to go to with MEpedia questions if it's someone else? (I'd also love to ask about either adding to the CAPTCHA whitelist or getting set up to take of that myself, which I'd be happy to do, just don't know who best to ask for the necessary permissions.) Thanks for any guidance! Canele (talk) 22:13, 19 March 2019 (EDT) I’m now in the Slack group, so if there’s anything you want me to bring up, let me know. The CAPTCHA whitelist has already been brought up. Is the only thing needed there is admin privs to edit the file? Or is the CAPTCHA whitelist more complicated than that? Pyrrhus (talk) 16:02, 29 March 2019 (EDT) Oh, thanks so much Pyrrhus. Yeah as best I can tell, it's just admin permissions that's required to edit that page and then appropriate URLs only need be pasted in. As Njt has pointed out, the best would be if we could set up a user right group that automatically turned off the CAPTCHA for trusted users (discussed further here: MediaWiki_talk:Captcha-addurl-whitelist#Regex_for_Whitelist), but setting that up is beyond my personal technological capacities, so until someone who is capable has the time to take up that project, I think granting someone who is around more (you, me, whomever) permission to add to the white list would go a long way to improving accessibility. (My personal opinion would be that whomever is delegated that authority should only whitelist the most rock-solid science and reference sites. Other kinds of sites are important to include on MEpedia but I've seen a fair number of places where they're not used in accordance with MEpedia:Science guidelines and I think it maybe wise to leave the speed bump up at least for now.) Canele (talk) 19:01, 29 March 2019 (EDT) User:Canele If you wanted to join Slack's MEpedia group you can email the info meaction.net email address, on contact User:JaimeS. Some discussion goes on in the Facebook group (linked to from the MEAction website) but not much. I have recently discovered that auto confirm by email address may remove the Captcha, fixing the email feature has been flagged up since there's a new issue with it. notjusttired (talk) 18:42, 29 March 2019 (EDT) Thanks Njt for the head's up about how to join the Slack. I may do so later; I find the tech a little overstimulating so I like discussing here where things don't move quite as rapidly. Meanwhile though, that sure would be great if eventually there were an automatic user right to disable the CAPTCHA. I suppose one thing I would say is, my ideal world would disable the CAPTCHA after a user has made a certain number of manual edits (500?) rather than just after confirming their email address, because it's so easy for spammers to dummy up an email address, confirm it, and then go to town. On a site about science, I do worry about manipulation by folks with a financial interest. But, we'll wait to see what's possible, I suppose? Canele (talk) 19:01, 29 March 2019 (EDT) That's a good suggestion on number of edits - could be put in the Dev queue. I would set it much lower - say 50 edits - the Captcha is easy to lookup for people so I presume it's mostly there to stop robots / automated edits. With automated edits I'm not sure how easy it is for robots to do the email confirm or if any have tried. Njt notjusttired (talk) 17:14, 30 March 2019 (EDT) Good idea, I'll add it to the dev queue! You're probably right that 50 would do the job for now. I know Wikipedia deals with folks that for-profit, not-necessarily-reliable sites are paying to go in and manually add many links to that site; I was thinking the CAPTCHA would be at least a speed bump for that. But no real reason to have it be so high until/unless that becomes an issue here. I assume once that feature is in place, it wouldn't be so hard to adjust the specific number. Canele (talk) 18:21, 30 March 2019 (EDT) Rename to MEpedia:Science guidelines Any objection if this page "Science Guidelines" is renamed to "MEpedia:Science guidelines"? Since this page is about the MEpedia project itself, it really should belong in the MEpedia namespace. ("MEpedia:") Pyrrhus (talk) 20:43, 18 March 2019 (EDT) I'm for it! (Again!) Canele (talk) 19:30, 19 March 2019 (EDT) Done! Pyrrhus (talk) 16:04, 29 March 2019 (EDT)

Deleting "When should I cite?"

Any objections to deleting the "When should I cite?" instruction? I feel like it's already (and more accurately) covered in the preceding "If you can't cite it, you can't say it" instruction. Using Citing References in Scientific Research Papers makes sense for those doing primary research since original arguments/findings don't need to be cited--but here (I think) we want everything added to be referenced so that readers should be able to go check to see what verifies a given claim, rather than having to trust whichever random editor who made the change was actually correct. Right?
Thoughts? (Sorry, tired, this might not be clear.) Canele (talk) 00:38, 11 April 2019 (EDT)

I’m not sure I understand, but I’ll tag User:JaimeS because she’s the Science Editor and she can give a definitive yes or no. As a general rule, I think we want to drill into people that everything needs to cited, even if we repeat our selves. Hope this helps.

Pyrrhus (talk) 12:02, 11 April 2019 (EDT)

Let me take another, more caffeinated crack at explaining (grin)! I'm worried that by specifying we must cite everything we get from other sources, Citing References... inadvertently implies some things don't need to be cited. Which is true for people publishing research or review studies--you don't need to cite ideas original to you--but MEpedia isn't (as I understand it) the place for folks to publish their own novel hypotheses: only things they can cite to another source.

The other thing is that as UI/UX matter, the longer instructions get, the less likely folks are to read any of them. So repeating may have the opposite of the desired effect, unfortunately. Canele (talk) 13:31, 11 April 2019 (EDT)

Generally speaking, the rule of thumb is not to cite things that are considered common knowledge. Therefore, we would not have to cite every sentence of an article, or even every complete thought. I'll check the original article, though, with an eye towards this kind of misinterpretation... ok, I read through. I think that section contains important information regarding that you must cite not just direct quotes, but anyplace where you get an idea, even if you've paraphrased it. You'd be surprised how many people think you don't have to cite unless you've directly quoted someone's work! I'm ok with folding that in to another section if you think it's confusing.JaimeS

Ahh ok, thanks JaimeS, now I understand what it's intended to convey and def agree makes sense to hit the "cite whether it's paraphrased or quoted" point. I'll see if I can make it sit with the "cite everything [unless common knowledge]"--maybe we can use the quote from that article in a footnote, for sake of concision/hitting the high notes as clearly as possible. Canele (talk) 19:30, 11 April 2019 (EDT)

Great, Canele! Thanks for that disambiguation, good looking out. JaimeS

RE: "Common knowledge" - I think this should be made clear that it's general knowledge or common knowledge but not what an editor might consider common knowledge about ME/CFS. Many consider "always caused by a virus" as common knowledge for instance, or consider ME or ME/CFS as a "fatigue" condition. I may consider starting a page on Facts vs Myths to cover some of these. notjusttired (talk) 07:17, 15 April 2019 (EDT)

I did also think it might be good to give an example of common knowledge that it's acceptable not to cite, to help people gauge, but I had a hard time coming up with a good one. "Humans generally have two kidneys"? Better suggestions welcome! Canele (talk) 14:43, 15 April 2019 (EDT)

I actually really like the following quotation for citations: "Common knowledge... is widely known, undisputed and easily verified, and it generally cannot be attributed to a specific person or paper". It's a really accurate definition that would stop someone from saying "ME is caused by a virus" because they know it is, in fact, disputed, not easily verified, and is not widely known. Here is where that quote came from, and it may be a good source in general, because it also has examples of common knowledge and questions to ask oneself before we can tell whether something is considered common knowledge: https://www.scribbr.com/plagiarism/common-knowledge/. I think we should define common knowledge as per above, and then make the explicit comment that you can find examples of common knowledge and guidance as to what constitutes common knowledge by clicking here (with link to above).JaimeS

+1 to adding a definition of Common Knowledge to the text. The definition quoted by JamieS on this talk thread was helpful in clarifying. EscapeTheFog (talk) 01:37, June 13, 2019 (EDT)

Has this issue been resolved? Do we need to update anything on the page?

Pyrrhus (talk) 13:27, October 24, 2019 (EDT)

Honorifics

I'm wondering about establishing a policy to standardize use--or not--of titles and honorifics (Dr., Professor, etc.) across MEpedia. I realize this is a challenge given the diversity of usage in different languages, fields, etc. Wikipedia deals with that by never using honorifics and only naming specific degrees and positions held (and only doing so the first time the degree or job is mentioned), and otherwise last name only (after the first use). But, I do see that for most readers of MEpedia, there's likely often value in having someone's title included each time that person is mentioned ("In 2018, Dr. So-and-so wrote a second editorial criticizing...") At the same time, I worry it can be confusing rather than clarifying when people who are MDs, PhDs, and DCs all have the same "Dr" in front of their names--it goes without saying that these are really different qualifications and I'm concerned about misleading readers about what a given person's qualifications are. That's compounded right now by variation within the project: some editors write "Dr. Klimas" every time, others "Dr. Nancy Klimas" once and then "Klimas" subsequently (or some other variation).
User:JaimeS, do you or others have thoughts on a rule for handling this to best communicate/contextualize scientific information for a broad audience? (In giving the Klimas example, I'm also realizing setting a consistent standard could also help us reduce unconscious bias.) Canele (talk) 19:32, 13 April 2019 (EDT)

I'm okay with having this be part of our general formatting. MEpedia 'really' has about 10 regular editors with a few people sweeping in once in awhile, so this will go a long way. However, it may be too much to ask or expect first-time users or someone going into one article to keep this in mind. We will either have lots of corrections to make or we'll have to let it slide sometimes. That said, if we choose a format, then the person's title or degree should be described in detail the first time in any given article (Dr Jane Doe, a professor of Immunology at the University of Y", and not again thereafter -- just using Doe or Jane Doe from then on. Jaime

Full title and specified credentials the first time, then only last name thereafter seems like a good rule to me. And yes agreed: for so many of these things, I don't expect new users to know them off the bat and it's really not a big deal if they mess them up. If anything honestly tidying that sort of thing (what WP calls "wikignoming") makes for a good task on foggy days. Just wanted to decide first what we'd like it to be. But that sounds great, I'll add it (as succinctly as possible). Canele (talk) 14:04, 16 April 2019 (EDT)

I like that phrase: “wikignoming”. That’s my word of the day!

Pyrrhus (talk) 19:16, 16 April 2019 (EDT)

Ah, so glad I could share! The wikignome is a venerated figure on WP. We could incorporate the term more here--really helps emphasize how much value many different types of edits add to the project. Canele (talk) 01:26, 17 April 2019 (EDT)

It's just occurred to me that using titles regularly can be problematic because they are changeable. Eg Dr Simon Wessely became Prof, then Sir. When referring to past work or doesn't seem quite right to consistently refer to someone's early work using their current title somehow. I generally use full name in the first instance on a page, and either last name or both names after - except with bio pages where title is relevant. Several pages will refer to several authors with the same surname too. This isn't a big deal to me. I see this as more a Manual of style or editorial thing since it's not about scientific understanding.notjusttired (talk) 16:44, 22 April 2019 (EDT)

Types of studies

This on in vivo etc is very long and not important. Hardly any research on ME/CFS uses these terms, and if they do they don't make them prominent.

I would prefer to remove this information, or to replace it with Clinical trials information - eg A case study is not a clinical trial, clinical trials will have a trial registration number, difference between Review articles, Editorials or Theories, and Research (eg research involves specific methods and results). notjusttired (talk) 11:01, 14 April 2019 (EDT)

Good idea! We could also make MEpedia:In vivo studies and MEpedia:In vitro studies--or just mainspace entries with those topics (honestly we should def do at least the latter, I will see about starting that today)--to offer here as "See also" but yes, I agree it's not among the main issues of science literacy for editing MEpedia well, and we could trim either to make the whole more concise/readable or to make space for other subjects. Canele (talk) 14:19, 14 April 2019 (EDT)

I started the mainspace pages (In vivo studies and In vitro studies) but def, I can see the usefulness of also having project space pages in case anyone goes seeking more specific guidance on how MEpedia wants these distinctions described. So we could move the info over there rather than delete it. Njt, I'm sure if you have ideas about what should be included instead, they'd be welcome--you could post a mock-up here for discussion? (If you are up for it, no worries if not.) Canele (talk) 16:43, 14 April 2019 (EDT)

That sounds good, although the way it's written suggests more suitably for studies involving tissue samples and animal testing - there's no animal model and I am unsure if there's is much that can be tested with tissue samples given the multi-systematic nature and the risk of trying to get samples from the key areas (brain and spinal column, just a little dangerous!).

What would be useful would be some of this glossary - I was looking earlier at Phase 1 to Phase 3 descriptions.

Also to keep in mind is not to explain very short / simple things - I've requested debtors add the Lingo extension - it creates a glossary page and links it to anywhere the term is found on any page - so much could be added there rather than separately in Science Guidelines.

My 2 big issues with science guidelines are to get editors not to cite Wikipedia or use ME-pedia pages as citations (they should be links), and to get across that science writers publishing only online are often poor sources for science (fine for personal experiences or Articles to link to) - some terrible "science" out there written by a single professional and either not peer reviewed or almost no improvements made. Journals should be the main source in my view given the lack of textbooks on ME/CFS. Science Direct is also excellent to search for book chapters or topics not specific to ME/CFS. Some clinics have good articles but others are just terrible. notjusttired (talk) 17:17, 14 April 2019 (EDT)

To your first point, I actually think for instance the Bhupesh Prusty talk at NIH last week specifically discussed ME/CFS research that does isolate certain kinds of material, and the difference that can make in the findings (see this Twitter thread), but I totally agree with you that that is far more advanced science than the main issues the Science guidelines need to convey to improve the scientific value of MEpedia.
Coming over from Wikipedia, I am accustomed to much, much higher standards for what can be included, but I understand where the mission statement is coming from about wanting to gather more than what's verified by review articles. To me the big challenge is getting people specify the quality of the source they're using. Like you say: is it one doctor's opinion? Is it some patients on a forum? Is it a peer-reviewed study? If it is, did have three patients or 300? Etc.
That glossary is a really good idea. I don't know how the extension you mention works but I'm also happy to start mainspace entries for some of the most common concepts. For all of this, I'm not remotely qualified to write a textbook-level explanation, but I hope if I can get things started, it's easier for folks who do have the scientific expertise to add on, amend, specify. Canele (talk) 20:31, 14 April 2019 (EDT)

Everything in that page would be cited anyway - e.g. from a medical dictionary, or a book chapter - science direct's website is excellent for finding book chapters. The page name defaults to Terminology and it needs particular syntax.

The huge issue with sources is that there aren't any textbooks on ME/CFS. Not one. All books are based on author's own views or research. Although editors really should be stating in the text where info comes from for single sources, e.g. Jason and Sunquist (2009) stated that... I also disagree with you on Wikipedia standards: the editorial / scientific standards mean anything even remotely controversial can be (intentionally) skewed by editors using extremely poor quality sources which count as tiertary sources because opinions and reviews are given greater weight than objective science - some tierary sources barely have any references. Skeptics (actually disbelievers) rule on Wikipedia and there is a bullying culture in evidence on some pages - all editors do not have equal rights or equal influence. Scientists get marginalized since qualifications don't count. The demographics of Wikipedia editors are not representative of society in general to say the least. Talk pages on controversial topics consist of unfounded accusations of using sock puppets (happened to Angela Kennedy), and I've seen editors regularly move content they don't like to talk pages to tie up other editor's time so it won't be included. We don't have edit wars here. We go for facts, not opinions. If a popular opinion has very weak evidence then repeating it often gives it more weight on Wikipedia. Craziness! Also some editors mis-cite sources - what looks like a good reference turns out not to say that at all. Wessely and Sharpe do the same in their own articles.

At the moment I think many editors just aren't seeing crucial pages like the Science guidelines or manual of style. I read all the welcome message and How to contribute when I first signed up, but only when an editor pointed out the Manual of style did I find it.

Editing massage: I'd like to change the message editors see before submitting to include links to Editorial guidelines, Science guidelines, Manual of style, and remind them that Wikipedia and blogs are not science sources, and that "if you can't cite it you can't say it". notjusttired (talk) 07:13, 15 April 2019 (EDT)

Equivocal versus neutral language

Wanted to seek more input about section on equivocal language. It seems to me that it's worth keeping "equivocal" in the title rather than revising to "neutral" because while editors should definitely describe findings neutrally, that's covered in the editorial guidelines whereas this is--it seems to me--making a different point about not overstating how definitive a finding might be. My concern for using "neutral" is that to non-scientists, saying something has been proven can seem entirely neutral ("neutral" to me mostly indicates you shouldn't say "shockingly, it was proven that...")
On the other hand, is there a risk that telling people always to equivocate could result in people mischaracterizing results? It's obviously not appropriate to say a finding is equivocal if it's just not. Would welcome input on both best practices in science writing as well as how we can be best convey them to folks not yet familiar with those conventions! Canele (talk) 16:26, 20 April 2019 (EDT)

Good point. I changed the wording only to try to simplify the language (improving readability). I'm open to other suggestions - or possibly examples? I feel the language used should be closer to "plain English" - short, simple, clear. I admit I had to lookup unequivocal and didn't make a great choice for an alternative word notjusttired (talk) 16:38, 22 April 2019 (EDT)

I put it in as equivocal very deliberately. Let's keep that word. Jaime

Important studies and works of age

What about listing these in a bullet points? Can we shorten the explanation to maybe 2 sentences. Also Holmes criteria introduced the term CFS - although somehow Fukuda criteria took over so this example needs changing. Key works in my view are:

• Fukuda criteria
• Canadian Consensus Criteria which are for ME/CFS
• International Consensus Criteria for ME
• PACE trial
• Wilshire et al analysis of PACE
• Ramsay definition
• McEvedy and Beard (1970) - mass hysteria
• Not sure what's best for biomedical studies? 2 Day exercise one?
• York Review or Cochrane unretracted exercise review - that brought in CBT & GET evidence or Original BPS trials eg Fulcher & White 1997

Do not give advice

I have added this to clarify. It is especially relevant when describing treatment protocols or dose suggestions for medications. I added similar to the scientific guidelines. User:JaimeS User:JenB User:Kmdenmark User:Pyrrhus User:Hip User:DxCFS User:MEandCFS notjusttired (talk) 09:56, 3 May 2019 (EDT)

This would belong in the science guidelines, not the editorial guidelines. I'll copy this discussion over there.

Pyrrhus (talk) 14:28, 3 May 2019 (EDT)

I agree wholeheartedly not to give medical advice. I feel uncomfortable with the pages that do. It can inadvertently cause harm. Kmdenmark (talk) 16:01, 11 May 2019 (EDT)

"blocks of information" -- EscapeTheFog (talk) 01:27, June 13, 2019 (EDT)

I'm confused by the following "Do your best to avoid ... paraphrasing blocks of information". Would the correct interpretation of this be "it's okay to paraphrase a few sentences, but avoid paraphrasing large blocks of text. Instead, prefer to summarize the key points, and link to the source for readers who want details."?

Paging User:JaimeS... :)

Pyrrhus (talk) 20:21, June 14, 2019 (EDT)

Re: "blocks of information" -- JaimeS (talk) 14:11, June 26, 2019 (EDT)

I wouldn't recommend paraphrasing whole paragraphs (multiple sentences in a row). When people paraphrase chunks of text, it's often an uncritical repetition, rather than an integration of ideas. You can see more on this here: https://writing.wisc.edu/handbook/assignments/quotingsources/. Scrolling down to 'How to Paraphrase a Source'.

"When reading a passage, try first to understand it as a whole, rather than pausing to write down specific ideas or phrases... choose and summarize the material that helps you make a point...

"METHODS OF PARAPHRASING

"Look away from the source then write.

"Read the text you want to paraphrase several times until you feel that you understand it and can use your own words to restate it to someone else. Then, look away from the original and rewrite the text in your own words.

"....If you find that you can’t do A or B, this may mean that you don’t understand the passage completely..."

This may seem more stylistic than factual, and it is. But as an instructor, I often found that people who paraphrased idea-by-idea for a paragraph or more often didn't understand the text well enough to integrate the idea into the rest. That resulted in work of lower quality.

Has this issue been resolved? Do we need to update anything in the page?

Pyrrhus (talk) 13:27, October 24, 2019 (EDT)

"if you can't cite it, you can't say it"

I'd like clarification on the spirit of this rule. Should it be interpreted as

• "Do not add text, unless it is either common knowledge or *immediately* supported with a citation." OR
• "Do not add text, unless it is either common knowledge or *possible* to cite it."

FWIW, I would hope it's the latter. While it may not be optimal, I feel like there is value in adding information that the author is certain is in the literature, but does not yet have the citation handy. Often getting an article started is the hardest part, and we facilitate that if we allow some flexibility on citations. Authors can return to add citations later, and other people can help to add missing citations once there is a skeleton of text in place. To support this, I would love if we also had a "citation needed" template like wikipedia.

-- EscapeTheFog (talk) 01:45, June 13, 2019 (EDT)

The second definition also supports the spirit of having various Roles. Some people are good at distilling down a pile of information to a summary, some people are good at the precision work of citing all key information, and some are good at both. Some people may be good at both, but requiring both at the same time increases friction and may reduce contributions. -- EscapeTheFog (talk) 01:48, June 13, 2019 (EDT)

Hi EscapeTheFog! Any questions about the guidelines should probably be directed at User:JaimeS (note the spelling) She can be hard to get a hold of, but linking to her (nonexistent) user page should trigger an alert for her. (per MEpedia:Discussion)

For more information on template tags such as "Citation needed", see the Tutorial. Hope this helps.

Pyrrhus (talk) 20:17, June 14, 2019 (EDT)

Re: "if you can't cite it, you can't say it" -- JaimeS (talk) 14:13, June 26, 2019 (EDT)

The former! :)

Definitively, we often think we know something is true, especially in a field like ME, where we may have heard it repeated often. However, you hunt down the source and you find it's been misquoted/misinterpreted/doesn't exist... OFTEN.

Moreover, sometimes you find it's just something people ~say~ that has a DUBIOUS source that you'd only know by finding said source.

Sources to avoid

I'm a bit concerned because this contradicts advice we give elsewhere, for instance the outlines that specifically asking for the Wikipedia page, or, lower down on this page, explaining that it's a good source of sources, just not a good thing to cite directly. Additionally, I'm not sure I agree with the idea a doctor's website should never be cite: to me the more important thing (similar to the Wikipedia issue) is making sure the editor understands and documents for the reader what exactly a source can and cannot be considered reliable for. A doctor's website is probably pretty reliable for documenting that doctor's opinion, which may be of interest to the community even if it differs from consensus (or more often, if there is no consensus). The important thing is that the prose make clear when something is individual opinion and not consensus. So I'm wondering if these warnings could be converted into more "advice for handling sources" rather than telling people they can't use them at all. Canele (talk) 21:34, July 13, 2019 (EDT)

I was hoping for some good feedback on this. Thank you.

It might help to look this old version for what I'm trying to avoid - especially sources 1, 3, and 9 (the last is a doctor), look at the mentions of Lyme and symptoms list.

Wikipedia is best removed from the Article outlines in my view. A link to webmd, merckx msnuals or scienxedirect's search results would be much better. But linking to it wikipedia learn more is not the same as citing it. The MEpedia:Editorial_guidelines already point out Wikipedia is not reliable so I think we are being consistent. We could change the wording of the new section to suggest reading the References section on Wikipedia only for suggested sources, but I think linking to Wikipedia as a source is not OK. I've just been trying to sort out another page that actually linked to a numbered Wikipedia reference rather than the reference itself. Bear in mind also that some of what we cover are "contested" diagnoses, and Wikipedians are not representative of the general population - they are prominently young, male, single, childless (none of which are bad things but limited life experience), lacking in a biological or health background and in my experience highly skeptical and prone to promote minority opinions, regardless of new facts.

I'm finding doctors websites frequently used for key information that should be taken from elsewhere - or containing various misunderstandings - like symptoms of a poorly understood condition - which actually don't match the proposed diagnostic symptoms and probably cover half the population - and for unproven benefits of various supplements. So many doctors use "Chronic fatigue" and "Chronic fatigue syndrome" interchangeably, others claim to have found a cure (and they will sell you it). Dr Axe has "4 ways to overcome CFS" - with a few references that he massively overstates - treatments we certainly don't to claim are effective based on that. The single source he uses is a normal article suggesting that those treatments should be investigated further - not the same as Dr Axe claiming they are the "most promising treatments" for CFS. This doctor claims CFS is an autoimmune gut condition, and only 8 months of his non-prescription pills will treat it - that's not something we should cite on the Autoimmunity hypothesis page which is one of the harder places to get sources (OMF research is looking into autoimmunity at the moment). Any doctor's website with no sources on the article shouldn't be used (unless perhaps it's a well known expert in the field or they are cited on a page about the treatment they themselves have invented eg Dr Gupta on the Gupta programme page). Doctor's websites often promote whatever their niche treatments are - eg the "leaky gut" doctors will list 30 symptoms and that's their opinion - based on no actual studies at all, but gets cited as if it's fact. There's also a tiny number of atrocitious websites with a single author and single reviewer that just make things up (patient.co.uk actually invented totally false information a few years ago - didn't even get the classification or symptoms right - this has happened on several pages there.) There's also many references to "verywellhealth" that are actually just blogs by people with the illness but get cited as medical facts. So, that's what I'm trying to address. There's a real avoidance of peer reviewed journals too, in favor of unreferenced websites. I'm certainly open to suggestion. :YouTube videos might be something else that shouldn't be cited as a reference (unless it's explaining a clinical trial, treatment or clearly described as an opinion). Linking to it is fine, or embedding on the page to support referenced facts. notjusttired (talk) 15:22, July 14, 2019 (EDT)

I think it might be helpful to us to start a policy whereby any changes to the guidelines are discussed before they are implemented. These are really good ideas being raised, it would be nice to get them right. Just my 2 cents!

Pyrrhus (talk) 00:31, July 15, 2019 (EDT)

Re: Sources to avoid -- JaimeS (talk) 11:42, July 15, 2019 (EDT)

Hmm.

Rather than saying we can never use a doctor's website, I think it's more appropriate to say that PRIMARY sources are always, always preferable to SECONDARY sources. If a doctor cites a statistic and doesn't say where it's from, and you can't find the PRIMARY source, you shouldn't cite it. In fact, I'm surprised that this adjuration doesn't already exist somewhere. It's kind of vital to citation prioritization! The same goes for Wikipedia: it's a good place to hunt down PRIMARY sources. One wouldn't cite Wikipedia itself, and that should have been clear in the guidelines' original incarnation.

We can also clarify the citation of facts versus opinions or theories. For example, let's say someone wants to reference Myhill's theories. She hasn't published often, but she's influenced a great number of patients' ideas about the disease, and treated a great number of patients; and anyway, we'd be citing her work as theory, not fact.

"Many researchers have found mitochondrial dysfunction" <-- cites Myhill's site [NOPE]

"Some clinicians, such as Dr Myhill, have treated people with ME with supplements geared to augment mitochondrial dysfunction" <-- cites Myhill's site [YES]

I do agree we should talk about any changes on policy pages before implementing them.

I should have discussed here first. My apologizes. Dr Myhill may be very popular but she has had complaints to the GMC and been forced to change her website to a degree as a result, not so much about ME but her anti-vaccine statements. What about changing the doctor one to:

"Information from a doctor's website must be verified for accuracy (read the original sources used), or presented as a minority opinion only, and not used for the introduction or symptom lists"

Re: Types of sources. Primary / secondary / tiertary are confusing teens to many. Wikipedia relies on tertiary eg systematic reviews like Cochrane or encyclopedia. We have very few and they are typically biased in ME or CFS Research. We actually prefer primary (eg peer reviewed articles) and secondary (Consensus statements, guidelines, etc). I find many charity websites to be very accurate, if articles are referenced. I think it would be better to talk about verifiable sources and academic sources. Perhaps with unreferenced articles from doctors/physios/etc considered one of the lowest quality of sources, only to cited as Dr X claims / believes / views that ...

I think it would be a positive to have an outright ban on using perhaps up to 5 extremely low quality websites that rank highly, including Wikipedia, but allow them under "learn more" or "articles and blogs" links since many explain patient experiences or different minority but valuable views. A few websites could be included due to being inaccessible in outside the country of origin - User:Pyrrhus has raised this before.

Feedback would also be welcome on proposed changes to the Cleanup template including links to scientific guidelines and being able to pick a default message to add to. notjusttired (talk) 12:10, July 15, 2019 (EDT)

Sources to be careful with

• Unreferenced articles: If the article you are referencing does not list its own references at the end of the article, it’s probably not a reliable scientific reference.
• Blogs may be useful references when talking about the blog author or advocacy in general, but blogs should never be used as a reference for scientific evidence, as they are not peer-reviewed.
• A doctor's website may be a good source when talking about the doctor or the doctor’s ideas, but a doctor's website should not be used as a reference for scientific evidence. Doctor websites often reflect the doctor's opinion rather than an expert consensus.
• Wikipedia or other wikis may be good to list under the “Learn more” section of an article, but they should not be used as a reliable reference. You may, however, find a reliable reference in the “References” section of a Wikipedia article.
• Generally speaking, be aware that sources that are not published in a peer reviewed journal or an edited book may reflect only the authors' opinion.
• And remember that it’s always best to hunt down the original source of an idea you are citing. If your source merely cites a different source for that idea, it is best to use the original (“primary”) source instead of a copycat (“secondary”) source. However, secondary sources are fine if they are published review articles or consensus statements.

Feel free to edit what's above. I left "Sources best avoided" with just "Unreferenced articles" below it in the science guidelines notjusttired (talk) 10:09, July 17, 2019 (EDT)

I have made my changes. I changed “Sources to avoid” to “Sources to be careful with” to allow for nuance. It may be a good idea to substitute “Wikipedia” in the “Learn more” sections of the article outlines with “Outside articles: e.g. Wikipedia, WebMD, Merck Manual, or ScienceDirect”. I haven’t really come across any attempts to use a YouTube video as a scientific reference, so I’m not sure it needs a mention. The only reference we had to remove because it was only accessible in the UK was the British Prescription Drug Formulary.

That’s what I’ve got. Any changes you would like to make User:JaimeS User:Canele?

Pyrrhus (talk) 00:40, July 20, 2019 (EDT)

JaimeS I like the more detailed work above. -- JaimeS (talk) 16:14, September 16, 2019 (EDT)===

Any last changes you would like to make, User:Canele?

Pyrrhus (talk) 17:39, September 19, 2019 (EDT)

"Animal model" of CFS claims - because these are based on a single symptom of short term exhaustion rather than a long term interest. They appear to be investigating fatigue - usually rats exhausted from swimming - not PEM, not a chronic illness. We should really take Wikipedia out of the article outlines. Njt (talk) 18:12, September 19, 2019 (EDT)

As there are no more changes, I will copy the above text to the Science guidelines.

Pyrrhus (talk) 13:27, October 24, 2019 (EDT)

Scientific consensus

I suggest adding this heading and stating typical places where this can be found, eg IACFS Primer, CCC, ICC, IOM report. notjusttired (talk) 15:51, August 8, 2019 (EDT)

The term has different meanings: outcome or method. I either case: not feasible. Guido den Broeder (talk) 17:05, September 16, 2019 (EDT)

Contested evidence

• How do we report on research on a researcher's page when there is substantial evidence or the conclusions from a piece of research are contested? Eg X does a clinical trial saying the Lightning Process is effective, but the Lightning Process page (and scientific consensus) says the opposite. This goes to neutral POV (point of view).
• 1. If most of the researcher's evidence is contested do we use maybe a single major source to refute each, or put it all in a Criticisms section instead? Or do both? I'm uncomfortable writing things like "and X reported that medically unexplained symptoms (MUS) including CFS..." when CFS isn't categorized as MUS - that's a minority opinion. Is it better to put X regards CFS as medically unexplained symptoms (MUS), and X's study on antidepressants for MUS found... - Would mentioning limitations reported by the study (if it does), or by others be helpful? Eg use of broad Oxford criteria, drop out rates, being very specific about severity or how many improved.

2. Bias: Wikipedia has a list of words to watch that suggest biased writing, these might be best in the editorial guidelines, but I think it useful to link to them here. Eg claim - "X claimed that..." is considered likely bias.

• 3. Experts- Wikipedia gives the same weight to all regardless of qualifications / Special interests (I personally hate this - experts should not always have authority but they are experts!) - what's our view?
• 4. Neutral point of view - Esther Crawley's page is one of our top 10, it's not neutral in my view. What guidelines can help with this?
• 5. A few pages have a list of articles on a particular point of view (list of articles explaining CBT not approx got CFS) - do we insist that an opposite list must also exist, and what about scientific consensus, if it supports the view that the list has do we still need the opposite?
• 6. I find it very hard to work out if some topics are scientific or editorial guidelines. notjusttired (talk) 15:51, August 8, 2019 (EDT)

Neutrality does not mean a balance between postive and negative. If someone is an angelically good or a notoriously bad researcher, than the article should reflect that. Guido den Broeder (talk) 17:14, September 16, 2019 (EDT)

Minor suggestions from last update

• Objective outcomes more reliable than subjective outcomes (e.g. step counters vs questionnaires)
• RCTs: objective outcomes vs subjective to mention? How to compare double blind with subjective outcomes better than unblinded with objective outcomes?
• Systematic reviews are often given more weight elsewhere than RCTs, do we have a policy on this given they major issues with certain systematic reviews eg the York Review used by the NHS, and Cochrane's
• Consider linking to Category:Research terminology (although this does need work)

• Is it worth pointing out that close attention should be paid to the diagnostic criteria used, eg CCC selects more severely ill patients and less patients than Fukuda CFS criteria or SEID criteria, Fukuda more than Oxford

• Trials with pre-specified outcomes given more weight except when outcome switching occurs?
• Possibly link to Research bias in ME/CFS or the research terminology category. notjusttired (talk) 15:52, September 16, 2019 (EDT)

With the exception of selection criteria (encompassing more than just diagnostic criteria), which should be mentioned: you can't capture everything with rules. Experienced editors will weigh the evidence on a case-by-case basis. Guido den Broeder (talk) 17:02, September 16, 2019 (EDT)

A minor remark: I don't think objective outcomes are seen as more reliable per se, it's just that they are more robust towards all sorts of biases. When a trial is properly controlled and blinded, then subjective outcomes can be equally or even more useful (for example because objective outcomes are often proxy's of what you want to measure).--Sisyphus (talk) 18:20, September 16, 2019 (EDT)

I agree, Sisyphus. Guido, guidelines are for everyone to follow. -- JaimeS

Wrong section for this comment, I think. What I'm saying here is that we shouldn't try and create guidelines for everything. Guido den Broeder (talk) 13:06, September 17, 2019 (EDT)

One study ain't enough

Suggestion: perhaps the guideline should mention 'one study ain't enough' or something like that. Because in my experience this is the most frequent mistake: people find a scientific study reporting X and they think X is a fact. So perhaps the guideline could add that it requires multiple studies with large sample size (more than 100) from several research teams for something to be considered established. If this isn't the case (which is the rule rather than the exception in ME/CFS) then this uncertainty should be reflected in the text. --Sisyphus (talk) 18:20, September 16, 2019 (EDT)

Re: One study ain't enough -- JaimeS (talk) 12:32, September 17, 2019 (EDT)

Oh, absolutely. And how many people are in a study -- and how this affects things like a genetics study. 200 is quite respectable for a small drug trial or looking at a blood marker, but it means zip in, say, GWAS. Sisyphus, can you add some suggested language for this? JaimeS

Michiel, I have added some space below for you to put your proposed text. You can put the text between the <pre> and the </pre>. User:Sisyphus

Pyrrhus (talk) 13:27, October 24, 2019 (EDT)

Balance

Suggestion: Another issue is imbalance. Sometimes people write things that are factually correct and well-referenced, but the information only presents one view on the subject. Suppose that someone only writes about the bad things a researcher did, and not about the good things (or the other way around). The person in question may not have bad intent: he may simply only know about the bad things. Even if the info is correct, it may look like MEpedia is trying to trash the researcher by only listing the bad things. Such a page would need a cleanup template on imbalance. So it might be good to mention this in the guideline. If you add info, be aware of the balance of the article. This is a wider issue than omitting references you disagree with.

Suggestion: A related issue I see, is people using a page to push a particular hypothesis. So the page might have correct statements and references but they are selected in such a way that they suggest a particular hypothesis/interpretation. Suppose someone believes ME/CFS is a stress-related illness and starts adding info on different pages that makes the connection with other stress-related illnesses or phenomena. Then he's pushing MEpedia content and the interpretation of readers towards suggesting a link between ME/CFS and stress. I think there's currently a danger of this happening with conditions that are seen as comorbid of ME/CFS. Mentioning them on several pages is suggesting a link between them and ME/CFS. One might have similar problems if people who are adamant about mold illness, chronic lyme start adding information. They might be well-intentioned and just adding the expertise they have, but because it's always from the same perspective and suggesting the same hypothesis, this might affect the balance and neutrality of MEpedia content as well.--Sisyphus (talk) 18:20, September 16, 2019 (EDT)

Re: Balance -- JaimeS (talk) 12:33, September 17, 2019 (EDT)

I think that this is already covered with "be careful of only citing what you agree with" (where it's deliberate). If people don't know about something, they can't help their slant because they're not aware of it. Others will have to come in and remove bias and add new information.