When I first began college, I thought that most of the information I found online was credible – especially when the author described findings from a research study. I assumed that if something were presented in the media using phrases like “researchers report that…” or “study finds that…”, it was true. What I learned over time is that there are varying degrees of quality information reported in the mass media and unfortunately research findings are often grossly misrepresented. The purpose of this blog post is to provide quality questions that distinguish between research opinions and accurate and reliable information.
Imagine that you just read an interesting article on your Facebook feed. Is what you read fact or fluff? How do you determine whether you can trust the information reported in the article? How do you gauge the quality of the information? Here are some questions you can ask yourself to identify quality information in a quantity world:
Can you trace the original source of information?
If after reading the article, you cannot identify the primary source, you may need to question what you read. For example, based on the content of the article, can you identify the name of the study, the author/s, the publication year, the organization that conducted the study, and/or the name of the peer-reviewed journal in which the study was published? If you answer no to most of these criteria, you may need to reconsider the accuracy of the information presented.
Is the publication source credible?
Before you believe what you read, make sure to identify who published the article. Determine who authored the article and the status of the information. Investigate whether the author has credentials and expertise in the area. Ask yourself if the publication is known for credible information. For example, on May 10, 2015, Jezebel and author Tracy Moore published an article titled, Study Finds Men are Excellent at Looking Busy. The publication Jezebel is not known for being a credible source of information. Furthermore, there is no quality information about the author. It is possible that she is a credible source of information. However, the fact that no information is provided about her background makes me question her credibility.
Did the author describe the sample?
After reading the article, do you know exactly who participated in the study? Did the author describe the number of participants and the sample demographics (e.g., age, gender, race, etc.). If you wanted to replicate this study – could you with the information that was provided? All of these details is vital when identifying quality research information.
Does the author support their claims?
When the author reports the findings from the study, do they provide additional information to support their claims? If the answer is yes, it demonstrates that they have synthesized information from a variety of different sources. Quality articles will cite their sources and provide references. The author may also provide links to additional information to support their claims. For instance, the following article represents a solid example of a study reported in the mass media that supported their claims.
Are the findings too good to be true?
Approach the findings you read with some skepticism. Rarely in the field of research does A = B without other factors like C, D, and E being part of the equation. Correlation doesn’t equal causation – though spurious correlations can be entertaining. In fact, one study claimed that eating a chocolate bar a day helped people lose weight! If only weight loss were that simple! The reality of this study was that a journalist scammed the media to demonstrate how easy it is to disseminate poor information. Sadly, thousands of people who read this article may still believe that a chocolate bar a day keeps the pounds away…instead, they probably gained weight because this finding was too good to be true.
Is the title of the article sexy?
Is the title of the article is flashy, misleading, or designed to quickly grab the attention of readers- see the questions above to assess the quality of the article. In my experience, the sexier the article title, the more likely it is that you are reading an opinion piece.
For further information on this topic, check out this excellent blog post by Skeptical Raptor that delves deeper into methods for evaluating scientific research quality in the mass media.
Feel free to contact me with any research and evaluation questions or needs at annette@researchevaluationconsulting.com
Related Posts:
You need to take part in a contest for one of the greatest blogs online. I am going to highly recommend this web site!
“These are in fact fantastic ideas in on the topic of blogging.
You have touched some nice factors here. Any way keep up wrinting.”
Today I finally think I’ve made some progress in an area I really needed it. Hopefully, the mistakes I’ve been making with my attitude and lack of faith in this area won’t need to be repeated. I’m grateful for my faith, my angels and for your always kind and wise words!