How to Evaluate Research Articles for Credibility

By Lon W. Schiffbauer, BA, MBA, PhD, SPHR

When pulling together a literature review, a researcher should evaluate the quality of a research article by considering the credibility of the publication, the credibility of the authors, the purpose of the study, the scope of the literature review, the selection of the methodology, the design of the study, the presentation of the findings, the stated biases, and the consideration of ethical guidelines. Let’s explore each one of these in detail.

Credibility of the Publication

First, let’s look at the credibility of the publication. An article published by a respected peer-reviewed journal is going to be considered more credible than, say, content published by some nameless blog no one has ever heard of (like Nutshell Brainery—sigh).

One tool you can use to determine the ranking of a journal in terms of prestige in the academic community is the SCImago Journal & Country Rank website. This site ranks research and academic journals from around the world, giving researchers an insight into the credibility of various academic publications.

In some cases, you may not necessarily be referencing an academic journal. However, you still need to consider credibility of the source. For example, I would trust business publications such as Harvard Business Review or the Wall Street Journal over opinion-driven blog articles.

In the case of the news media, I would try to find sources that are generally considered balanced and unbiased, such as Reuters and the Associated Press. For a better picture of where news outlets fall in terms of left, right, or centrist, check out All Sides.com. This site ranks news outlets and other media companies along a continuum of progressive, centrist, and conservative.

Credibility of the Authors

The next thing you’ll want to consider is the credibility of the authors. Are the authors practitioners working in the field, academics publishing in their domain, students cutting their teeth, or consultants with a financial motive? What are the researchers’ credentials? For example, I have a PhD in Industrial/Organizational Psychology and am a certified Senior Human Resources Professional, so I’m qualified to speak about HR and organizational management. But now if I were to try and speak with any definitive authority on, say, political science, I wouldn’t be considered a credible source, my degrees and certifications notwithstanding.

You’ll also want to take a look at their publishing history. How many peer-reviewed studies have they conducted? How often have they been cited by other researchers? Who backed the researcher? In other words, where was the research conducted and who paid for it?

Purpose of the Study

Next, you’ll want to look at the purpose of the study. Is the purpose of the study clearly stated in context of the current body of knowledge? Does it include a specific set of research questions and hypotheses? Do the authors clearly state how this work differs from, adds to, or validates other work? If it’s not clear to you what is driving the research then it may not be clear to the researchers themselves, or worse still, it is clear, but they want to obfuscate their intentions.

Scope of the Literature Review

Next, consider the scope of the literature review. Does the literature review include pertinent citations as related to the field of study? Does it include the most important seminal works in the field? Does it include the latest research on the topic? The better the authors couch their research in the larger body of knowledge, the more you can trust the credibility of the work.

Selection of the Methodology

The next thing you’ll want to look at is the selection of the methodology. Is the selected methodology appropriate, given the research question? Is the study population well-defined? How were study participants recruited and evaluated? Was the final sample size adequate for the chosen methodology? Remember that the researchers are drawing conclusions from this population, so if the population is less than robust, then likewise are the findings.

Design of the Study

Next, consider the design of the study. Was the design of the study appropriate, given the research question? Are the variables and controls clearly defined? How were the outcomes measured and codified? Was the design described in sufficient detail as to allow other researchers to replicate the study? Like the methodology, a lot hinges on the study’s design. If any aspect of it is ill-defined or slipshod then the findings should be immediately suspect.

Presentation of the Findings

Next, have a look at the presentation of the findings. You’ll want to make sure the findings are presented in such a way as to facilitate understanding with data and supporting examples. This can include the use of graphs, charts, tables, and other tools to help make the findings clear and transparent. It also means that the discussion section offers a clear narrative of the findings, no slight-of-hand to try and bewilder and befuddle the reader. Ultimately, you want to ask yourself whether the stated conclusions align with the research findings as described in the data.

Stated Biases

This brings us to considering the stated biases in the study. Do the authors call out specific biases and limitations of the study? Did they put in place measures to mitigate the effects of these biases? Everyone has biases, including the very best researchers. The goal is not to find a study with no biases, but rather a study that recognizes, details, and—to the extent possible—mitigates for these biases.

Consideration of Ethical Guidelines

Finally, there is the consideration of ethical guidelines. In all cases, you’ll want to make sure the research was conducted in accordance with ethical guidelines. By looking for and critically examining the answers to these questions, you’ll not only be able to better identify quality work, but you’ll also learn how to deliver that same level of research discipline to your own work.

Lon is an Associate Professor of Business Management at Salt Lake Community College and holds an MBA, a PhD, and is a certified Senior Professional in Human Resources (SPHR). In addition to his academic background, Lon spent close to 30 years working and consulting for such companies as FedEx, Intel, eBay, and PayPal, as well as a variety of small to mid-sized companies around the world.