New Practitioner: how to interpret research and the 4 keys

Use these four keys when trying to interpret research and decide whether the results are reliable, or if they may have been swayed

Research helps us separate fact from fiction, identify cause and effect, and find connections previously unknown — but not all studies provide reliable and you interpret the research and results.

This requires reviewers, especially new practitioners, to have a critical mind when going through the data and deciding whether the conclusion reached is clear and convincing, or if it has enough issues to question its validity. Here are a few factors to consider when making this analysis.

Study design

How was the study conducted? Did it occur in a laboratory setting, where outside variables were heavily controlled or was data collected in a more open environment in which multiple, uncontrolled (and potentially unidentified) variables could have influenced the results? The more variables that exist, the greater the ability to create unreliable results as it can be difficult to interpret research and determine which variables had influence and to what extent.

Some studies are conducted by survey. In cases such as this, looking at the exact questions asked can be helpful. Were they misleading in any way? Were the response options variable enough to create a conclusive response? Additionally, was the survey lengthy, potentially creating fatigue in the respondent?

Survey-based studies may also limit who responded. If the survey was administered via email, for instance, respondents would need internet access to provide their input. This could preclude anyone who is not online from sharing their opinions. Considering all these design-based factors can help determine how much credence can be put into the results found.

Interpret research keeping an eye on study subjects

It’s not uncommon in science to apply what is learned in animal studies to the human population. This type of research is often used to help find new treatments or to better understand a disease. While some suggest that this is acceptable in that both animals and humans have similar biological processes, others contend that what we learn from animals doesn’t necessarily transfer to humans and is, therefore, unreliable.

There is no clear consensus either way. At a minimum, keeping this factor in mind when reviewing an animal study helps avoid tunnel vision when it comes to the results it provides.

Another factor to consider when it comes to the subjects being studied is if a majority of them were in a certain demographic. Were they similar in age, gender, socioeconomic status, or political affiliation, for instance? If so, this can make the findings difficult to apply to other demographics, sometimes even skewing the results.

Also, was there a control group from which to compare the results? If there was, is the data collected from the intervention group different enough to say that it provides statistical significance? If no control exists, all the study subjects may have been exposed to conditions that could have somehow influenced the results.

Study size

Results from a study involving 20,000 people would be considered more reliable than a study involving 20 or 200 people. The more subjects being analyzed, the better the ability to apply the results to the population at large versus being a matter of coincidence or only applying to smaller and perhaps more unique demographics.

Does this mean that individual case studies are unreliable? Not necessarily. However, following up with additional studies conducted on larger populations can help confirm the results of the case study, making their findings more credible.

Researcher bias and drug-funded studies

One of the reasons people distrust drug-based studies is that some are funded by the company manufacturing the drug at the center of the research. The fear is that this creates a bias in the researchers as their goal is to prove that the drug does, in fact, provide the results desired.

The organization behind the research doesn’t inherently make the study good or bad. It is just one more factor that needs to be considered when trying to interpret research and decide whether the results are reliable or if they may have been swayed.

Researcher bias can also exist if the need to prove the hypothesis is so great that other potential explanations for the outcome are ignored. This bias can be either conscious or unconscious, sometimes making it difficult to discern whether it exists and could have influenced the study in any way.

Keeping a critical mind when considering these factors can help you discern whether new research is valid, and whether it applies to the population at large or smaller segments of a specific demographic. This helps you decide who may benefit most from the findings, enabling you to provide more informed guidance to patients who would be affected most.