Chiropractic studies: How to evaluate and analyze the good & the bad

Do a Google search for chiropractic studies involving instruments and you’ll receive almost one million results.

While some of these pieces of research are worthwhile, providing compelling evidence of what does or does not work in regard to instruments used within the field of chiropractic care, others are seriously flawed, meaning that any conclusions or findings that they have reached cannot be trusted at all.

This often brings up the question: How can you tell one from the other?

Specifically, what separates a good chiropractic, instrument-based study from a bad one so you know which findings are valid and which ones won’t hold up to further scrutiny, thus making them unreliable when it comes to their findings?

To know for sure requires that you look for a few different things.

A clearly defined purpose

According to Pie Tutors, an important characteristic of a good study is “that the purpose of the research is clearly defined.” Typically the purpose is stated at the beginning of the study, informing the reader as to why the research was conducted or what prompted the study.

Thus, when looking at a chiropractic study yourself, pay attention to whether the purpose is defined in clear and concise terms or if it’s vague in nature. The more precise it is, the better the study and the more trustworthy the results.

Lack of research bias

High quality research studies also contain no traceable research bias. This means that the study was conducted in a way that did not favor one idea or result over another. This is sometimes difficult as an article published in the Plastic and Reconstructive Surgery journal explains that “bias can occur in the planning, data collection, analysis, and publication phases of research.”

The key to determining whether bias exists in a study, according to the article, is to first understand that prejudices can occur at any stage of the research. It involves staying open-minded as you read about what was done, looking for anything that makes you believe that the researchers were trying to prove one specific thing versus letting the findings speak for themselves.

Also examine who performed or funded the research study and whether or not they may have a vested interest in one result over another.

Absence of validity threats

Another important component of a good research study is the absence of validity threats. Research Connections states that there are three common types of threats to a study’s validity. They are: internal, external, and construct.

An internal validity threat occurs if there are other possible outcomes for the findings, other than the ones discussed in the study itself. This is the case when a study finds links between two ideas or actions, but cannot necessarily establish cause and effect.

External validity issues refer to “generalizability of the findings.” In other words, when reading a study, ask yourself: What is the likelihood that the same study held in a different setting or with different participants would yield the same results?

For example, would a study conducted on teens in California get the same findings as a study which involved adults from Connecticut? If not, it may lack external validity.

Finally, construct validity involves assessing whether the study’s units of measures are efficient enough to correctly ascertain the results. If a study was being conducted on fat loss, for instance, researchers would have more valid findings if they tested body fat percentage using calipers versus relying solely on what a scale says since the scale measures water and muscle loss too.

Objective data collection methods

There are many different data collection methods which provide valid results, but some offer more compelling proof due primarily to their lack of subjectivity. For example, if you’re testing a person’s response to a specific medication used to lower cholesterol, doing so via blood tests conducted before, during, and after the study would provide more unbiased results than simply asking the participants how they feel once the research was complete.

In this case, blood tests cannot be manipulated to provide a specific response whereas a person’s thoughts regarding a particular treatment regimen can change based on their feelings toward the study, expectations of treatment results, and a myriad of other factors.

Thus, their answers to study-related question would be open to scrutiny due to their subjectivity whereas blood markers would be more trusted since they’d be much harder to manipulate.

Not all instrument-related chiropractic studies are valid. But as long as you look for these four things, you’ll be better able to spot the good ones from the bad and provide your patients the best information possible based only on reliable research.