Hi, welcome back! Hope your reviews are progressing well. If not, then don’t worry and follow each step at a time. Focusing on one step is easier than trying to understand the entire process. In the last post, we discussed data screening and extraction (https://scienceiq.blog/2019/03/23/how-to-screen-articles-and-extract-data-for-a-systematic-review/). Today, we will move on to one of the most fundamental steps of a systematic review, i.e., Assessment of Risk of Bias of the included studies. I will briefly discuss – what is a bias, why it should be assessed, types of biases and how to assess it, with more focus on interventional studies.
What is Bias?
We research to know the truth or reality of the situation. Bias is a deviation from the truth. Example, if a study is not well conducted, then its results may be deviated from the actual reality. Therefore, any methodological flaw in the study may lead to biases in the results or its interpretation.
Why should bias be assessed?
We assess the risk of bias in a study to detect if the results of that particular study are underestimated or overestimated. We do not want to go on face value and need to find the actual truth. If a study has many risks of bias or it is not well-conducted, then it is likely to be reflecting a false/skewed picture. We need to be careful when accepting the results of any study. Especially when we are performing a synthesis of the literature, we need to assess the risk of bias to generalize the findings of the study.
The quality of your systematic review is the quality of studies included in the review. So you need to make sure that the studies included have good quality/fewer biases. If not, then you may either exclude those studies or add a word of caution about interpretation of the results. This is the most important step in a review and requires some experience and understanding of the subject.
What are the different types of biases?
Bias can be present in any form and requires different ways to assess it. Most common biases that may manipulate the results are selection bias, performance bias, detection bias, attrition bias and reporting bias.
Selection bias may be present when the participants in the experimental and control group are not similar at baseline. For example in the experimental group, all the participants are younger than in the control group so the outcomes are automatically better in the experimental group. Such kind of flaw is present due to selection bias. This bias can be minimized by the random selection of participants through random sequence generation and allocation concealment.
Performance bias is present when the participants in one group are more likely to perform an intervention better because of their or investigators’ awareness about the group to which they belong. Example, if participants in the experimental group receive more care and motivation compared to the control group to achieve additional effect of an intervention. It can be minimized by the blinding of participants or personnel to the intervention and allocation concealment.
Detection bias is similar to performance bias, but it is present due to outcome assessor. If the assessors are aware of the group of participants, then they may measure outcomes differently. Detection bias often exists in the subjective assessment of outcomes. Example, an outcome assessor, may give better scores to the experimental group and poor scores to control group to show more effect of the given intervention. It can be minimized by blinding of outcome assessor to the intervention.
Attrition Bias is present when one group has more attrition/dropouts compared to the other group. The statistical analysis may vary due to the difference in participant number in both the groups which introduces bias. It can be minimized by complete data collection and providing reasons for dropouts in each group.
Reporting bias is present when some of the outcomes are reported while some are skipped. It usually happens when an author reports only the significant outcomes and do not mention about the outcomes that were not significant. It can be minimized by reporting all the outcomes of the study.
How to assess Risk of Bias?
Risk of bias can be measured by looking into the methodology and different domains at risk. Example, in an interventional study, you need to ask questions: were the participants randomly allocated to the experimental or control group? Was there any allocation concealment? Were the outcome assessor and participants blinded? Were all the outcomes were reported? What was the attrition rate in both groups?
Cochrane tool for risk of bias measurement is the recommended tool for interventional studies. For other forms of study design, I have tabulated the commonly used tools. Scales that provide summary scores are NOT recommended for risk evaluation because of the validity of such scales.
| Type of study design | ROB Tools |
| Interventional study | Cochrane Risk of Bias tool, JBI Critical appraisal tool |
| Diagnostic study | Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) Critical Appraisal Skill Program (CASP) JBI Critical appraisal tool |
| Prognostic/ Prevalence study | CEBM Prognosis Appraisal worksheet JBI Critical appraisal tool CASP cohort study checklist |
| Accuracy of Outcome Measurement | COSMIN checklist |
| Qualitative Study | CASP Qualitative checklist JBI Critical appraisal tool |
Once you have assessed the risk of bias for each study, summarize the biases in each study, across the different domains, and overall risks present. You can use RevMan to generate appealing tables and graphs for Risk of Bias that you find in Cochrane reviews. You may refer to the following articles if you wish to read further.
1. BMJ 2011;343:d5928 – The Cochrane Collaboration’s tool for assessing risk of bias in randomised trial
2. Alex Pollock and Eivind Berge, 2018 https://doi.org/10.1177%2F1747493017743796
Science IQ 
Like!! Thank you for publishing this awesome article.
LikeLike