Assessing Quality of Studies and Risk of Bias

check-boxesQuality assessment can be affected by the study size, methodological design and other factors that result in increased bias. Several medical research organizations have adopted the GRADE approach, which defines the quality of a body of evidence as “the extent to which one can be confident that an estimate of effect or association is close to the quantity of specific interest.”
According to the GRADE approach, quality of evidence can be affected negatively by the following factors:

  • Limitations in the design and implementation of available studies suggesting high likelihood of bias
  • Indirectness of evidence (indirect population, intervention, control, outcomes)
  • Unexplained heterogeneity or inconsistency of results (including problems with subgroup analyses)
  • Imprecision of results (wide confidence intervals)
  • High probability of publication bias

In addition to the overall quality of evidence assessment, there are several tools available for assessing individual study bias. For randomized clinical trials (RCTs), we recommend the Cochrane Collaboration’s Risk of Bias tool for assessing the following categories:

  • Selection bias
  • Performance bias
  • Detection bias
  • Attrition bias
  • Reporting bias

Using Cochrane’s Risk of Bias tool, studies are categorized as having a low, high, or unclear risk of bias, allowing investigators to acknowledge and account for potentially misleading studies in their analyses.

Other tools, including those incorporating non-RCT study designs, have been developed for study quality and risk of bias assessment in systematic reviews as well as other documents (e.g. guidelines, Health Technology Assessment and consensus statements). We are able to identify and adapt Quality Assessment and Risk of Bias tools to suit the needs of each project.

Please contact us to discuss your needs and questions regarding Quality Assessment of various study types.