Skip to Main Content

Systematic Reviews

A guide directing researchers on the systematic review process. Layout based on Doing a Systematic Review: A Student's Guide, 2nd Edition, by Angela Boland, M. Gemma Cherry, and Rumona Dickson.

heading

Quality Assessment

Now that the body of literature has been compiled and data has been extracted, it is time to assess quality of the studies reviewed for relevance to the research question.

evaluating for bias

Evaluating for Bias

When reviewing study quality, assessing for bias is necessary to ensure reliability. Biases to assess for include the following, to name a few (Boland, Cherry, & Dickson, 2017):

  • Selection bias

How generalizable are the results of the study to your target population? Is the sample representative?

  • Allocation bias

How were participants assigned to treatment groups? Is this a predictable assignment, or was it random?

  • Performance bias

Were participants and/or researchers aware of their treatment group assignment? Were studies single-blind? Double-blind? If participants are aware of their assignment, they may feel an obligation to perform a certain way in alignment with the study's goal.

  • Detection bias

Were data analyzers and those reviewing study outcomes aware of individual identities/treatment group assignments? This may bias study results.

  • Attrition bias

What portion of participants stopped treatment prior to the end of the study? Did participants dropout of the study, withdraw, or not meet inclusion criteria? The more participants that drop out, the less generalizable the results.

  • Reporting bias

Were all outcomes reported on? Did authors fail to report outcomes that did not hold significance?

assess for quality

Assessing Study Quality

In assessing for quality, there are numerous tools available for researcher use. See the following resources for more information:

assess yourself

Assessing the Systematic Review for Quality

The following are standard tool used for quality assessment. Note, AMSTAR and PRISMA are standards in assessing systematic reviews, while MOOSE is the standard for assessing meta-analyses.

This checklist notes steps needing to be taken to conduct a systematic review, specifically in the healthcare fields (Shea et. al, 2017). This tool can be used to inform necessary pieces of the review.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is a resource available for those conducting systematic reviews (Page et. al, 2021). PRISMA offers tools including checklists for authors to ensure all necessary components are complete and flowcharts for illustrating the author's search process. The checklist (most recent version is 2020) provides details regarding every required aspect of a protocol/methodology for a systematic review. This should be treated as a bare minimum for authors to include.

This checklist is for authors and reviewers of meta-analyses (Stroup et. al, 2013).