Research Guides: Systematic Reviews: Appraise & Synthesize

Step 5: Appraise & Synthesize

During this step, individual studies are evaluated for potential bias, based on study design and reporting. For example, design flaws that raise questions about the validity of findings or an overestimate of the intervention effect. Extract data from all relevant studies. There are several software programs that assist in organizing and visualizing your data. See the data management guide for more information about these products.

Systematic Review Tools

Tools for Avoiding Bias

Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions
From the AHRQ - "Methods Guide for Effectiveness and Comparative Effectiveness Reviews"
Avoiding Bias in Selecting Studies (AHRQ)
This is a chapter from "Methods Guide for Effectiveness and Comparative Effectiveness Reviews."
Peer Review of Electronic Search Strategies (PRESS)
The guideline statement for Peer Review of Electronic Search Strategies (PRESS) for systematic reviews (SRs), health technology assessments, and other evidence syntheses.
Table 7.2.a Definitions of some types of non-reporting biases, Cochrane Handbook
A list of the key types of non-reporting bias.
What is PRESS?
This infographic explains PRESS: What is it? Why is it important? When should it be done? How is it done?

Tools for Appraisal

AMSTAR Checklist
AMSTAR stands for A MeaSurement Tool to Assess systematic Reviews.
CASP: Critical Appraisal Skills Program Checklists
This set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
GRADE Working Group
The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations.
JADAD - Jadad quality assessment scale for rating Randomized Controlled Trials
The higher the score, the higher the quality, this permits an author to rank the studies in a review. The scale was introduced in the journal article Jadad, A., Moore, R., Carroll, D., Jenkinson, C., Reynolds, D., Gavaghan, D., & McQuay, H. (1996). Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Controlled Clinical Trials, 17(1), 1-12.
Mixed Methods Appraisal Tool
The Mixed Methods Appraisal Tool (MMAT) is intended to be used as a checklist for concomitantly appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods
studies).
MOOSE Guidelines for Meta-Analyses and Systematic Reviews of Observational Studies
Modified from Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational
studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group.
JAMA 2000;283:2008–12.
The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses
The Newcastle-Ottawa Scale (NOS) was developed to assess the quality of nonrandomised studies with its design, content and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results.
ROBIS (Risk of Bias in Systematic Reviews)
ROBIS is a tool for assessing the risk of bias in systematic reviews (rather than in primary studies)
SURE for Systematic Reviews
Questions to assist with the critical appraisal of a systematic review

Statistical Help Available at UK

While UK Libraries does not provide assistance with statistics, the University of Kentucky offers assistance through the Applied Statistics Laboratory: