section name header

Front Matter

Level of Evidence and Strength of Recommendation: Criteria and Wording in EBM Guidelines Evidence Summaries

All EBM Guidelines Evidence Summaries contain an evaluation of the quality of the evidence, determined according to the principles of the GRADE system. The evidence statements have PICO format (Patient, Intervention, Comparison, Outcome), and they use standard wording for each four levels of evidence. If the quality of evidence has been downgraded or upgraded, a comment after the evidence statement lists the reasons for downgrading or upgrading using standard phrases.

A recommendation using standard wording is included in those Evidence Summaries where a recommendation can be given on the basis of the evidence and consensus within the EBM Guidelines Editorial Team. A comment using standard expressions states the reasons for giving a weak recommendation on the basis of high quality evidence or a strong recommendation on the basis of low quality evidence.

Disclaimer: The standard wordings in the following tables have been formulated by the EBM Guidelines Editorial Team on the basis of the GRADE Group publications, and they should be considered as a work under development. They will be revised after feedback from users and stakeholders.

An example of an Evidence Summary with standard wording Psa for Screening of Prostate Cancer

Wording of evidence statements

Level of evidenceDefinitionWording *
A (high)We are very confident that the true effect lies close to that of the estimate of the effect.…is effective / has some effect / is not effective / is harmful…
B (moderate)We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.appears to be effective / appears to have some effect / appears not to be effective / appears to be harmful…
C (low)Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect.may be effective / may have limited effect / may not be effective / may be harmful…
D (very low)We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.might possibly/might possibly not ….be effective/have limited effect/be harmful… but the evidence is insufficient**There is insufficient evidence on the effect... ***
* A suitable verb can be used instead of the word “effective”, e.g. “...appears to improve survival”; “...may not prevent exacerbations”.
** “Might possibly be effective/have limited effect/be harmful” can be used when the point estimate suggests clinically meaningful/almost clinically meaningful beneficial or harmful effect.
*** This expression is used if no point estimate of the effect is available.
Rating down/up in RCTs
FactorComment
Study limitations (in RCTs)
  • lack of allocation concealment
  • lack of blinding
  • incomplete accounting of patients and outcome events
  • selective outcome reporting bias
  • other limitations
The quality of evidence is downgraded by study limitations (lack of/unclear allocation concealment).
The quality of evidence is downgraded by study limitations (lack of blinding).
The quality of evidence is downgraded by study limitations (high loss to follow-up in relation to observed absolute effect/failure to adhere to the intention-to-treat principle/differential loss to follow-up in the compared groups).
In noninferiority trials: high loss to follow-up in relation to observed absolute effect/failure to analyze both those adherent to treatment and all those whose outcome data are available.
The quality of evidence is downgraded by study limitations (selective outcome reporting).
The quality of evidence is downgraded by study limitations (stopping early for benefit/use of unvalidated outcome measures/carryover effects in crossover trial/recruitment bias in cluster-randomized trials).
Inconsistency
The quality of evidence is downgraded by inconsistency (unexplained variability in results).
Indirectness of evidence
The quality of evidence is downgraded by indirectness (differences between the population of interest and those studied).
The quality of evidence is downgraded by indirectness (differences between the interventions of interest and those studied).
The quality of evidence is downgraded by indirectness (differences between the outcomes of interest and those reported: only short-term outcomes reported/only surrogate outcomes reported/patient-important outcomes not reported)
The quality of evidence is downgraded by indirectness (direct comparisons not available)
Imprecise results
The quality of evidence is downgraded by imprecise results (wide confidence intervals).
The quality of evidence is downgraded by imprecise results (few patients/few outcome events/few patients and outcome events).
Publication bias
The quality of evidence is downgraded by suspected publication bias (only small and mostly commercially funded studies/asymmetrical funnel plot/statistically significant asymmetry)
Upgrading/Rating up
The quality of evidence is upgraded by large magnitude of effect.
The quality of evidence is upgraded by the fact that all plausible confounding would have reduced the effect.
The quality of evidence is upgraded by a clear dose-response gradient.
Formulation/wording of recommendation
Strong recommendation for using an intervention (= beneficial)is / are recommended
Weak recommendation for using an intervention (= likely to be beneficial)is / are suggested
Weak recommendation against using an intervention (= unlikely to be beneficial)cannot be suggested
Strong recommendation against using an intervention (= likely to be harmful)is / are not recommended

Determinants of strength of recommendations

FactorComment
Balance between desirable and undesirable effects
The recommendation is weak because of uncertain trade-off between benefits and harms/burden of treatment.
The recommendation is weak because of lack of information on comparative effectiveness with other interventions.*
The recommendation is strong because potential benefits of the intervention clearly exceed plausible harms.**
Quality of evidence
(The recommendation is weak because of low quality of evidence)***
Values and preferences
The recommendation attaches a relatively high value to …
Costs (resource allocation)
The recommendation is weak because of unclear/questionable cost-effectiveness.
* This expression can be used if alternative interventions exist and they might be as good or better options, but comparative studies are not available.
**This expression can be used if a strong recommendation is given in spite of low quality evidence. The resource consumption of the intervension should usually be low in this case.
*** No comment is needed if low quality of the evidence is the only reason for a weak recommendation.
References
Guyatt GH, Oxman AD, Schünemann HJ et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 2011;64(4):380-2. [PubMed]
  • Guyatt G, Oxman AD, Akl EA et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011;64(4):383-94. [PubMed]
  • Guyatt GH, Oxman AD, Kunz R et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 2011;64(4):395-400. [PubMed]
  • Balshem H, Helfand M, Schünemann HJ et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64(4):401-6. [PubMed]
  • Guyatt GH, Oxman AD, Vist G et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol 2011;64(4):407-15. [PubMed]
  • Guyatt GH, Oxman AD, Montori V et al. GRADE guidelines: 5. Rating the quality of evidence-publication bias. J Clin Epidemiol 2011 Jul 29 [Epub ahead of print]. [PubMed]
  • Guyatt G, Oxman AD, Kunz R et al. GRADE guidelines 6. Rating the quality of evidence-imprecision. J Clin Epidemiol 2011 Aug 10 [Epub ahead of print].[PubMed]
  • Guyatt GH, Oxman AD, Kunz R et al. GRADE guidelines: 7. Rating the quality of evidence-inconsistency. J Clin Epidemiol 2011 Jul 30 [Epub ahead of print]. [PubMed]
  • Guyatt GH, Oxman AD, Kunz R et al. GRADE guidelines: 8. Rating the quality of evidence-indirectness. J Clin Epidemiol 2011 Jul 29 [Epub ahead of print]. [PubMed]
  • Guyatt GH, Oxman AD, Sultan S et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 2011 Jul 29 [Epub ahead of print].[PubMed]