Evidence-Based Practice in Health

What is “the best available evidence”?  The hierarchy of evidence is a core principal of Evidence-Based Practice (EBP) and attempts to address this question.  The evidence higherarchy allows you to take a top-down approach to locating the best evidence whereby you first search for a recent well-conducted systematic review and if that is not available, then move down to the next level of evidence to answer your question.

EBP hierarchies rank study types based on the rigour (strength and precision) of their research methods.  Different hierarchies exist for different question types, and even experts may disagree on the exact rank of information in the evidence hierarchies.  The following image represents the hierarchy of evidence provided by the National Health and Medical Research Council (NHMRC).¹

Most experts agree that the higher up the hierarchy the study design is positioned, the more rigorous the methodology and hence the more likely it is that the study design can minimise the effect of bias on the results of the study.  In most evidence hierachies current, well designed systematic reviews and meta-analyses are at the top of the pyramid, and expert opinion and anecdotal experience are at the bottom.²

Systematic Reviews and Meta Analyses

Well done systematic reviews, with or without an included meta-analysis, are generally considered to provide the best evidence for all question types as they are based on the findings of multiple studies that were identified in comprehensive, systematic literature searches.  However, the position of systematic reviews at the top of the evidence hierarchy is not an absolute.  For example:

• The process of a rigorous systematic review can take years to complete and findings can therefore be superseded by more recent evidence.
• The methodological rigor and strength of findings must be appraised by the reader before being applied to patients.
• A large, well conducted Randomised Controlled Trial (RCT) may provide more convincing evidence than a systematic review of smaller RCTs.⁴

Primary Studies

If a current, well designed systematic review is not available, go to primary studies to answer your question. The best research designs for a primary study varies depending on the question type.  The table below lists optimal study methodologies for the main types of questions.

Note that the Clinical Queries filter available in some databases such as PubMed and CINAHL matches the question type to studies with appropriate research designs.

When searching primary literature, look first for reports of clinical trials that used the best research designs. Remember as you search, though, that the best available evidence may not come from the optimal study type. For example, if treatment effects found in well designed cohort studies are sufficiently large and consistent, those cohort studies may provide more convincing evidence than the findings of a weaker RCT.

What is a Systematic Review?

A systematic review synthesises the results from all available studies in a particular area, and provides a thorough analysis of the results, strengths and weaknesses of the collated studies.  A systematic review has several qualities:

1. It addresses a focused, clearly formulated question.
2. It uses systematic and explicit methods:

a. to identify, select and critically appraise relevant research, and
b. to collect and analyse data from the studies that are included in the review

Systematic reviews may or may not include a meta-analysis used to summarise and analyse the statistical results of included studies. This requires the studies to have the same outcome measure.

What is a Narrative Review?

Narrative reviews (often just called Reviews) are opinion with selective illustrations from the literature.  They do not qualify as adequate evidence to answer clinical questions.  Rather than answering a specific clinical question, they provide an overview of the research landscape on a given topic and so maybe useful for background information.  Narrative reviews usually lack systematic search protocols or explicit criteria for selecting and appraising evidence and are threfore very prone to bias.⁵

Filtered information appraises the quality of a study and recommend its application in practice.  The critical appraisal of the individual articles has already been done for you—which is a great time saver.  Because the critical appraisal has been completed, filtered literature is appropriate to use for clinical decision-making at the point-of-care. In addition to saving time, filtered literature will often provide a more definitive answer than individual research reports.  Examples of filtered resources include, Cochrane Database of Systematic Reviews, BMJ Clincial Evidence, and ACP Journal Club.

Unfiltered information are original research studies that have not yet been synthesized or aggregated. As such, they are the more difficult to read, interpret, and apply to practice.  Examples of unfiltered resources include, CINAHL, EMBASE, Medline, and PubMed.³

The Cochrane Collaboration is an international voluntary organization that prepares, maintains and promotes the accessibility of systematic reviews of the effects of healthcare.

The Cochrane Library is a database from the Cochrane Collaboration that allows simultaneous searching of six EBP databases.  Cochrane Reviews are systematic reviews authored by members of the Cochrane Collaboration and available via The Cochrane Database of Systematic Reviews.  They are widely recognised as the gold standard in systematic reviews due to the rigorous methodology used.

Abstracts of completed Cochrane Reviews are freely available through PubMed and Meta-Search engines such as TRIP database.

National access to the Cochrane Library is provided by the Australian Government via the National Health and Medical Research Council (NHMRC).

Before you get started with this assignment, please complete the Research Hierarchy Activity to provide context for the assignment.

Now that you have completed the Research Hierarchy activity, you can see how each type of research fits in a particular hierarchy. This is important because it provides information about each type of research study: its validity, rigor, soundness, reliability, and so on. With this context in mind, you will now examine the twenty articles provided here to give you practice in determining which article goes with each level of research hierarchy.

Overview

To Submit Your Assignment:

Writing Assignment Rubric

Note: Scholarly resources are defined as evidence-based practice, peer-reviewed journals; textbook (do not rely solely on your textbook as a reference); and National Standard Guidelines. Review assignment instructions, as this will provide any additional requirements that are not specifically listed on the rubric.

Writing Assignment Rubric – 100 PointsCriteriaExemplary
Exceeds ExpectationsAdvanced
Meets ExpectationsIntermediate
Needs ImprovementNovice
InadequateTotal PointsContent of PaperThe writer demonstrates a well-articulated understanding of the subject matter in a clear, complex, and informative manner. The paper content and theories are well developed and linked to the paper requirements and practical experience. The paper includes relevant material that fulfills all objectives of the paper.

Cites five or more references, using at least two new scholarly resources that were not provided in the course materials.

All instruction requirements noted.

30 pointsThe writer demonstrates an understanding of the subject matter, and components of the paper are accurately represented with explanations and application of knowledge to include evidence-based practice, ethics, theory, and/or role. Course materials and scholarly resources support required concepts. The paper includes relevant material that fulfills all objectives of the paper.

Cites four references.

All instruction requirements noted.

26 pointsThe writer demonstrates a moderate understanding of the subject matter as evidenced by components of the paper being summarized with minimal application to evidence-based practice, theory, or role-development. Course content is present but missing depth and or development.

Cites three references.

Most instruction requirements are noted.

23 pointsAbsent application to evidence-based practice, theory, or role development. Use of course content is superficial.

Demonstrates incomplete understanding of content and/or inadequate preparation.

Content of paper is inaccurately portrayed or missing.

Cites two or fewer references.

Missing some instruction requirements.

20 points30Analysis and Synthesis of Paper Content and MeaningThrough critical analysis, the submitted paper provides an accurate, clear, concise, and complete presentation of the required content.

Information from scholarly resources is synthesized, providing new information or insight related to the context of the assignment by providing both supportive and alternative information or viewpoints.

All instruction requirements noted.

30 pointsPaper is complete, providing evidence of further synthesis of course content via scholarly resources.

Information is synthesized to help fulfill paper requirements. The content supports at least one viewpoint.

All instruction requirements noted.

26 pointsPaper lacks clarification or new information. Scholarly reference supports the content without adding any new information or insight. The paper’s content may be confusing or unclear, and the summary may be incomplete.

Most instruction requirements are noted.

23 pointsSubmission is primarily a summation of the assignment without further synthesis of course content or analysis of the scenario.

Demonstrates incomplete understanding of content and/or inadequate preparation.

Missing some instruction requirements.

Submits assignment late.

20 points30Application of KnowledgeThe summary of the paper provides information validated via scholarly resources that offer a multidisciplinary approach.

The student’s application in practice is accurate and plausible, and additional scholarly resource(s) supporting the application is provided.

All questions posed within the assignment are answered in a well-developed manner with citations for validation.

All instruction requirements noted.

30 pointsA summary of the paper’s content, findings, and knowledge gained from the assignment is presented.

Student indicates how the information will be used within their professional practice.

All instruction requirements noted.

26 pointsObjective criteria are not clearly used, allowing for a more superficial application of content between the assignment and the broader course content.

Student’s indication of how they will apply this new knowledge to their clinical practice is vague.

Most instruction requirements are noted.

23 pointsThe application of knowledge is significantly lacking.

Student’s indication of how they will apply this new knowledge to their clinical practice is not practical or feasible.

Demonstrates incomplete understanding of content and/or inadequate preparation.

Application of knowledge is incorrect and/or student fails to explain how the information will be used within their personal practice.

Missing several instruction requirements.

Submits assignment late.

20 points30OrganizationWell-organized content with a clear and complex purpose statement and content argument. Writing is concise with a logical flow of ideas.

5 pointsOrganized content with an informative purpose statement and supportive content and summary statement. Argument content is developed with minimal issues in content flow.

4 pointsPoor organization, and flow of ideas distract from content. Narrative is difficult to follow and frequently causes reader to reread work.

Purpose statement is noted.

3 pointsIllogical flow of ideas. Missing significant content. Prose rambles. Purpose statement is unclear or missing.

Demonstrates incomplete understanding of content and/or inadequate preparation.

No purpose statement.

Submits assignment late.

2 points5APA, Grammar, and SpellingCorrect APA formatting with no errors.

The writer correctly identifies reading audience, as demonstrated by appropriate language (avoids jargon and simplifies complex concepts appropriately).

Writing is concise, in active voice, and avoids awkward transitions and overuse of conjunctions.

There are no spelling, punctuation, or word-usage errors

5 pointsCorrect and consistent APA formatting of references and cites all references used. No more than two unique APA errors.

The writer demonstrates correct usage of formal English language in sentence construction. Variation in sentence structure and word usage promotes readability.

There are minimal to no grammar, punctuation, or word-usage errors.

4 pointsThree to four unique APA formatting errors.

The writer occasionally uses awkward sentence construction or overuses/inappropriately uses complex sentence structure. Problems with word usage (evidence of incorrect use of thesaurus) and punctuation persist, often causing some difficulties with grammar. Some words, transitional phrases, and conjunctions are overused.

Multiple grammar, punctuation, or word usage errors.

3 pointsFive or more unique formatting errors or no attempt to format in APA.

The writer demonstrates limited understanding of formal written language use; writing is colloquial (conforms to spoken language).

The writer struggles with limited vocabulary and has difficulty conveying meaning such that only the broadest, most general messages are presented.

Grammar and punctuation are consistently incorrect. Spelling errors are numerous.

Submits assignment late.

2 points5Total Points100

When searching for evidence-based information, one should select the highest level of evidence possible–systematic reviews or meta-analysis. Systematic reviews, meta-analysis, and critically-appraised topics/articles have all gone through an evaluation process: they have been “filtered”.

Information that has not been critically appraised is considered “unfiltered”.

As you move up the pyramid, however, fewer studies are available; it’s important to recognize that high levels of evidence may not exist for your clinical question.  If this is the case, you’ll need to move down the pyramid if your quest for resources at the top of the pyramid is unsuccessful.

• Meta-Analysis  A systematic review that uses quantitative methods to summarize the results.

• Systematic Review    An article in which the authors have systematically searched for, appraised, and summarised all of the medical literature for a specific topic.

• Critically Appraised Topic     Authors of critically-appraised topics evaluate and synthesize multiple research studies.

• Critically Appraised Articles  Authors of critically-appraised individual articles evaluate and synopsize individual research studies.

• Randomized Controlled Trials  RCT’s include a randomized group of patients in an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.

• Cohort Study  Identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

• Case-Control Study  Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking to see if they had the exposure of interest.

• Background Information / Expert Opinion   Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition.  While background information presents a convenient summary, often it takes about three years for this type of literature to be published.

• Animal Research / Lab Studies  Information begins at the bottom of the pyramid: this is where ideas and laboratory
  research takes place. Ideas turn into therapies and diagnostic tools, which then are tested with lab models and
  animals.

Sources:
Greenhalgh, Trisha.  How to Read a Paper: the Basics of Evidence Based Medicine.  London: BMJ, 2000.
Glover, Jan; Izzo, David; Odato, Karen & Lei Wang. EBM Pyramid.  Dartmouth University/Yale University. 2006.

Study Design

Different types of clinical questions are best answered by different types of research studies.

You might not always find the highest level of evidence (i.e., systematic review or meta-analysis) to answer your question. When this happens, work your way down the Evidence Pyramid to the next highest level of evidence.

This table suggests study designs best suited to answer each type of clinical question.

1. M Hassan Murad,
2. Noor Asi,
3. Mouaz Alsawas,
4. Fares Alahdab

Author affiliations

http://dx.doi.org/10.1136/ebmed-2016-110401

Statistics from Altmetric.com

See more details

Blogged by 2

Tweeted by 315

On 22 Facebook pages

Mentioned in 2 Google+ posts

On 2 videos

751 readers on Mendeley

• Linked Articles
• Request Permissions

The first and earliest principle of evidence-based medicine indicated that a hierarchy of evidence exists. Not all evidence is the same. This principle became well known in the early 1990s as practising physicians learnt basic clinical epidemiology skills and started to appraise and apply evidence to their practice. Since evidence was described as a hierarchy, a compelling rationale for a pyramid was made. Evidence-based healthcare practitioners became familiar with this pyramid when reading the literature, applying evidence or teaching students.

Various versions of the evidence pyramid have been described, but all of them focused on showing weaker study designs in the bottom (basic science and case series), followed by case–control and cohort studies in the middle, then randomised controlled trials (RCTs), and at the very top, systematic reviews and meta-analysis. This description is intuitive and likely correct in many instances. The placement of systematic reviews at the top had undergone several alterations in interpretations, but was still thought of as an item in a hierarchy.1 Most versions of the pyramid clearly represented a hierarchy of internal validity (risk of bias). Some versions incorporated external validity (applicability) in the pyramid by either placing N-1 trials above RCTs (because their results are most applicable to individual patients2) or by separating internal and external validity.3

Another version (the 6S pyramid) was also developed to describe the sources of evidence that can be used by evidence-based medicine (EBM) practitioners for answering foreground questions, showing a hierarchy ranging from studies, synopses, synthesis, synopses of synthesis, summaries and systems.4 This hierarchy may imply some sort of increasing validity and applicability although its main purpose is to emphasise that the lower sources of evidence in the hierarchy are least preferred in practice because they require more expertise and time to identify, appraise and apply.

The traditional pyramid was deemed too simplistic at times, thus the importance of leaving room for argument and counterargument for the methodological merit of different designs has been emphasised.5 Other barriers challenged the placement of systematic reviews and meta-analyses at the top of the pyramid. For instance, heterogeneity (clinical, methodological or statistical) is an inherent limitation of meta-analyses that can be minimised or explained but never eliminated.6 The methodological intricacies and dilemmas of systematic reviews could potentially result in uncertainty and error.7 One evaluation of 163 meta-analyses demonstrated that the estimation of treatment outcomes differed substantially depending on the analytical strategy being used.7 Therefore, we suggest, in this perspective, two visual modifications to the pyramid to illustrate two contemporary methodological principles (figure 1). We provide the rationale and an example for each modification.

 

Figure 1

The proposed new evidence-based medicine pyramid. (A) The traditional pyramid. (B) Revising the pyramid: (1) lines separating the study designs become wavy (Grading of Recommendations Assessment, Development and Evaluation), (2) systematic reviews are ‘chopped off’ the pyramid. (C) The revised pyramid: systematic reviews are a lens through which evidence is viewed (applied).

Rationale for modification 1

In the early 2000s, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group developed a framework in which the certainty in evidence was based on numerous factors and not solely on study design which challenges the pyramid concept.8 Study design alone appears to be insufficient on its own as a surrogate for risk of bias. Certain methodological limitations of a study, imprecision, inconsistency and indirectness, were factors independent from study design and can affect the quality of evidence derived from any study design. For example, a meta-analysis of RCTs evaluating intensive glycaemic control in non-critically ill hospitalised patients showed a non-significant reduction in mortality (relative risk of 0.95 (95% CI 0.72 to 1.25)9). Allocation concealment and blinding were not adequate in most trials. The quality of this evidence is rated down due to the methodological imitations of the trials and imprecision (wide CI that includes substantial benefit and harm). Hence, despite the fact of having five RCTs, such evidence should not be rated high in any pyramid. The quality of evidence can also be rated up. For example, we are quite certain about the benefits of hip replacement in a patient with disabling hip osteoarthritis. Although not tested in RCTs, the quality of this evidence is rated up despite the study design (non-randomised observational studies).10

Therefore, the first modification to the pyramid is to change the straight lines separating study designs in the pyramid to wavy lines (going up and down to reflect the GRADE approach of rating up and down based on the various domains of the quality of evidence).

5Broomfield

Rationale for modification 2

Another challenge to the notion of having systematic reviews on the top of the evidence pyramid relates to the framework presented in the Journal of the American Medical Association User’s Guide on systematic reviews and meta-analysis. The Guide presented a two-step approach in which the credibility of the process of a systematic review is evaluated first (comprehensive literature search, rigorous study selection process, etc). If the systematic review was deemed sufficiently credible, then a second step takes place in which we evaluate the certainty in evidence based on the GRADE approach.11 In other words, a meta-analysis of well-conducted RCTs at low risk of bias cannot be equated with a meta-analysis of observational studies at higher risk of bias. For example, a meta-analysis of 112 surgical case series showed that in patients with thoracic aortic transection, the mortality rate was significantly lower in patients who underwent endovascular repair, followed by open repair and non-operative management (9%, 19% and 46%, respectively, p<0.01). Clearly, this meta-analysis should not be on top of the pyramid similar to a meta-analysis of RCTs. After all, the evidence remains consistent of non-randomised studies and likely subject to numerous confounders.

Therefore, the second modification to the pyramid is to remove systematic reviews from the top of the pyramid and use them as a lens through which other types of studies should be seen (ie, appraised and applied). The systematic review (the process of selecting the studies) and meta-analysis (the statistical aggregation that produces a single effect size) are tools to consume and apply the evidence by stakeholders.

Implications and limitations

Changing how systematic reviews and meta-analyses are perceived by stakeholders (patients, clinicians and stakeholders) has important implications. For example, the American Heart Association considers evidence derived from meta-analyses to have a level ‘A’ (ie, warrants the most confidence). Re-evaluation of evidence using GRADE shows that level ‘A’ evidence could have been high, moderate, low or of very low quality.12 The quality of evidence drives the strength of recommendation, which is one of the last translational steps of research, most proximal to patient care.

One of the limitations of all ‘pyramids’ and depictions of evidence hierarchy relates to the underpinning of such schemas. The construct of internal validity may have varying definitions, or be understood differently among evidence consumers. A limitation of considering systematic review and meta-analyses as tools to consume evidence may undermine their role in new discovery (eg, identifying a new side effect that was not demonstrated in individual studies13).

This pyramid can be also used as a teaching tool. EBM teachers can compare it to the existing pyramids to explain how certainty in the evidence (also called quality of evidence) is evaluated. It can be used to teach how evidence-based practitioners can appraise and apply systematic reviews in practice, and to demonstrate the evolution in EBM thinking and the modern understanding of certainty in evidence.