Evidence Based Medicine (EBM) Resources

What is EBM?

Definition of EBM

“Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” (Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-2.)

EBM Training Modules

• Module 1: Evidence-based medicine
• Module 2: Randomized controlled trials
• Module 3: Introduction to systematic reviews
• Module 4: Understanding and using systematic reviews

Glossary of EBM Terms

https://ebm-tools.knowledgetranslation.net/resource/glossary

https://jamaevidence.mhmedical.com/glossary.aspx

Components of EBM

Practical Components of EBM (AAMC Core EPA 7)

EPA 7 Toolkit

Podcast

"The 5As of the Health Information Cycle: Robert Hayward, MD, defines the 5As of the health information cycle and helps learners understand the process."

Recommended Reading

Publications with further discussion of EBM:

• Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, Shaneyfelt T, Haynes RB, Guyatt G, Glasziou P. Core Competencies in Evidence-Based Practice for Health Professionals: Consensus Statement Based on a Systematic Review and Delphi Survey. JAMA Netw Open. 2018 Jun 1;1(2):e180281. doi: 10.1001/jamanetworkopen.2018.0281. PMID: 30646073.
• Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996 Jan 13;312(7023):71-2. doi: 10.1136/bmj.312.7023.71. PMID: 8555924; PMCID: PMC2349778.
• Dowie J. Evidence based medicine. Needs to be within framework of decision making based on decision analysis. BMJ. 1996 Jul 20;313(7050):170; author reply 170-1. doi: 10.1136/bmj.313.7050.170a. PMID: 8688793; PMCID: PMC2351544.
• Maynard A. Evidence based medicine. Cost effectiveness and equity are ignored. BMJ. 1996 Jul 20;313(7050):170-1. doi: 10.1136/bmj.313.7050.170b. PMID: 8688794; PMCID: PMC2351563.
• Melnyk BM, Fineout-Overholt E, Stillwell SB, Williamson KM. Evidence-based practice: step by step: igniting a spirit of inquiry: an essential foundation for evidence-based practice. Am J Nurs. 2009 Nov;109(11):49-52. doi: 10.1097/01.NAJ.0000363354.53883.58. PMID: 19858857.

Key Elements of EBM Skills

The following are skills needed to practice evidence-based medicine. For each skill, websites, publications, and other resources will be provided.

Asking the Question

Websites

• Dartmouth Library — Evidence-Based Medicine: Asking the Clinical Question
• BMJ Best Practice — How to Clarify a Clinical Question

Publications

• Kendall S, Ryu M, Walsh C. Evidence-based medicine resources tool kit revisited. Can Fam Physician. 2017 Jun;63(6):490-492. PMID: 28615402; PMCID: PMC5471090.
• Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995 Nov-Dec;123(3):A12-3. PMID: 7582737.
• Geddes, J. (1999). Asking structured and focused clinical questions: essential first step of evidence-based practice. Evidence Based Mental Health, 2, 35-36 doi: http://dx.doi.org/10.1136/ebmh.2.2.35

The Search

Websites

• University of Oxford Centre for Evidence-Based Medicine — Finding the Evidence: A How-To Guide
• University of Oxford Centre for Evidence-Based Medicine — Practice Exercise

Publications

• Vogel EW, Block KR, Wallingford KT. Finding the evidence: teaching medical residents to search MEDLINE. J Med Libr Assoc. 2002 Jul;90(3):327-30. PMID: 12113519; PMCID: PMC116407.
• Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc. 2018 Oct;106(4):531-541. doi: 10.5195/jmla.2018.283. Epub 2018 Oct 1. PMID: 30271302; PMCID: PMC6148622.
• Vandenbroucke JP, Pearce N. From ideas to studies: how to get ideas and sharpen them into research questions. Clin Epidemiol. 2018 Mar 6;10:253-264. doi: 10.2147/CLEP.S142940. PMID: 29563838; PMCID: PMC5846748.
• Borges Migliavaca C, Stein C, Colpani V, Barker TH, Munn Z, Falavigna M; Prevalence Estimates Reviews – Systematic Review Methodology Group (PERSyst). How are systematic reviews of prevalence conducted? A methodological study. BMC Med Res Methodol. 2020 Apr 26;20(1):96. doi: 10.1186/s12874-020-00975-3. PMID: 32336279; PMCID: PMC7184711.
• Swanberg SM, Dennison CC, Farrell A, Machel V, Marton C, O'Brien KK, Pannabecker V, Thuna M, Holyoke AN. Instructional methods used by health sciences librarians to teach evidence-based practice (EBP): a systematic review. J Med Libr Assoc. 2016 Jul;104(3):197-208. doi: 10.3163/1536-5050.104.3.004. PMID: 27366120; PMCID: PMC4915637.
• Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. Int J Evid Based Healthc. 2015 Sep;13(3):147-53. doi: 10.1097/XEB.0000000000000054. PMID: 26317388.

Levels of Evidence

Websites

• Dartmouth Library — Evidence-Based Medicine: Finding the Best Evidence
• University of Oxford Centre for Evidence-Based Medicine — Levels of Evidence (March 2009)

Publications

• Jeremy Howick, Iain Chalmers, Paul Glasziou, Trish Greenhalgh, Carl Heneghan, Alessandro Liberati, Ivan Moschetti, Bob Phillips, and Hazel Thornton. “The 2011 Oxford CEBM Evidence Levels of Evidence (Introductory Document)”. Oxford Centre for Evidence-Based Medicine.
• Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 26;336(7650):924-6
• Jonas WB, Crawford C, Hilton L, Elfenbaum P. Scientific Evaluation and Review of Claims in Health Care (SEaRCH): A Streamlined, Systematic, Phased Approach for Determining "What Works" in Healthcare. J Altern Complement Med. 2017 Jan;23(1):18-25. doi: 10.1089/acm.2016.0291. Epub 2016 Dec 27. PMID: 28026968; PMCID: PMC5248545.

Understanding Summary Statistics in Clinical Studies

Websites

• JAMA Guide to Statistics and Methods
• University of Oxford Centre for Evidence-Based Medicine — Making a Decision
• Evidence-Based Medicine Toolbox — More Details on Likelihood Ratios
• Evidence-Based Medicine Toolbox — Calculation of Odds Ratio (OR) / Relative Risk (RR)
• Evidence-Based Medicine Toolbox — More Details on Sensitivity and Specificity

Critical Appraisal & Harm-Benefit (concept and in numbers)

Useful Resources

• Tips for learning and teaching evidence-based medicine:
  • https://www.cmaj.ca/content/171/4/347
  • https://pubmed.ncbi.nlm.nih.gov/15313994/
• Tips for teachers of evidence-based medicine: Relative risk reduction, absolute risk reduction and number needed to treat
  • https://www.cmaj.ca/content/suppl/2005/04/15/171.4.353.DC1/
  • https://pubmed.ncbi.nlm.nih.gov/15313996/
• Tips for learners of evidence-based medicine: 2. Measures of precision (confidence intervals)
  • https://www.cmaj.ca/content/171/6/611
• Critical appraisal checklists:
  • https://researchguides.dartmouth.edu/c.php?g=944702&p=6833650
• Weighing up risks and benefits:
  • https://bestpractice.bmj.com/info/us/toolkit/practise-ebm/weighing-up-risks-and-benefits/

Publications

• Montori VM, Jaeschke R, Schünemann HJ, Bhandari M, Brozek JL, Devereaux PJ, Guyatt GH. Users' guide to detecting misleading claims in clinical research reports. BMJ. 2004 Nov 6;329(7474):1093-6. doi: 10.1136/bmj.329.7474.1093. PMID: 15528623; PMCID: PMC526126.
• Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2613-2619. doi: 10.1073/pnas.1710755115. PMID: 29531025; PMCID: PMC5856510.
• Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev. 2011 Nov 9;2011(11):CD001270. doi: 10.1002/14651858.CD001270.pub2. PMID: 22071800; PMCID: PMC7389530.
• McGee S. Simplifying likelihood ratios. J Gen Intern Med. 2002 Aug;17(8):646-9. doi: 10.1046/j.1525-1497.2002.10750.x. PMID: 12213147; PMCID: PMC1495095.
• Greenhalgh T, Kostopoulou O, Harries C. Making decisions about benefits and harms of medicines. BMJ. 2004 Jul 3;329(7456):47-50. doi: 10.1136/bmj.329.7456.47. PMID: 15231628; PMCID: PMC443458.
• Aronson Jeff, . Good prescribing: benefits, hazards, harms, and risks BMJ 2016; 352 :i537

Online Calculators

• Measures of risk increase/decrease and number needed to treat or harm (Dr. Alan Schwartz)
• Effect size calculator (Center for Evaluation and Monitoring)
• EBM calculators
• Number needed to treat/Likelihood ratios for many conditions

Short Articles that Explain Clinical Stat Concepts

• AKT (Applied Knowledge Test) Statistics Topics
• Odds Ratios Explained
• Odds vs. Risks
• Meta-Analysis Stats

Applying the Evidence

Websites

• Shared Decision-Making Resources

Publications

• McAlister FA, Straus SE, Guyatt GH, Haynes RB. Users' guides to the medical literature: XX. Integrating research evidence with the care of the individual patient. Evidence-Based Medicine Working Group. JAMA. 2000 Jun 7;283(21):2829-36. doi: 10.1001/jama.283.21.2829. PMID: 10838653.
• McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA. 2000 Jul 5;284(1):79-84. doi: 10.1001/jama.284.1.79. PMID: 10872017.

Evaluation

University of Canberra — Evidence-Based Practice in Health: Audit

How to Build Brief EBM Exercises for the Classroom or Small Groups

Online Resources

• EBM Worksheets
• EBM Teaching Materials

External Examples of EBM Work-Throughs

• Can drinking cola help students concentrate? / Blueberries and other foods rich in anti-oxidants could be used to increase brain power when taking exam
• Formulating the question
• Examples of cases that include critical appraisal of evidence
• Syllabi for EBM across specialties – many with case studies
• Discussion of EBM along with case studies where EBM alone is not enough for decision-making:
  Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS. Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group. JAMA. 2000 Sep 13;284(10):1290-6. doi: 10.1001/jama.284.10.1290. PMID: 10979117.

Publications with Cases, Exercises or Multiple Choice Questions

• Al Achkar M, Davies MK. A small group learning model for evidence-based medicine. Adv Med Educ Pract. 2016 Oct 25;7:611-615. doi: 10.2147/AMEP.S117672. PMID: 27822132; PMCID: PMC5087589.
• Patell R, Raska P, Lee N, Luciano G, DiNardo DJ, Baduashvili A, Anderson ML, Merritt F, Rothberg MB. Development and Validation of a Test for Competence in Evidence-Based Medicine. J Gen Intern Med. 2020 May;35(5):1530-1536. doi: 10.1007/s11606-019-05595-2. Epub 2019 Dec 17. PMID: 31848856; PMCID: PMC7210361.
• Leung WC. Multiple choice questions in evidence based medicine. Postgrad Med J. 2000 Sep;76(899):594-5. doi: 10.1136/pmj.76.899.594. PMID: 10964140; PMCID: PMC1741721.
• Hift RJ. Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014 Nov 28;14:249. doi: 10.1186/s12909-014-0249-2. PMID: 25431359; PMCID: PMC4275935.

General Discussion about Available Tools to Assess EBM Skill (with some examples):

• Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. Development and validation of the ACE tool: assessing medical trainees' competency in evidence based medicine. BMC Med Educ. 2014 Jun 9;14:114. doi: 10.1186/1472-6920-14-114. PMID: 24909434; PMCID: PMC4062508.
• Ilic D. Assessing competency in Evidence Based Practice: strengths and limitations of current tools in practice. BMC Med Educ. 2009 Aug 6;9:53. doi: 10.1186/1472-6920-9-53. PMID: 19656412; PMCID: PMC2728711.
• Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C, Green M. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006 Sep 6;296(9):1116-27. doi: 10.1001/jama.296.9.1116. PMID: 16954491.
• Albarqouni L, Hoffmann T, Glasziou P. Evidence-based practice educational intervention studies: a systematic review of what is taught and how it is measured. BMC Med Educ. 2018 Aug 1;18(1):177. doi: 10.1186/s12909-018-1284-1. PMID: 30068343; PMCID: PMC6090869.
• Kumaravel B, Jenkins H, Chepkin S, Kirisnathas S, Hearn J, Stocker CJ, Petersen S. A prospective study evaluating the integration of a multifaceted evidence-based medicine curriculum into early years in an undergraduate medical school. BMC Med Educ. 2020 Aug 24;20(1):278. doi: 10.1186/s12909-020-02140-2. PMID: 32838775; PMCID: PMC7445898.
• Bird JB, Olvet DM, Willey JM, Brenner J. Patients don't come with multiple choice options: essay-based assessment in UME. Med Educ Online. 2019 Dec;24(1):1649959. doi: 10.1080/10872981.2019.1649959. PMID: 31438809; PMCID: PMC6720218.
• Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003 Feb 8;326(7384):319-21. doi: 10.1136/bmj.326.7384.319. PMID: 12574047; PMCID: PMC143529.

Models and Benefit of Longitudinal Integration of EBM into Curriculum

• Kumaravel B, Jenkins H, Chepkin S, Kirisnathas S, Hearn J, Stocker CJ, Petersen S. A prospective study evaluating the integration of a multifaceted evidence-based medicine curriculum into early years in an undergraduate medical school. BMC Med Educ. 2020 Aug 24;20(1):278. doi: 10.1186/s12909-020-02140-2. PMID: 32838775; PMCID: PMC7445898.
• West CP, Jaeger TM, McDonald FS. Extended evaluation of a longitudinal medical school evidence-based medicine curriculum. J Gen Intern Med. 2011 Jun;26(6):611-5. doi: 10.1007/s11606-011-1642-8. PMID: 21286836; PMCID: PMC3101983.
• Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014 Jan 28;9(1):e86706. doi: 10.1371/journal.pone.0086706. PMID: 24489771; PMCID: PMC3904944.
• Menard L, Blevins AE, Trujillo DJ, Lazarus KH. (IV)Integrating evidence-based medicine skills into a medical school curriculum: a quantitative outcomes assessment. BMJ Evid Based Med. 2020 Oct 22:bmjebm-2020-111391. doi: 10.1136/bmjebm-2020-111391. Epub ahead of print. PMID: 33093190.
• Martin BA, Kraus CK, Kim SY. Longitudinal teaching of evidence-based decision making. Am J Pharm Educ. 2012 Dec 12;76(10):197. doi: 10.5688/ajpe7610197. PMID: 23275662; PMCID: PMC3530059.
• Lupi C, Lefevre F, Ward-Peterson M, 2017, 'Evidence-based medicine skills that last: A transferable model utilizing integration, spaced learning, and repetition with a single study design among second-year medical students', MedEdPublish, 6, [4], 52.

Additional Resources

• Centre for Evidence-Based Medicine, University of Oxford
• CEBM Set of tools for EBM and EBM reviews of many topics
  • Xia AD, Schaefer CP, Szende A, Jahn E, Hirst MJ. RWE Framework: An Interactive Visual Tool to Support a Real-World Evidence Study Design. Drugs Real World Outcomes. 2019 Dec;6(4):193-203. doi: 10.1007/s40801-019-00167-6. PMID: 31741199; PMCID: PMC6879703.Real-World Evidence (RWE) Tools
  • RWE Framework
  • FDA and RWE sources
• Visual Rx examples
• JBI - Many resources and tools
• E-book: Evidence-Based Medicine: How to Practice and Teach EBM
  Straus, Sharon E. author.; Glasziou, Paul, 1954- author.; Richardson, W. Scott (Warren Scott), author.; Haynes, R. Brian, author.; Pattani, Reena, contributor.; Veroniki, Areti Angeliki, contributor. Edinburgh : Elsevier, 2019