Skip to main content
Download PDF

Artificial intelligence and echocardiography

Echo Research & Practice volume 5, pages R115–R125 (2018)Cite this article

Abstract

Echocardiography plays a crucial role in the diagnosis and management of cardiovascular disease. However, interpretation remains largely reliant on the subjective expertise of the operator. As a result inter-operator variability and experience can lead to incorrect diagnoses. Artificial intelligence (AI) technologies provide new possibilities for echocardiography to generate accurate, consistent and automated interpretation of echocardiograms, thus potentially reducing the risk of human error. In this review, we discuss a subfield of AI relevant to image interpretation, called machine learning, and its potential to enhance the diagnostic performance of echocardiography. We discuss recent applications of these methods and future directions for AI-assisted interpretation of echocardiograms. The research suggests it is feasible to apply machine learning models to provide rapid, highly accurate and consistent assessment of echocardiograms, comparable to clinicians. These algorithms are capable of accurately quantifying a wide range of features, such as the severity of valvular heart disease or the ischaemic burden in patients with coronary artery disease. However, the applications and their use are still in their infancy within the field of echocardiography. Research to refine methods and validate their use for automation, quantification and diagnosis are in progress. Widespread adoption of robust AI tools in clinical echocardiography practice should follow and have the potential to deliver significant benefits for patient outcome.

References

  1. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, Picard MH, Polk DM, Ragosta M, Parker Ward R, et al. ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography. A Report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Society of Echocardiography, American Heart Association, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance American College of Chest Physicians. Journal of the American Society of Echocardiography 2011 24 229–267. (https://doi.org/10.1016/j.echo.2010.12.008)

    PubMed  Google Scholar 

  2. Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, Flachskampf FA, Foster E, Goldstein SA, Kuznetsova T, et al. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Journal of the American Society of Echocardiography 2015 28 1.e14–39.e14. (https://doi.org/10.1093/ehjci/jew041)

    Google Scholar 

  3. Hoffmann R, Lethen H, Marwick T, Arnese M, Fioretti P, Pingitore A, Picano E, Buck T, Erbel R, Flachskampf FA, et al. Analysis of interinstitutional observer agreement in interpretation of dobutamine stress echocardiograms. Journal of the American College of Cardiology 1996 27 330–336. (https://doi.org/10.1016/0735-1097(95)00483-1)

    CAS  PubMed  Google Scholar 

  4. Lee JG, Jun S, Cho YW, Lee H, Kim GB, Seo JB & Kim N. Deep learning in medical imaging: general overview. Korean Journal of Radiology 2017 18 570–584. (https://doi.org/10.3348/kjr.2017.18.4.570)

    PubMed  PubMed Central  Google Scholar 

  5. Narula S, Shameer K, Salem Omar AM, Dudley JT & Sengupta PP. Machine-learning algorithms to automate morphological and functional assessments in 2D echocardiography. Journal of the American College of Cardiology 2016 68 2287–2295. (https://doi.org/10.1016/j.jacc.2016.08.062)

    PubMed  Google Scholar 

  6. Jeganathan J, Knio Z, Amador Y, Hai T, Khamooshian A, Matyal R, Khabbaz KR & Mahmood F. Artificial intelligence in mitral valve analysis. Annals of Cardiac Anaesthesia 2017 20 129–134. (https://doi.org/10.4103/aca.ACA_243_16)

    PubMed  PubMed Central  Google Scholar 

  7. Madani A, Arnaout R, Mofrad M & Arnaout R. Fast and accurate view classification of echocardiograms using deep learning. npj Digital Medicine 2018 1 6. (https://doi.org/10.1038/s41746-017-0013-1)

    PubMed  PubMed Central  Google Scholar 

  8. Mayr A, Binder H, Gefeller O & Schmid M. The evolution of boosting algorithms. From machine learning to statistical modelling. Methods of Information in Medicine 2014 53 419–427. (https://doi.org/10.3414/ME13-01-0122)

    CAS  PubMed  Google Scholar 

  9. Shameer K, Johnson KW, Glicksberg BS, Dudley JT & Sengupta PP. Machine learning in cardiovascular medicine: are we there yet? Heart 2018 104 1156–1164. (https://doi.org/10.1136/heartjnl-2017-311198)

    PubMed  Google Scholar 

  10. Fatima M & Pasha M. Survey of machine learning algorithms for disease diagnostic. Journal of Intelligent Learning Systems and Applications 2017 9 1–16. (https://doi.org/10.4236/jilsa.2017.91001)

    Google Scholar 

  11. Krittanawong C, Tunhasiriwet A, Zhang H, Wang Z, Aydar M & Kitai T. Deep learning with unsupervised feature in echocardiographic imaging. Journal of the American College of Cardiology 2017 69 2100–2101. (https://doi.org/10.1016/j.jacc.2016.12.047)

    PubMed  Google Scholar 

  12. Sahba F, Tizhoosh HR & Salama MMA. A reinforcement learning framework for medical image segmentation. The 2006 IEEE International Joint Conference on Neural Network Proceedings. 2006. (https://doi.org/10.1109/IJCNN.2006.246725)

    Google Scholar 

  13. Barto RSSaAG. Reinforcement Learning: An Introduction, 2nd edn. Cambridge, MA, USA; London, UK: MIT Press, 2014-2015.

  14. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature 2015 518 529–533. (https://doi.org/10.1038/nature14236)

    CAS  PubMed  Google Scholar 

  15. Kubat M. An Introduction to Machine Learning. Cham, Switzerland: Springer International Publishing, 2017.

    Google Scholar 

  16. Awan SE, Sohel F, Sanfilippo FM, Bennamoun M & Dwivedi G. Machine learning in heart failure: ready for prime time. Current Opinion in Cardiology 2018 33 190–195. (https://doi.org/10.1097/HCO.0000000000000491)

    PubMed  Google Scholar 

  17. Marcus G. Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631, 2018.

    Google Scholar 

  18. Cabitza F, Rasoini R & Gensini GF. Unintended consequences of machine learning in medicine. JAMA 2017 318 517–518. (https://doi.org/10.1001/jama.2017.7797)

    PubMed  Google Scholar 

  19. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM & Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017 542 115–118. (https://doi.org/10.1038/nature21056)

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Wolterink JM, Leiner T, Takx RA, Viergever MA & Isgum I. Automatic coronary calcium scoring in non-contrast-enhanced ECG-triggered cardiac CT with ambiguity detection. IEEE Transactions on Medical Imaging 2015 34 1867–1878. (https://doi.org/10.1109/TMI.2015.2412651)

    PubMed  Google Scholar 

  21. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016 316 2402–2410. (https://doi.org/10.1001/jama.2016.17216)

    PubMed  Google Scholar 

  22. Cobey FC, Patel V, Gosling A & Ursprung E. The emperor has no clothes: recognizing the limits of current echocardiographic technology in perioperative quantification of mitral regurgitation. Journal of Cardiothoracic and Vascular Anesthesia 2017 31 1692–1694. (https://doi.org/10.1053/j.jvca.2017.03.012)

    PubMed  Google Scholar 

  23. Nishimura RA, Otto CM, Bonow RO, Carabello BA, Erwin JP 3rd, Guyton RA, O’Gara PT, Ruiz CE, Skubas NJ, Sorajja P, et al. 2014 AHA/ACC Guideline for the Management of Patients With Valvular Heart Disease: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014 129 2440–2492. (https://doi.org/10.1161/CIR.0000000000000029)

    PubMed  Google Scholar 

  24. Knackstedt C, Bekkers SC, Schummers G, Schreckenberg M, Muraru D, Badano LP, Franke A, Bavishi C, Omar AM & Sengupta PP. Fully automated versus standard tracking of left ventricular ejection fraction and longitudinal strain: the FAST-EFs multicenter study. Journal of the American College of Cardiology 2015 66 1456–1466. (https://doi.org/10.1016/j.jacc.2015.07.052)

    PubMed  Google Scholar 

  25. Thavendiranathan P, Popović ZB, Flamm SD, Dahiya A, Grimm RA & Marwick TH. Improved interobserver variability and accuracy of echocardiographic visual left ventricular ejection fraction assessment through a self-directed learning program using cardiac magnetic resonance images. Journal of the American Society of Echocardiography 2013 26 1267–1273. (https://doi.org/10.1016/j.echo.2013.07.017)

    PubMed  Google Scholar 

  26. Johnson KW, Soto JT, Glicksberg BS, Shameer K, Miotto R, Ali M, Ashley E & Dudley JT. Artificial intelligence in cardiology. Journal of the American College of Cardiology 2018 71 2668–2679. (https://doi.org/10.1016/j.jacc.2018.03.521)

    PubMed  Google Scholar 

  27. Sengupta PP, Huang YM, Bansal M, Ashrafi A, Fisher M, Shameer K, Gall W & Dudley JT. Cognitive machine-learning algorithm for cardiac imaging: a pilot study for differentiating constrictive pericarditis from restrictive cardiomyopathy. Circulation: Cardiovascular Imaging 2016 9 e004330. (https://doi.org/10.1161/CIRCIMAGING.115.004330)

    Google Scholar 

  28. Levy F, Dan Schouver E, Iacuzio L, Civaia F, Rusek S, Dommerc C, Marechaux S, Dor V, Tribouilloy C & Dreyfus G. Performance of new automated transthoracic three-dimensional echocardiographic software for left ventricular volumes and function assessment in routine clinical practice: Comparison with 3 Tesla cardiac magnetic resonance. Archives of Cardiovascular Diseases 2017 110 580–589. (https://doi.org/10.1016/j.acvd.2016.12.015)

    PubMed  Google Scholar 

  29. Chao PK, Wang CL & Chan HL. An intelligent classifier for prognosis of cardiac resynchronization therapy based on speckle-tracking echocardiograms. Artificial Intelligence in Medicine 2012 54 181–188. (https://doi.org/10.1016/j.artmed.2011.09.006)

    PubMed  Google Scholar 

  30. Mahmoud A, Bansal M & Sengupta PP. New cardiac imaging algorithms to diagnose constrictive pericarditis versus restrictive cardiomyopathy. Current Cardiology Reports 2017 19 43. (https://doi.org/10.1007/s11886-017-0851-0)

    PubMed  Google Scholar 

  31. Arsanjani R, Dey D, Khachatryan T, Shalev A, Hayes SW, Fish M, Nakanishi R, Germano G, Berman DS & Slomka P. Prediction of revascularization after myocardial perfusion SPECT by machine learning in a large population. Journal of Nuclear Cardiology 2015 22 877–884. (https://doi.org/10.1007/s12350-014-0027-x)

    PubMed  Google Scholar 

  32. Haro Alonso D, Wernick MN, Yang Y, Germano G, Berman DS & Slomka P. Prediction of cardiac death after adenosine myocardial perfusion SPECT based on machine learning. Journal of Nuclear Cardiology 2018 [epub]. (https://doi.org/10.1007/s12350-018-1250-7)

    Google Scholar 

  33. Betancur J, Otaki Y, Motwani M, Fish MB, Lemley M, Dey D, Gransar H, Tamarappoo B, Germano G, Sharir T, et al. Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning. JACC: Cardiovascular Imaging 2018 11 1000–1009. (https://doi.org/10.1016/j.jcmg.2017.07.024)

    PubMed  Google Scholar 

  34. DeCara JM, Lang RM, Koch R, Bala R, Penzotti J & Spencer KT. The use of small personal ultrasound devices by internists without formal training in echocardiography. European Journal of Echocardiography 2003 4 141–147. (https://doi.org/10.1053/euje.4.2.141)

    CAS  PubMed  Google Scholar 

  35. Narula J. Are we up to speed?: from big data to rich insights in CV imaging for a hyperconnected world. JACC: Cardiovascular Imaging 2013 6 1222–1224. (https://doi.org/10.1016/j.jcmg.2013.09.007)

    PubMed  Google Scholar 

  36. Khamis H, Zurakhov G, Azar V, Raz A, Friedman Z & Adam D. Automatic apical view classification of echocardiograms using a discriminative learning dictionary. Medical Image Analysis 2017 36 15–21. (https://doi.org/10.1016/j.media.2016.10.007)

    PubMed  Google Scholar 

  37. Gregg Belous AB & Rowlands D. Segmentation of the left ventricle from ultrasound using random forest with active shape model. In Artificial Intelligence, Modelling and Simulation (AIMS). Kota Kinabalu, Malaysia: IEEE, 2013. (https://doi.org/10.1109/AIMS.2013.58)

    Google Scholar 

  38. Cannesson M, Tanabe M, Suffoletto MS, McNamara DM, Madan S, Lacomis JM, Gorcsan J 3rd. A novel two-dimensional echocardiographic image analysis system using artificial intelligence-learned pattern recognition for rapid automated ejection fraction. Journal of the American College of Cardiology 2007 49 217–226. (https://doi.org/10.1016/j.jacc.2006.08.045)

    PubMed  Google Scholar 

  39. Rahmouni HW, Ky B, Plappert T, Duffy K, Wiegers SE, Ferrari VA, Keane MG, Kirkpatrick JN, Silvestry FE, St John Sutton M. Clinical utility of automated assessment of left ventricular ejection fraction using artificial intelligence-assisted border detection. American Heart Journal 2008 155 562–570. (https://doi.org/10.1016/j.ahj.2007.11.002)

    PubMed  Google Scholar 

  40. Alsharqi M, Upton R, Mumith A & Leeson P. Artificial intelligence: a new clinical support tool for stress echocardiography. Expert Review of Medical Devices 2018 15 513–515. (https://doi.org/10.1080/17434440.2018.1497482)

    CAS  PubMed  Google Scholar 

  41. Domingos JS, Stebbing RV, Leeson P, Noble JA. Structured Random Forests for Myocardium Delineation in 3D Echocardiography. Cham, Switzerland: Springer International Publishing, 2014. (https://doi.org/10.1007/978-3-319-10581-9_27)

    Google Scholar 

  42. Stebbing RV, Namburete AI, Upton R, Leeson P & Noble JA. Data-driven shape parameterization for segmentation of the right ventricle from 3D+t echocardiography. Medical Image Analysis 2015 21 29–39. (https://doi.org/10.1016/j.media.2014.12.002)

    PubMed  Google Scholar 

  43. Tsang W, Salgo IS, Medvedofsky D, Takeuchi M, Prater D, Weinert L, Yamat M, Mor-Avi V, Patel AR & Lang RM. Transthoracic 3D echocardiographic left heart chamber quantification using an automated adaptive analytics algorithm. JACC: Cardiovascular Imaging 2016 9 769–782. (https://doi.org/10.1016/j.jcmg.2015.12.020)

    PubMed  Google Scholar 

  44. Otani K, Nakazono A, Salgo IS, Lang RM & Takeuchi M. Three-dimensional echocardiographic assessment of left heart chamber size and function with fully automated quantification software in patients with atrial fibrillation. Journal of the American Society of Echocardiography 2016 29 955–965. (https://doi.org/10.1016/j.echo.2016.06.010)

    PubMed  Google Scholar 

  45. Bernard O, Bosch JG, Heyde B, Alessandrini M, Barbosa D, Camarasu-Pop S, Cervenansky F, Valette S, Mirea O, Bernier M, et al. Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography. IEEE Transactions on Medical Imaging 2016 35 967–977. (https://doi.org/10.1109/TMI.2015.2503890)

    PubMed  Google Scholar 

  46. Cobey FC. Intelligent algorithms in perioperative echocardiography: a new era. Journal of the American Society of Echocardiography 2017 30 A26–A27. (https://doi.org/10.1016/j.echo.2017.08.008)

    PubMed  Google Scholar 

  47. Moghaddasi H & Nourian S. Automatic assessment of mitral regurgitation severity based on extensive textural features on 2D echocardiography videos. Computers in Biology and Medicine 2016 73 47–55. (https://doi.org/10.1016/j.compbiomed.2016.03.026)

    PubMed  Google Scholar 

  48. Prihadi EA, van Rosendael PJ, Vollema EM, Bax JJ, Delgado V, Ajmone Marsan N. Feasibility, accuracy, and reproducibility of aortic annular and root sizing for transcatheter aortic valve replacement using novel automated three-dimensional echocardiographic software: comparison with multi-detector row computed tomography. Journal of the American Society of Echocardiography 2018 31 505.e3–514.e3. (https://doi.org/10.1016/j.echo.2017.10.003)

    Google Scholar 

  49. Queirós S, Papachristidis A, Morais P, Theodoropoulos KC, Fonseca JC, Monaghan MJ, Vilaca JL, Fully Automatic Dhooge J. 3-D-TEE segmentation for the planning of transcatheter aortic valve implantation. IEEE Transactions on Biomedical Engineering 2017 64 1711–1720. (https://doi.org/10.1109/TBME.2016.2617401)

  50. Queiros S, Morais P, Dubois C, Voigt JU, Fehske W, Kuhn A, Achenbach T, Fonseca JC, Vilaça JL, D’hooge J. Validation of a novel software tool for automatic aortic annular sizing in three-dimensional transesophageal echocardiographic images. Journal of the American Society of Echocardiography 2018 31 515.e5–525.e5. (https://doi.org/10.1016/j.echo.2018.01.007)

    Google Scholar 

  51. Ladapo JA, Blecker S & Douglas PS. Physician decision making and trends in the use of cardiac stress testing in the United States: an analysis of repeated cross-sectional data. Annals of Internal Medicine 2014 161 482–490. (https://doi.org/10.7326/M14-0296)

    PubMed  PubMed Central  Google Scholar 

  52. Geleijnse ML, Krenning BJ, van Dalen BM, Nemes A, Soliman OI, Bosch JG, Galema TW, ten Cate FJ & Boersma E. Factors affecting sensitivity and specificity of diagnostic testing: dobutamine stress echocardiography. Journal of the American Society of Echocardiography 2009 22 1199–1208. (https://doi.org/10.1016/j.echo.2009.07.006)

    PubMed  Google Scholar 

  53. Chykeyuk K, Clifton DA & Noble JA. Feature extraction and wall motion classification of 2D stress echocardiography with relevance vector machines. In Proceedings - International Symposium on Biomedical Imaging, 2011. (https://doi.org/10.1109/ISBI.2011.5872497)

    Google Scholar 

  54. Raghavendra U, Fujita H, Gudigar A, Shetty R, Nayak K, Pai U, Samanth J & Acharya UR. Automated technique for coronary artery disease characterization and classification using DD-DTDWT in ultrasound images. Biomedical Signal Processing and Control 2018 40 324–334. (https://doi.org/10.1016/j.bspc.2017.09.030)

    Google Scholar 

  55. Omar HA, Domingos JS, Patra A, Upton R, Leeson P & Noble JA. Quantification of cardiac bull’s-eye map based on principal strain analysis for myocardial wall motion assessment in stress echocardiography. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018. (https://doi.org/10.1109/ISBI.2018.8363785)

    Google Scholar 

  56. Mansor S, Hughes NP & Noble JA. Wall motion classification of stress echocardiography based on combined rest-and-stress data. Medical Image Computing and Computer-Assisted Intervention 2008 11 139–146. (https://doi.org/10.1007/978-3-540-85990-1_17)

    PubMed  Google Scholar 

  57. Sanchez-Martinez S, Duchateau N, Erdei T, Fraser AG, Bijnens BH & Piella G. Characterization of myocardial motion patterns by unsupervised multiple kernel learning. Medical Image Analysis 2017 35 70–82. (https://doi.org/10.1016/j.media.2016.06.007)

    PubMed  Google Scholar 

Download references

Funding

This research did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.

Author information

Authors and Affiliations

  1. Oxford Cardiovascular Clinical Research Facility, Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, Oxford, UK

    M. Alsharqi MSc, W. J. Woodward BSc & P. Leeson PhD FRCP

  2. Ultromics Ltd, Magdalen Centre, Robert Robinson Ave, Oxford, UK

    J. A. Mumith PhD, D. C. Markham PhD & R. Upton MSc

Authors
  1. M. Alsharqi MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. J. Woodward BSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. A. Mumith PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. C. Markham PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. Upton MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. P. Leeson PhD FRCP
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Leeson PhD FRCP.

Rights and permissions

This work is licensed under a Creative Commons Attribution 4.0 International License.(http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alsharqi, M., Woodward, W.J., Mumith, J.A. et al. Artificial intelligence and echocardiography. Echo Res Pract 5, R115–R125 (2018). https://doi.org/10.1530/ERP-18-0056

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1530/ERP-18-0056

Key Words

Echo Research & Practice

ISSN: 2055-0464

Contact us