Advertisement

Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs

  • Author Footnotes
    ∗ These authors contributed equally as first authors.
    Zhixi Li
    Footnotes
    ∗ These authors contributed equally as first authors.
    Affiliations
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou 510060, China
    Search for articles by this author
  • Author Footnotes
    ∗ These authors contributed equally as first authors.
    Yifan He
    Footnotes
    ∗ These authors contributed equally as first authors.
    Affiliations
    Guangzhou Healgoo Interactive Medical Technology Co. Ltd., Guangzhou, China
    Search for articles by this author
  • Author Footnotes
    ∗ These authors contributed equally as first authors.
    Stuart Keel
    Footnotes
    ∗ These authors contributed equally as first authors.
    Affiliations
    Centre for Eye Research Australia; Departments of Ophthalmology and Surgery, University of Melbourne, Melbourne, Australia
    Search for articles by this author
  • Wei Meng
    Affiliations
    Guangzhou Healgoo Interactive Medical Technology Co. Ltd., Guangzhou, China
    Search for articles by this author
  • Robert T. Chang
    Affiliations
    Department of Ophthalmology, Byers Eye Institute, Stanford University, Palo Alto, California
    Search for articles by this author
  • Mingguang He
    Correspondence
    Correspondence: Mingguang He, MD, PhD, Stata Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou 510060, People's Republic of China.
    Affiliations
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou 510060, China

    Centre for Eye Research Australia; Departments of Ophthalmology and Surgery, University of Melbourne, Melbourne, Australia
    Search for articles by this author
  • Author Footnotes
    ∗ These authors contributed equally as first authors.

      Purpose

      To assess the performance of a deep learning algorithm for detecting referable glaucomatous optic neuropathy (GON) based on color fundus photographs.

      Design

      A deep learning system for the classification of GON was developed for automated classification of GON on color fundus photographs.

      Participants

      We retrospectively included 48 116 fundus photographs for the development and validation of a deep learning algorithm.

      Methods

      This study recruited 21 trained ophthalmologists to classify the photographs. Referable GON was defined as vertical cup-to-disc ratio of 0.7 or more and other typical changes of GON. The reference standard was made until 3 graders achieved agreement. A separate validation dataset of 8000 fully gradable fundus photographs was used to assess the performance of this algorithm.

      Main Outcome Measures

      The area under receiver operator characteristic curve (AUC) with sensitivity and specificity was applied to evaluate the efficacy of the deep learning algorithm detecting referable GON.

      Results

      In the validation dataset, this deep learning system achieved an AUC of 0.986 with sensitivity of 95.6% and specificity of 92.0%. The most common reasons for false-negative grading (n = 87) were GON with coexisting eye conditions (n = 44 [50.6%]), including pathologic or high myopia (n = 37 [42.6%]), diabetic retinopathy (n = 4 [4.6%]), and age-related macular degeneration (n = 3 [3.4%]). The leading reason for false-positive results (n = 480) was having other eye conditions (n = 458 [95.4%]), mainly including physiologic cupping (n = 267 [55.6%]). Misclassification as false-positive results amidst a normal-appearing fundus occurred in only 22 eyes (4.6%).

      Conclusions

      A deep learning system can detect referable GON with high sensitivity and specificity. Coexistence of high or pathologic myopia is the most common cause resulting in false-negative results. Physiologic cupping and pathologic myopia were the most common reasons for false-positive results.

      Abbreviations and Acronyms:

      AMD (age related macular degeneration), AUC (area under receiver operator characteristic curve), DD (disc diameter), GON (glaucomatous optic neuropathy), RNFL (retinal nerve fiber layer), VCDR (vertical cup to disc ratio)
      To read this article in full you will need to make a payment

      Subscribe:

      Subscribe to Ophthalmology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Stevens G.A.
        • White R.A.
        • Flaxman S.R.
        • et al.
        Global prevalence of vision impairment and blindness: magnitude and temporal trends, 1990–2010.
        Ophthalmology. 2013; 120: 2377-2384
        • Bourne R.R.
        • Stevens G.A.
        • White R.A.
        • et al.
        Causes of vision loss worldwide, 1990–2010: a systematic analysis.
        Lancet Glob Health. 2013; 1: e339-e349
        • Tham Y.C.
        • Li X.
        • Wong T.Y.
        • et al.
        Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis.
        Ophthalmology. 2014; 121: 2081-2090
        • Hood D.C.
        • Raza A.S.
        • de Moraes C.G.
        • et al.
        Glaucomatous damage of the macula.
        Prog Retin Eye Res. 2013; 32: 1-21
        • Tatham A.J.
        • Weinreb R.N.
        • Medeiros F.A.
        Strategies for improving early detection of glaucoma: the combined structure-function index.
        Clin Ophthalmol. 2014; 8: 611-621
        • Tatham A.J.
        • Medeiros F.A.
        • Zangwill L.M.
        • Weinreb R.N.
        Strategies to improve early diagnosis in glaucoma.
        Prog Brain Res. 2015; 221: 103-133
        • Baskaran M.
        • Foo R.C.
        • Cheng C.Y.
        • et al.
        The prevalence and types of glaucoma in an urban Chinese population: the Singapore Chinese Eye Study.
        JAMA Ophthalmol. 2015; 133: 874-880
        • Varma R.
        • Ying-Lai M.
        • Francis B.A.
        • et al.
        Prevalence of open-angle glaucoma and ocular hypertension in Latinos: the Los Angeles Latino Eye Study.
        Ophthalmology. 2004; 111: 1439-1448
        • Quigley H.A.
        • West S.K.
        • Rodriguez J.
        • et al.
        The prevalence of glaucoma in a population-based study of Hispanic subjects: Proyecto VER.
        Arch Ophthalmol. 2001; 119: 1819-1826
        • Gupta P.
        • Zhao D.
        • Guallar E.
        • et al.
        Prevalence of glaucoma in the United States: the 2005–2008 National Health and Nutrition Examination Survey.
        Invest Ophthalmol Vis Sci. 2016; 57: 2577-2585
        • Munoz B.
        • West S.K.
        • Rubin G.S.
        • et al.
        Causes of blindness and visual impairment in a population of older Americans: the Salisbury Eye Evaluation Study.
        Arch Ophthalmol. 2000; 118: 819-825
        • Shaikh Y.
        • Yu F.
        • Coleman A.L.
        Burden of undetected and untreated glaucoma in the United States.
        Am J Ophthalmol. 2014; 158: 1121-1129
        • Lee P.P.
        • Walt J.G.
        • Doyle J.J.
        • et al.
        A multicenter, retrospective pilot study of resource use and costs associated with severity of disease in glaucoma.
        Arch Ophthalmol. 2006; 124: 12-19
        • Kirwan J.F.
        • Gouws P.
        • Linnell A.E.
        • et al.
        Pharmacological mydriasis and optic disc examination.
        Br J Ophthalmol. 2000; 84: 894-898
        • Chan H.H.
        • Ong D.N.
        • Kong Y.X.
        • et al.
        Glaucomatous optic neuropathy evaluation (GONE) project: the effect of monoscopic versus stereoscopic viewing conditions on optic nerve evaluation.
        Am J Ophthalmol. 2014; 157: 936-944
        • Moyer V.A.
        Screening for glaucoma: U.S. Preventive Services Task Force Recommendation Statement.
        Ann Intern Med. 2013; 159: 484-489
        • Fleming C.
        • Whitlock E.P.
        • Beil T.
        • et al.
        Screening for primary open-angle glaucoma in the primary care setting: an update for the US preventive services task force.
        Ann Fam Med. 2005; 3: 167-170
        • Miller S.E.
        • Thapa S.
        • Robin A.L.
        • et al.
        Glaucoma screening in Nepal: cup to disc estimate with standard mydriatic fundus camera compared to portable non-mydriatic camera.
        Am J Ophthalmol. 2017; 182: 99-106
        • Pizzi L.T.
        • Waisbourd M.
        • Hark L.
        • et al.
        Costs of a community-based glaucoma detection programme: analysis of the Philadelphia Glaucoma Detection and Treatment Project.
        Br J Ophthalmol. 2018; 102: 225-232
        • Zhao D.
        • Guallar E.
        • Gajwani P.
        • et al.
        Optimizing glaucoma screening in high-risk population: design and 1-Year findings of the Screening to Prevent (SToP) Glaucoma Study.
        Am J Ophthalmol. 2017; 180: 18-28
        • Gargeya R.
        • Leng T.
        Automated identification of diabetic retinopathy using deep learning.
        Ophthalmology. 2017; 124: 962-969
        • Gulshan V.
        • Peng L.
        • Coram M.
        • et al.
        Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
        JAMA. 2016; 316: 2402-2410
        • Iwase A.
        • Suzuki Y.
        • Araie M.
        • et al.
        The prevalence of primary open-angle glaucoma in Japanese: the Tajimi Study.
        Ophthalmology. 2004; 111: 1641-1648
        • He M.
        • Foster P.J.
        • Ge J.
        • et al.
        Prevalence and clinical characteristics of glaucoma in adult Chinese: a population-based study in Liwan District, Guangzhou.
        Invest Ophthalmol Vis Sci. 2006; 47: 2782-2788
        • Topouzis F.
        • Wilson M.R.
        • Harris A.
        • et al.
        Prevalence of open-angle glaucoma in Greece: the Thessaloniki Eye Study.
        Am J Ophthalmol. 2007; 144: 511-519
        • Foster P.J.
        • Buhrmann R.
        • Quigley H.A.
        • Johnson G.J.
        The definition and classification of glaucoma in prevalence surveys.
        Br J Ophthalmol. 2002; 86: 238-242
        • Ebner M.
        Color constancy based on local space average color.
        Machine Vision and Applications. 2009; 11: 283-301
        • Szegedy C.
        • Vanhouke V.
        • Ioffe S.
        • et al.
        Rethinking the inception architecture for computer vision.
        (Accessed May 10, 2017)
        • Radford A.
        • Metz L.
        • Chintala S.
        Unsupervised representation learning with deep convolutional generative adversarial networks.
        (Accessed June 12, 2017)
        • Singh A.
        • Dutta M.K.
        • ParthaSarathi M.
        • et al.
        Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image.
        Comput Methods Programs Biomed. 2016; 124: 108-120
        • Chakrabarty L.
        • Joshi G.D.
        • Chakravarty A.
        • et al.
        Automated detection of glaucoma from topographic features of the optic nerve head in color fundus photographs.
        J Glaucoma. 2016; 25: 590-597
        • Issac A.
        • Partha S.M.
        • Dutta M.K.
        An adaptive threshold based image processing technique for improved glaucoma detection and classification.
        Comput Methods Programs Biomed. 2015; 122: 229-244
        • Xiangyu C.
        • Yanwu X.
        • Damon W.K.W.
        • et al.
        Glaucoma detection based on deep convolutional neural network.
        Conf Proc IEEE Eng Med Biol Soc. 2015; : 715-718
        • Annan L.
        • Jun C.
        • Damon W.K.W.
        • Jiang L.
        Integrating holistic and local deep features for glaucoma classification.
        Conf Proc IEEE Eng Med Biol Soc. 2016; : 1328-1331
        • Yang M.
        • Zhang L.
        • Shiu S.C.
        • Zhang D.
        Robust kernel representation with statistical local features for face recognition.
        IEEE Trans Neural Netw Learn Syst. 2013; 24: 900-912
        • Srivastava N.
        • Hinton G.
        • Krizhevsky A.
        • et al.
        Dropout: a simple way to prevent neural networks from overfitting.
        J Mach Learn Res. 2014; 15: 1929-1958
        • Krizhevsky A.
        • Sutskever I.
        • Hinton G.E.
        ImageNet classification with deep convolutional neural networks.
        Adv Neural Inf Process Syst. 2012; : 1-9
        • Ding X.
        • Chang R.T.
        • Guo X.
        • et al.
        Visual field defect classification in the Zhongshan Ophthalmic Center-Brien Holden Vision Institute High Myopia Registry Study.
        Br J Ophthalmol. 2016; 100: 1697-1702
        • Weinreb R.N.
        • Aung T.
        • Medeiros F.A.
        The pathophysiology and treatment of glaucoma: a review.
        JAMA. 2014; 311: 1901-1911
        • Wong T.Y.
        • Bressler N.M.
        Artificial intelligence with deep learning technology looks into diabetic retinopathy screening.
        JAMA. 2016; 316: 2366-2367