Machine learning models using age, sex, and up to three visual function tests classified stages of diabetic eye disease with area under the curve values of 0.94 or higher, similar to those achieved using a nine-test battery, in a cross-sectional study published in BMJ Open Ophthalmology.
Researchers evaluated 1,901 eyes from 1,032 participants drawn primarily from the Northern Ireland Sensory Ageing Study and diabetes clinics. Eyes were classified into four groups: no diabetes mellitus (DM); DM without diabetic retinopathy (DR); DR without diabetic macular edema (DME); and DR with DME. Median age ranged from 60 to 67 years across groups, and 56% to 79% of eyes were from female patients, depending on subgroup.
Participants underwent pharmacologic dilation and multimodal retinal imaging, including color fundus photography, ultra-widefield imaging, and spectral-domain optical coherence tomography. Diabetic retinopathy was graded using the national screening system for England and Wales, and DME was determined on optical coherence tomography. Nine visual function tests were performed following full refraction, generating 12 visual function variables, including visual acuity measures, low-luminance testing, contrast sensitivity, perimetry, and microperimetry.
The researchers defined three clinically relevant classification tasks: distinguishing DM without DR vs no DM; DR without DME vs DM without DR; and DR with DME vs DR without DME.
For each task, they fitted ensemble machine learning models using SuperLearner algorithms that incorporated regression-based, spline-based, neural network, and tree-based methods. All one-, two-, and three-test combinations were evaluated, with age and sex included in every model. Performance was assessed using 10-fold cross-validation at the participant level.
The top 30 models for each task achieved area under the curve (AUC) values of at least 0.94, with top-ranked models reaching 0.99 or 1.00. In contrast, logistic regression models using the same combinations achieved AUC values of 0.80, 0.66, and 0.89 for the three tasks in the data set with complete testing.
For distinguishing DM without DR vs no DM, 17 of the top 30 models in the full testing data set included distance visual acuity. Low-luminance visual acuity and Pelli-Robson contrast sensitivity each appeared in the top 30 models. In analyses excluding perimetry, reading index featured in 22 of the top 30 models.
For distinguishing DR without DME vs DM without DR, mesopic microperimetry was included in 19 of the top 30 models. Reading index, near visual acuity, and matrix perimetry also ranked highly. In the data set without perimetry, reading index appeared in 21 of the top 30 models and Smith-Kettlewell low-luminance near visual acuity in 15.
For distinguishing DR with DME vs DR without DME, 17 of the top 30 models in the full data set included distance visual acuity. Smith-Kettlewell low-luminance near visual acuity and near visual acuity were both highly ranked. In the no-perimetry data set, reading index featured in 22 of the top 30 models and Moorfields chart acuity featured in 11.
Most top-performing models combined three visual function tests. Only a small number of two-test models ranked in the top 30, and single-test models were uncommon, appearing for limited tasks and data sets.
The researchers also provided detailed distributions for each visual function measure across disease stages to inform power and sample size calculations in future studies.
The study was cross-sectional, and external validation was not performed. Two time-intensive perimetry tests were not conducted in all participants, and some tests, including the Moorfields chart, were introduced after recruitment began, resulting in incomplete data for certain combinations. The researchers noted that longitudinal studies are needed to determine whether visual function measurements can be used to predict morphologic changes as diabetic eye disease progresses.
“Using a data set comprising over 1,000 participants, we demonstrated machine learning models to classify status of diabetic eye disease with high performance, using just age, sex, and at most three from a battery of nine visual function tests,” the researchers wrote, adding that these models “had similar performance to models using the entire battery.”
Disclosures: The researchers reported no competing interests. The study was supported by multiple public and charitable funding sources, as detailed in the article.
Source: BMJ Open Ophthalmology
