Precision and agreement of higher order aberrations measured with ray tracing and Hartmann-Shack aberrometers

Wavefront aberrations include defocus, astigmatism, and higher order aberrations. Higher order aberrations (HOAs) are small irregularities or imperfections of the eye that cannot be corrected by conventional spectacles [1]. Several wavefront analysers (known as aberrometers) have been developed to detect wavefront aberrations (especially HOAs) [2, 3]. During the past decade, aberrometers have been used in many fields of ophthalmology and optometry [4, 5], including the observation of refractive errors [6], the diagnosis of dry eye diseases [7] and keratoconus [8], and refractive surgery [9, 10]. Traditionally, corneal topographers can provide corneal aberrometry according to special algorithms based on elevation data. Recently, ocular aberrations have been obtained using data from the aberrometers. Thus intraocular aberrations could be obtained by subtracting the ocular aberrations from corneal aberrations. The principles of these aberrometers can be divided into the Hartmann-Shack method, the ray tracing method, the Tscherning principle, etc. Both the Topcon KR-1 W system (Hartmann-Shack method) and the iTrace system (ray tracing method) are devices composed of an aberrometer and a corneal topographer.

Several studies have assessed the repeatability or reproducibility of HOA measurements obtained by Topcon KR-1 W and iTrace, respectively [11–14]. However, to the best of our knowledge, few studies have reported the assessment of precision (repeatability and reproducibility) and agreement of HOAs obtained by the two devices, simultaneously.

In a previous study, we evaluated the repeatability and reproducibility of corneal power measurements obtained by Topcon KR-1 W and iTrace [15]. Since the more important function of the two devices is the measurement of HOAs, in this study, we estimated the precision (repeatability and reproducibility) and agreement of ocular, corneal and internal HOAs under 4 mm pupil diameter obtained by Topcon KR-1 W and iTrace in normal eyes.

In this prospective study, ninety-two normal subjects (37 males) were enrolled. The mean age was 34.67 ± 12.18 years (range 21 to 69 years), and the mean spherical equivalent refraction was − 2.88 ± 3.10 diopters (D, range − 9.00 to + 1.00 D).

Comparison of the HOA measurements obtained with Topcon KR-1 W and iTrace

Table 7 displays differences between the HOA measurements obtained from the two devices. For ocular HOAs, the tHOA, coma, Second Astig and trefoil values obtained with the Topcon KR-1 W were statistically smaller than those obtained with iTrace (all P < 0.001), while the SA values were comparable (P = 0.522). Meanwhile, the 95% LoA in Bland-Altman plots for tHOA, SA, coma, Second Astig and trefoil were 0.46 μm, 0.19 μm, 0.23 μm, 0.10 μm and 0.18 μm, respectively (Fig. 1). These results mean that the agreement among these devices was relatively bad, since 0.1 μm is normally considered to be clinically significant for HOAs. It is also clearly observed in Fig. 1 that, for several HOAs (i.e., ocular tHOA), the differences between the devices show a tendency to decrease with increasing HOA magnitude.

Aberrations	Mean Difference ± SD	95% CI	t	P value*
Ocular
tHOA	−0.060 ± 0.118	−0.084 to − 0.036	−4.884	< 0.001
SA	0.003 ± 0.049	−0.007 to 0.014	0.644	0.522
Coma	−0.023 ± 0.058	−0.035 to − 0.011	−3.767	< 0.001
Second Astig	− 0.010 ± 0.025	− 0.016 to − 0.005	−3.913	< 0.001
Trefoil	− 0.017 ± 0.046	−0.027 to − 0.008	−3.602	< 0.001
Cornea
tHOA	−0.047 ± 0.118	−0.071 to − 0.022	−3.801	< 0.001
SA	0.033 ± 0.049	0.024 to 0.043	6.760	< 0.001
Coma	−0.013 ± 0.056	−0.024 to − 0.001	−2.179	0.032
Second Astig	−0.008 ± 0.027	− 0.013 to − 0.002	−2.748	0.007
Trefoil	− 0.009 ± 0.056	−0.021 to 0.002	−1.573	0.119
Internal
tHOA	−0.042 ± 0.053	−0.054 to − 0.031	−7.589	< 0.001
SA	− 0.010 ± 0.053	− 0.021 to 0.001	− 1.813	0.007
Coma	− 0.009 ± 0.064	− 0.022 to − 0.005	−1.285	0.202
Second  Astig	0.012 ± 0.048	0.002 to − 0.022	2.343	0.021
Trefoil	−0.030 ± 0.046	−0.039 to − 0.020	−6.202	< 0.001

Note: tHOA Total high order aberration, SA Spherical aberration, Second Astig Second astigmatism, SD Standard deviation, *two tailed probability

For corneal HOAs, the tHOAs, coma and Second Astig values obtained with Topcon KR-1 W were statistically smaller than those obtained with iTrace (all P < 0.05), while the trefoil values were similar (P = 0.119). In contrast, the SA values obtained from Topcon KR-1 W were statistically larger than those obtained from iTrace (P < 0.001). Meanwhile, the 95% LoA in Bland-Altman plots for tHOA, SA, coma, Second Astig and trefoil were 0.46 μm, 0.19 μm, 0.22 μm, 0.11 μm and 0.22 μm, respectively (Fig. 2), all of which indicated poor agreement (95% LoA > 0.1 μm). It is also clearly observed in Fig. 2 that, for several HOAs (i.e., corneal SA), the differences between devices show a tendency to decrease with increasing HOA magnitude.

For internal HOAs, the tHOA, SA and trefoil values obtained by Topcon KR-1 W were statistically smaller than those obtained with iTrace, while the Second Astig values were statistically larger than those obtained with iTrace (P = 0.021), and the coma values were comparable (P = 0.202). Like in ocular HOAs and corneal HOA measurements, poor agreement (95% LoA > 0.1 μm) was also observed in internal HOA measurements: the 95% LoA in Bland-Altman plots for tHOA, SA, coma, Second Astig and trefoil were 0.21 μm, 0.20 μm, 0.25 μm, 0.18 μm and 0.18 μm, respectively (Fig. 3). It is also clearly observed in Fig. 3 that, for several HOAs (i.e., internal trefoil), the differences between the devices showed a tendency to decrease with increasing HOA magnitude.

As mentioned above, accurate HOA measurements are essential for evaluating the imaging quality of the ocular refractive system. In this study, we comprehensively assessed intraobserver repeatability and the interobserver and intersession reproducibility of the ocular, corneal, and internal HOA measurements generated from the Hartmann-Shack aberrometer (Topcon KR-1 W) and the Ray-Tracing aberrometer (iTrace). While the Hartmann-Shack principle was used in many kinds of aberrometers, the Ray-Tracing principle has been used in iTrace only. Moreover, we compared the ocular, corneal and internal HOA measurements obtained with the two devices. We found that both devices were repeatable in intraobserver section but were not sufficiently reproducible in terms of interobserver and intersession measurement. We also found that most of the ocular, corneal and internal HOA values (except SA) obtained with Topcon KR-1 W were significantly smaller than those obtained with iTrace.

Both Topcon KR-1 W and iTrace yielded a high repeatability; however, the interobserver and intersession reproducibility of the ocular tHOAs was less good. For Topcon KR-1 W, the ICC of the ocular tHOAs was 0.957 or more for the repeatability assessment. In the study by Lopez-Miguel et al. [11], the ICC of the ocular tHOAs of the 6-mm-diameter pupil was 0.902, while in a study by Pinero et al. [12], the ICCs of 6 mm and 4 mm diameter pupils were 0.864 and 0.795, respectively. For intersession reproducibility assessment, Lopez-Miguel et al. reported an ICC of 0.822 for the ocular tHOAs, while we found the ocular tHOA to be 0.736. Furthermore, we obtained ICCs of 0.772 and 2.77Sw of 0.076 for interobserver reproducibility assessment. For iTrace, similar ICC results were obtained. This day-to-day and observer-to-observer inconsistency may be partially due to the fact that HOA is fluctuant [18–20], and this should be considered in clinical applications.

Some researchers have suggested that only patients whose eyes have HOAs that are larger than the TRT (≈2.77Sw) of the aberrometer measurement should be considered as candidates for wavefront-guided excimer laser surgery [21]. This is reasonable because we cannot ascertain whether HOA values that are smaller than TRT are actually random noise of the aberrometer measurement or not. If they are, then the surgery may bring in even larger tHOAs for eyes [21]. In the present study, the TRT of the ocular tHOA repeatability was 0.041 μm in Topcon KR-1 W and 0.071-0.073 μm in iTrace, respectively, while 0.10 μm was normally considered to be clinically significant for HOAs. This should be useful information for surgeons.

The SA seemed to produce more repeatable and reproducible results compared to the other parameters, regardless of what kind of aberrometer. For Topcon KR-1 W, the study by Lopez-Miguel et al. showed ICC values of 0.902 and 0.793 for intraobserver repeatability and intersession reproducibility, respectively [11]. In the study by Pinero et al., the ICC repeatability was 0.949 [12]. Similar results could also be achieved in other Hartmann-Shack aberrometers. In another study by Lopez-Miguel et al. [21], the counterpart was 0.90, as obtained by Zywave. Other systems, such as Irx3, Keratron, LADARWave and AMO WaveScan, could also produce a good repeatability of the SA measurements [22–24]. For iTrace, the repeatability the ICCs of the ocular SA were also good (0.992-0.995) in our study. It is worth mentioning that the SA measurements in both Topcon KR-1 W (0.027-0.031 μm in our study, 0.091 μm in a previous study) and iTrace (0.016-0.019 um) were much lower than that of the Zywave Hartmann-Shack aberrometer (0.186 um) [21]. All these results showed optimistic SA measurements for the two devices.

Lower precision was observed in the coma measurement, although the repeatability of the coma measurement was still satisfactory. In our present research, the ICCs of intraobserver repeatability of the ocular structure were 0.931-0.945 for Topcon KR-1 W and 0.989-0.992 for iTrace, which were higher than the counterpart in previous studies (0.869 in one study [11] and 0.673 in another reference [12]) with Topcon KR-1 W. However, the interobserver repeatability was less satisfactory. The ICC of the interobserver repeatability of ocular coma measurement was 0.820 with Topcon KR-1 W and 0.661 with iTrace. The intersession reproducibility of coma was low, especially for the internal coma. The ICC of the internal coma was still 0.628 with Topcon KR-1 W but only 0.246 with iTrace, which is similar to the 0.223 value reported in a previous study [11]. In fact, the coma values significantly change diurnally, as revealed in the study by Read et al. [18] Srivannaboon et al. [25] noted that post-blink changes in HOAs after blinking could have more influence on changes of coma-like aberrations than on spherical aberrations. This prediction may partially explain the lower precision in the coma measurement.

The precision of the Second Astig measurement was also rather poor. For Topcon KR-1 W, although we still obtained acceptable Second Astig intraobserver repeatability (all ICCs > 0.802) in our study, we had poor reproducibility (all ICCs < 0.696). This was similar to Pinero’s result (ICC < 0.635) [12]. For the iTrace, the Second Astig (especially the corneal and internal Second Astig) showed even poorer precision. This is indirectly consistent with Pollack et al.’s research [18], in which they found a statistically significant change in the Second Astig during the day. Thus, the Second Astig measurement may not be reliable from our standpoint.

The precision of the corneal HOA measurement based on Placido-disk corneal topography is usually lower than that of the ocular HOA measurement based on aberrometry as reflected in the two devices (Tables 1, 2, 3, 4, 5, and 6). Meanwhile, the precision of the corneal HOA measurement in Topcon KR-1 W is mostly more reliable than that in iTrace. From the data of the first observer in the present study, the ICCs of the intraobserver repeatability, interobserver and intrasession reproducibility for corneal HOAs were 0.926, 0.790 and 0.682, respectively, as obtained with Topcon KR-1 W. In contrast, the counterparts were 0.813, 0.398, 0.332, respectively, as obtained with iTrace. In the study by Visser et al. [22] on the corneal tHOA, iTrace showed better repeatability than Hartmann-Shack aberrometers (Irx3 and Keratron). Two conclusions may be made from these results: first, Topcon KR-1 W may be more reliable than the other Hartmann-Shack aberrometers; second, both Topcon KR-1 W and iTrace are reliable in terms of intraobserver repeatability but not in terms of reproducibility in corneal HOA measurements.

Unlike ocular and corneal HOA measurements, which have become a common and effective ophthalmic procedure, the internal HOA measurement is not straightforward and could be obtained indirectly only by subtracting the corneal HOAs from the ocular HOAs. Thus, it is reasonable to assume that the precision of the internal tHOA measurements tend to be worse compared with the ocular and corneal tHOAs because the measurement variability of the internal tHOAs are derived from both the ocular and the corneal HOA measurements [11]. This is consistent with the results from the study by Lopez-Miguel et al., in which the repeatability and the intrasession reproducibility of ICCs were 0.813 and 0.538, respectively [11]. Similar results were obtained in the present study: the intraobserver repeatability, interobserver and intrasession reproducibility of ICCs for the intraocular HOAs were 0.728, 0.742 and 0.638, respectively. Thus, surgeons should note that the internal HOA measurements may not be as reliable as the ocular and corneal HOA measurements.

The system noise of the instruments may be partially responsible for the measurement inconsistency. Some researchers noted that the ray-tracing aberrometers may be less sensitive when measuring low values of aberrations but have more advantages when measuring high values of aberrations, compared with the Hartmann-Shack aberrometers. The reason may be that the ray-tracing aberrometers operate by detecting individual retinal spots, while the Hartmann-Shack aberrometers operate by detecting all the retinal spots at the same time. Thus, the ray-tracing aberrometers should be more reliable when these retinal spots are substantially larger than the instrument noise [26]. Since the subjects in our study were all healthy, normal people, it is expected that we have found more reliable results in the HOA measurement using the Hartmann-Shack aberrometer (Topcon KR-1 W) as reflected in Tables 1, 2, 3, 4, 5, and 6.

In addition to the instrument noise, there are some other factors that may account for the decreased precision (especially day-to-day and observer-to-observer inconsistency). These factors include fluctuations of HOAs, eye movements, etc. [27–29]. Researchers have already found that the wavefront aberrations of the eye are not static but are instead dynamic. This could be due to several reasons. The first is an accommodative response caused by pupil translation, particularly in eyes with low refractive errors [30]. To minimize the effects of pupil-diameter-change, data analysis was limited to 4-mm diameter for every examination. Dynamic changes in tear film thickness in front of the cornea could also influence the fluctuations of HOAs, which could be due to evaporation, blinking [31] and disruption of the tear film. Thus, in our study, the measurement data was obtained after a brisk blink to ensure high-quality results by limiting changes in the tear film. Decreased precision could also be correlated to eye movements because of very slight changes in fixation [11]. Thus, other stricter support methods (for example, a dental bite) could be used to improve the stability of head position in future studies.

The present study indicated that most of the HOA parameters obtained with Topcon KR-1 W were statistically smaller than those obtained with iTrace. For the ocular HOA measurements, iTrace generated higher values than KR-1 W (all Ps < 0.001) except SA (P = 0.522). For cornea HOAs (absolute value), iTrace showed higher values than KR-1 W (all Ps ≤ 0.032) except trefoil (P = 0.119). For the internal HOAs, iTrace showed higher values than KR-1 W (all Ps ≤ 0.007) except Trefoil (P = 0.202) and Second Astig. In Rozema’s research, the HOA measurements obtained with iTrace tended to be larger than those obtained with Shack–Hartmann aberrometers including Zywave (Bausch & Lomb), WASCA (Zeiss/Meditec) and MultiSpot 250-AD, and significant differences between the devices were found in the coma measurements [32]. The results were consistent: ray-tracing aberrometers tend to give larger HOA values than Shack–Hartmann aberrometers, at least when measuring low values of HOA. As a mainstream method, the Shack–Hartmann principle is used as a basis for wavefront analyses in various companies such as Topcon, Visx, Alcon, Bausch and Lomb, Meditec and Schwind.

It should be noted that Ray Tracing aberrometers also tend to give larger HOA values than aberrometers based on other principles. In Visser’s study [22], the SA value obtained with iTrace was 0.064 ± 0.076 μm, which was significantly higher than the value obtained with an aberrometer based on the principle of slit skiascopy (OPD-Scan) [33]. Similar results were also found in the study by Won et al. [34], in which the ocular SA obtained with iTrace (0.038 ± 0.043 um) was significantly higher than that obtained with the OPD-Scan (0.011 ± 0.039 μm, P < 0.001). So were the internal coma and trefoil. Similar results were also obtained when comparing iTrace with a Tscherning Aberrometer (WaveLight) [32].

Since the HOA parameters obtained with Topcon KR-1 W were significantly different from those obtained with iTrace, it was expected that the agreement among these two devices was not good. 95% LoA of ocular, corneal and internal tHOA were 0.21 μm-0.46 μm, which is much larger than 0.1 μm (normally considered to be clinically significant for HOAs). Thus, these two devices should not be interchangeable in clinical applications.

This study had some limitations. First, we evaluated only the precision (repeatability and reproducibility) of the HOAs in normal subjects without eye problems. Second, we compared only the HOA values measurement with the aberrometers based on ray tracing and Shack–Hartmann principles. Third, we referred only to HOA measurements under the pupil of 4 mm, and the HOA measurements under 6 mm or others were not covered.

In conclusion, the ocular, corneal and internal HOAs obtained with Topcon KR-1 W and iTrace were repeatable in intraobservers but less reproducible in interobserver and intersession measurements except for SA. Aberrometers based on the ray tracing principle and aberrometers based on the Hartmann–Shack principle should not be interchanged in clinical applications.