- Research article
- Open Access
- Open Peer Review
Segmentation error in spectral domain optical coherence tomography measures of the retinal nerve fibre layer thickness in idiopathic intracranial hypertension
BMC Ophthalmologyvolume 17, Article number: 257 (2018)
Optical Coherence Tomography (OCT) imaging is being increasingly used in clinical practice for the monitoring of papilloedema. The aim is to characterise the extent and location of the Retinal Nerve Fibre Layer (RNFL) Thickness automated segmentation error (SegE) by manual refinement, in a cohort of Idiopathic Intracranial Hypertension (IIH) patients with papilloedema and compare this to controls.
Baseline Spectral Domain OCT (SDOCT) scans from patients with IIH, and controls with no retinal or optic nerve pathology, were examined. The internal limiting membrane and RNFL thickness of the most severely affected eye was examined for SegE and re-segmented. Using ImageJ, the total area of the RNFL thickness was calculated pre and post re-segmentation and the percentage change was determined. The distribution of RNFL thickness error was qualitatively assessed.
Significantly greater SegE (p = 0.009) was present in RNFL thickness total area, assessed using ImageJ, in IIH patients (n = 46, 5% ± 0–58%) compared to controls (n = 14, 1% ± 0–6%). This was particularly evident in moderate to severe optic disc swelling (n = 23, 10% ± 0–58%, p < 0.001). RNFL thickness was unable to be quantified using SDOCT in patients with severe papilloedema.
SegE remain a concern for clinicians using SDOCT to monitor papilloedema in IIH, particularly in the assessment of eyes with moderate to severe oedema. Systematic assessment and manual refinement of SegE is therefore important to ensure the accuracy in longitudinal monitoring of patients.
Quantifying papilloedema clinically is subjective and prone to inter–observer variability and inaccuracy during prospective monitoring . Spectral Domain Optical Coherence Tomography (SDOCT) is increasingly used both in the clinical environment and as outcome measures in Idiopathic Intracranial Hypertension (IIH) clinical trials to objectively quantify papilloedema . Commercially available SD-OCT imaging systems, such as the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA) and Spectralis (Heidelberg Engineering, Heidelberg, Germany), have proprietary in-built OCT software logarithms which use the difference in signal intensity between adjacent retinal layers to perform automated segmentation to segment inner and outer retinal boundaries, from which retinal nerve fibre layer thickness (RNFL thickness) measurements can be calculated. Optic disc swelling can be monitored by repeated assessments of the RNFL thickness [2,3,4]. Autosegmentation has been found to be inaccurate in some retinal pathologies such as neovascular age related macular degeneration and central serous retinopathy  and in optic nerve head pathologies such as glaucoma . In papilloedema, where the interface between the retinal layers is disturbed by oedema, errors in autosegmentation have been noted [7,8,9], with large studies using the Cirrus HD-OCT. The aim of this study was to evaluate the extent and location of the RNFL thickness SegE in an IIH cohort, comparing this to normal controls using the Spectralis SD-OCT.
Fifty-two consecutive IIH patients with a mean age of 31 years (standard deviation (SD) 9.4 years) and 14 controls with a mean age of 35.9 years (SD 7.21 years) at University Hospitals Birmingham NHS Foundation Trust (UHB NHS FT), a large tertiary referral centre, were enrolled. All participants were female. The study followed the tenets of the Declaration of Helsinki; informed consent was obtained; and the research was approved as a service evaluation by the UHB NHS FT research and development department. For inclusion, subjects were required to have active disease (papilloedema with at least Frisén grade 1 in one eye) and fulfil the accepted revised diagnostic for IIH . The IIH cohort had a median body mass index (BMI) of 38.7kgm−2 (range 24.3–51.3kgm−2) and a median baseline lumbar puncture opening pressure (LP OP) of 35.5 cm CSF (25.0–60.0 cm CSF). Using the Modified Frisén Scale , two independent reviewers (SPM and JH) stratified anonymised fundal photos according to the degree of papilloedema; 23 were classified as mild disc swelling (Frisén grade 1–2), and 29 with moderate/severe disc swelling (Frisén grade 3–5). Healthy control subjects were recruited only after retinal and/or optic nerve pathology was excluded. Lumbar punctures and BMI indices were not conducted in the control group.
SDOCT RNFL thickness peripapillary circular scans were acquired from all subjects using Heidelberg Engineering SPECTRALIS HRA + OCT (Heidelberg Engineering, Heidelberg, Germany). Spectralis OCT uses a dual-beam SDOCT, a confocal laser-scanning ophthalmoscope with a wavelength of 870 nm, and an infrared reference image to obtain images of ocular microstructures with an acquisition rate of 40,000 A-scans per second. Sufficient OCT scan quality was considered as a Q score of greater than 12 and the absence of posterior vitreous detachment, fovea malalignment or media opacity secondary to cataract. The Spectralis OCT software allows for automatic segmentation of the upper and lower borders of the RNFL to calculate the average RNFL thickness. Peripapillary RNFL thickness values are divided into 4 quadrants, namely superior, inferior, nasal and temporal. The SD-OCT scans were qualitatively and quantitatively examined for RNFL thickness SegE and analysed with both the Spectralis automated software and then ImageJ Software package (https://imagej.nih.gov/ij/).
Examination for OCT segmentation error (SegE)
A masked reviewer (AA) assessed anonymised OCT scans for SegE using Heidelberg Eye Explorer software, version 1.9.1. (Heidelberg Engineering, Heidelberg, Germany). For each subject, only one eye was analysed using the baseline OCT scan from the most severely affected eye; this was identified by the highest single point maximum RNFL thickness value (μm). Initially the SegE was qualitatively assessed and the location recorded. Quantitative analysis then involved evaluating the internal limiting membrane and RNFL thickness for the presence of SegE and accordingly using the Heidelberg Eye Explorer software which automatically identifies the layer border and allows for manual correction of the segmentation. Pre- and post re-segmentation, average and maximum height of the RNFL thickness (μm) was recorded in the following areas: global RNFL thickness and the superior, nasal, inferior and temporal retinal quadrant. Finally, pre and post re-segmentation, the RNFL thickness total area was delineated and quantified independently of the Heidelberg Eye Explorer software, using the ImageJ software polygon and analyse area tool, respectively. The percentage change in the total area of the RNFL area was subsequently calculated. Quality assurance was undertaken with a further masked observer (JH) independently examining SegE in a quarter (n = 20) of the subjects in the cohort to ensure there was sufficient concordance.
Descriptive statistics were used to compare demographic characteristics by group (IIH and healthy controls). Statistical analysis was performed using SPSS software, version 23.0 (IBM, Armonk, NY). Due to the lack of normal distribution, data were analysed using the Kruskal-Wallis test and tested for significant pairwise comparisons. Values were expressed as the median ±range. A two-tailed Spearman’s correlation test was used to conduct correlation analysis. Values were deemed statistically significant at p < 0.05.
Of the 52 IIH patients, the scans from 6 patients were excluded as they had such severe papilloedema that the optic nerve head elevation was truncated by the scan image and therefore the height could not be visualised and no further accurate refinement of the RNFL thickness could be performed. Forty-six IIH subjects and 14 controls were therefore included in the quantitative analysis (Fig. 1). The reliability between the two independent raters (AA and JH) was 0.732, with 95% CI (0.232–0.926), p < 0.05.
Quantification of the difference between the automated and the manually corrected total RNFL thickness area using ImageJ revealed significantly greater SegE in IIH patients [5% change post segmentation refinement, range = 0–58%] compared to controls [1% change post segmentation refinement, range = 0–6%] p = 0.009 (Table 1; Additional file 1: Table S1, S2). This was particularly evident in IIH patients with moderate to severe papilloedema [10% change post segmentation refinement, range = 0–58%, p < 0.001]. (Additional file 1: Table S3).
The error in automated overall average RNFL thickness values was significantly greater in IIH compared to controls (p = 0.031): median176μm (range 76-581 μm) pre re-segmentation versus 159 μm (range 83-391 μm) post re-segmentation in IIH (4% change post re-segmentation, range 0–58%) this was compared to 98 μm (range 63-125 μm) pre segmentation versus 100 μm (range 65-126 μm) post re-segmentation in controls (2% change post re-segmentation, range = 0–6%). IIH patients with moderate to severe papilloedema displayed significantly greater error in the overall average RNFL thickness values compared to those with mild papilloedema [10% change post re-segmentation (range 0–58% in moderate and severe papilloedema, p = 0.002)] (Table 1 and Fig. 2). In those with moderate to severe papilloedema the SegE was significantly greater in the superior retinal quadrant [11% change post re-segmentation, range = 0–375%, p = 0.001] (Fig. 3).
Qualitative assessment of any error in the RNFL thickness segmentation was more often identified in IIH (any apparent error in 98% of the IIH group, (45/46)) compared to control subjects (error in 79% of patients, (11/14)) (Table 2). It was clearly observed that the magnitude of the SegE was more pronounced in the IIH compared to control subjects. There was no clear pattern between subjects of whether the error was inflation or deflation of their disc height. In IIH patients, the RNFL thickness SegE was predominantly noted in the superior retinal quadrant, but to a lesser degree in the inferior retinal quadrant. In contrast, the control subjects displayed minimal error that had no predominant distribution.
The highest single point of maximum thickness in the RNFL thickness was then assessed and there was no statistically significant difference in RNFL thickness segmentation between the automated and the manually corrected segmentation between IIH and controls. However error in the maximal height in the superior region in those with moderate or severe papilledema did have significant error pre- and post re-segmentation in the superior retinal quadrant: 345 μm (range 139-1007 μm) automated value versus 297 μm (range 178-445 μm) post re-segmentation (12% change post re-segmentation, range = 0–60%, p = 0.017), compared to 174 μm (range 144 - 211 μm) automated values and 184 μm (range 150–211 μm) post re-segmentation for the controls (3% change post re-segmentation, range = 0–26%).
OCT imaging is increasingly utilised for quantification and monitoring of papilloedema in IIH in the clinical setting. The largest prospective controlled cohort in papillodema reporting use of OCT in IIH is data from the IIHTT ; this study investigated 126 participants and utilised the Cirrus OCT at multiple centres and found that 3-dimensional scanning was less prone to failures of segmentation than 2-dimensional images. There is limited literature on the accuracy of autosegmentation in papilloedema using the Spectralis OCT. Detection of SegE, and manually refining the interfaces could help improve the accuracy of the RNFL thickness values between consecutive tests, and improve the clinical utility of the SD-OCT in longitudinal monitoring of IIH. This study highlights the issue of significant error in the automated RNFL thickness values generated from peripapillary RNFL thickness circle scans using the Spectralis SD-OCT. Like other SD OCT devices, the Spectralis in built algorithms are not specifically designed to autosegment papilloedema and although we identified significant error in the overall average RNFL thickness value with a 4% change following manual re-segmentation, this is much less error than previously reported using time domain OCT platforms . What is yet to be determined is the clinical significance of the magnitude of this error.
The SegE was most apparent in those with moderate to severe papilloedema (10% error) and particularly in the superior retinal quadrant (11% error). The majority of the RNFL thickness error was accounted for by inaccurate automated identification of the lower boarder of the RNFL at the junction with the ganglion cell layer. It is likely that oedema and vessel artefact lead to error in the average RNFL thickness automated values, as postulated by previous authors [9, 13].
In 6% of the cohort, with severe papilloedema, the extreme elevation of the optic nerve head obscured the upper boarder limit of the RNFL and it was therefore not possible to refine the segmentation in these patients due to the truncation of the image. This truncation artefact has been previously reported by other authors [the type 1–8 paper] . It may be less of an issue with newer OCT systems, using swept source technology, that provide a greater depth of imaging (e.g., Topcon DRI OCT-1 Triton has a depth range of 2.6 mm – greater than the 1.9 mm depth range of the Heidelberg system, based on spectral domain technology) .
SegE is not the only cause for erroneous RNFL thickness values; a number of factors have been associated with artifacts in OCT scanning including decentration error, refractive error, posterior vitreous detachment artifacts, reduced visual acuity, small pupils, presence of media opacities, advanced stage of glaucoma and dry eyes [9, 15,16,17,18] Eye tracking on Spectralis ensures better alignment and is reported to decreases error in malalignment [19, 20].
Manual refinement of segmentation has several limitations which include the time taken to perform this accurately; indeed the accuracy of the manual markings, in which experts invariably disagree on where to draw the margins when the boarders between layers are hazy. In this study we performed an inter-user variability check to ensure that there was sufficient agreement between two masked individuals. The clinical impact of this error has not been evaluated in this study but would be a useful area for future investigation. As the Spectralis SD OCT platform was used in this study our results may not be generalizable to results from other types of SD-OCT machines.
RNFL thickness peripapillary scans are not the only OCT scan modality used to assess papilloedema. Other scanning modalities include volumetric analysis of the optic nerve head and macula; and Bruchs Membrane Opening (BMO) rim analysis. However, the degree of oedema in moderate to severe papilloedema is also known to cause error in these scans due to optical penetration. Future solutions include better depth penetration and a wider scan window to account for the elevation found in disc oedema. Polarization-sensitive OCT, which is not currently commercially available, has the potential to delineate the RNFL boundary better based on pigment differences in the retinal layers and be less prone to SegE .
In the setting of virtual IIH clinics where patient’s management may be judged exclusively by objective OCT and Humphrey visual field values; there could be clinical risk in the misinterpretation of the degree of papilledema and its course over time if the SegE is not identified and corrected for at the time of acquisition of the scans . Here we have highlighted the limitations of using automated results from OCT RNFL thickness scans, particularly in those with marked papilloedema. We have developed a suggested paradigm to guide healthcare professionals performing OCT RNFL thickness peripapillary scans in IIH (Fig. 4). Scans should be evaluated for error at the time of acquisition to ensure the accuracy of the data at the time of the clinical visit when management decisions may be being made. However, as demonstrated with the results from the IIHTT  we would recommend the use of optic nerve volume scans in the routine clinical assessment of papilloedema.
Using the Spectralis SD-OCT, SegE in RNFL thickness values were found to be greater in IIH than controls, with the error increasing with the severity of the papilloedema. Imaging was not useful in very severe papilloedema where the image was truncated. This is the largest cohort assessing SegE in IIH, using the Spectralis SD-OCT. Achieving accurate and reproducible image analysis is important in the longitudinal monitoring in IIH, hence recognition of SegE and manual refinement should be understood by technicians and clinicians alike.
Bruchs membrane opening
Idiopathic intracranial hypertension
Optical coherence tomography
Retinal nerve fibre layer
Spectral domain optical coherence tomography
Sinclair AJ, Burdon MA, Nightingale PG, et al. Rating papilloedema: an evaluation of the Frisén classification in idiopathic intracranial hypertension. J Neurol. 2012;259:1406–12.
Optical Coherence Tomography Substudy Committee, NORDIC Idiopathic Intracranial Hypertension Study Group. Papilledema outcomes from the optical coherence tomography substudy of the idiopathic intracranial hypertension treatment trial. Ophthalmology. 2015;122:1939-45.e2.
Skau M, Milea D, Sander B, Wegener M, Jensen R. OCT for optic disc evaluation in idiopathic intracranial hypertension. Graefes Arch Clin Exp Ophthalmol. 2011;249:723–30.
Kaufhold F, Kadas EM, Schmidt C, et al. Optic nerve head quantification in idiopathic intracranial hypertension by spectral domain OCT. PLoS One. 2012;7:e36965.
Novosel J, Vermeer K, de Jong JH, Wang Z, Van Vliet L. Joint segmentation of retinal layers and focal lesions in 3D OCT data of topologically disrupted retinas. IEEE Trans Med Imaging. 2017; https://doi.org/10.1109/TMI.2017.2666045. [Epub ahead of print].
Mansberger SL, Menda SA, Fortune BA, Gardiner SK, Demirel S. Automated segmentation errors when using optical coherence tomography to measure retinal nerve fiber layer thickness in glaucoma. Am J Ophthalmol. 2017;174:1–8.
Karam EZ, Hedges TR. Optical coherence tomography of the retinal nerve fibre layer in mild papilloedema and pseudopapilloedema. Br J Ophthalmol. 2005;89:294–8.
Scott CJ, Kardon RH, Lee AG, Frisén L, Wall M. Diagnosis and grading of papilledema in patients with raised intracranial pressure using optical coherence tomography vs clinical expert assessment using a clinical staging scale. Arch Ophthalmol. 2010;128:705–11.
Chen JJ, Kardon RH. Avoiding clinical misinterpretation and artifacts of optical coherence tomography analysis of the optic nerve, retinal nerve fiber layer, and ganglion cell layer. J Neuroophthalmol. 2016;36:417–38.
Friedman DI, Liu GT, Digre KB. Revised diagnostic criteria for the pseudotumour cerebri syndrome in adults and children. Neurology. 2013;81:1159–65.
Frisen L. Swelling of the optic nerve head: a staging scheme. J Neurol Neurosurg Psychiatry. 1982;45:13–8.
Giani A, Cigada M, Esmaili DD, et al. Artifacts in automatic retinal segmentation using different optical coherence tomography instruments. Retina. 2010;30:607–16.
OCT Sub-Study Committee for the NORDIC Idiopathic Intracranial Hypertension Study Group. Baseline OCT measurements in the idiopathic intracranial hypertension treatment trial, part I: quality control, comparisons, and variability. Invest Ophthalmol Vis Sci. 2014;55:8180–8.
Spaide RF. Visualization of the posterior vitreous with dynamic focusing and windowed averaging swept source optical coherence tomography. Am J Ophthalmol. 2014;158:1267–74.
Asrani S, Essaid L, Alder BD, Santiago-Turla C. Artifacts in spectral-domain optical coherence tomography measurements in glaucoma. JAMA Ophthalmol. 2014;132:396–402.
Cheung CY, Leung CK, Lin D, et al. Relationship between retinal nerve fiber layer measurement and signal strength in optical coherence tomography. Ophthalmology. 2008;115:1347–51.
Wu Z, Vazeen M, Varma R, et al. Factors associated with variability in retinal nerve fiber layer thickness measurements obtained by optical coherence tomography. Ophthalmology. 2007;114:1505–12.
Liu Y, Simavli H, Que C, et al. Patient characteristics associated with artifacts in Spectralis optical coherence tomography imaging of the retinal nerve fiber layer in glaucoma. Am J Ophthalmol. 2015;159:565–76.
Hammer DX, Ferguson RD, Magill JC, et al. Active retinal tracker for clinical optical coherence tomography systems. J Biomed Opt. 2005;10:024038.
Giani A, Cigada M, Choudhry N, et al. Reproducibility of retinal thickness measurements on normal and pathologic eyes by different optical coherence tomography instruments. Am J Ophthalmol. 2010;150(6):815–24.
Pircher M, Hitzenberger CK, Schmidt-Erfurth U. Polarization sensitive optical coherence tomography in the human eye. Prog Retin Eye Res. 2011;30(6):431–51.
PAK has received a proportion of his funding from the Department of Health’s NIHR Biomedical Research Centre for Ophthalmology at Moorfields Eye Hospital and UCL Institute of Ophthalmology. The views expressed in the publication are those of the author and not necessarily those of the Department of Health.
AS is funded by an NIHR Clinician Scientist Fellowship (NIHR-CS-011-028) and by the Medical Research Council, UK (MR/K015184/1).
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The study followed the tenets of the Declaration of Helsinki; informed consent was obtained; and the research was approved as a service evaluation by the UHB NHS FT research and development department.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
IIH Cohort showing absolute median values (range) pre and post segmentation with % change. Table S2. Controls showing absolute median values (range) pre and post segmentation with % change. Table S3. Moderate to severe IIH showing absolute median values (range) pre and post segmentation with % change. Table S4. Mild IIH showing absolute median values (range) pre and post segmentation with % change. (DOCX 35 kb)
About this article
- Idiopathic intracranial hypertension
- Pseudotumour Cerebri
- Optical coherence tomography
- Retinal nerve fibre layer