Skip to main content

Modeling the impact of COVID-19 on Retina Clinic Performance

Abstract

Background

COVID-19, a highly contagious respiratory virus, presents unique challenges to ophthalmology practice as a high-volume, office-based specialty. In response to the COVID-19 pandemic, many operational changes were adopted in our ophthalmology clinic to enhance patient and provider safety while maintaining necessary clinical operations. The aim of this study was to evaluate how measures adopted during the pandemic period affected retina clinic performance and patient satisfaction, and to model future clinic flow to predict operational performance under conditions of increasing patient and provider volumes.

Methods

Clinic event timestamps and demographics were extracted from the electronic medical records of in-person retina encounters from March 15 to May 15, 2020 and compared with the same period in 2019 to assess patient flow through the clinical encounter. Patient satisfaction was evaluated by Press Ganey patient experience surveys obtained from randomly selected outpatient encounters. A discrete-events simulation was designed to model the clinic with COVID-era restrictions to assess operational performance under conditions of increasing patient and provider volumes.

Results

Retina clinic volume declined by 62 % during the COVID-19 health emergency. Average check-in-to-technician time declined 79 %, total visit length declined by 46 %, and time in the provider phase of care declined 53 %. Patient satisfaction regarding access nearly doubled during the COVID-period compared with the prior year (p < 0.0001), while satisfaction with overall care and safety remained high during both periods. A model incorporating COVID-related changes demonstrated that wait time before rooming reached levels similar to the pre-COVID era by 30 patients-per-provider in a 1-provider model and 25 patients-per-provider in a 2-provider model (p < 0.001). Capacity to maintain distancing between patients was exceeded only in the two 2-provider model above 25 patients-per-provider.

Conclusions

Clinic throughput was optimized in response to the COVID-19 health emergency. Modeling these clinic changes can help plan for eventual volume increases in the setting of limits imposed in the COVID-era.

Peer Review reports

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a highly contagious respiratory virus, presents unique challenges to ophthalmology practice [1,2,3]. Ophthalmologists operate in close physical proximity to their patients because of the nature of the eye examination, putting providers at relatively greater risk of exposure to the virus that causes coronavirus disease 2019 (COVID-19). Additionally, ophthalmology is typically a high-volume, office-based specialty, adding to the risk of transmission.

On March 15, 2020, the state of Massachusetts issued a stay-at-home order in an effort to control the spread of COVID-19 [4]. Despite this directive, Massachusetts was heavily affected by the pandemic. It had the third highest mortality and number of cases per capita across all states during the spring of 2020 [5, 6]. In response to the stay-at-home order, many operational changes were adopted in our ophthalmology clinic to enhance the safety of patients and providers, while maintaining necessary clinical operations. Foremost was an immediate reduction in clinic volume. Only emergency and urgent visits took place during this period, such as for those patients with conditions that required time-sensitive and necessary in-person management [7]. Patients with non-urgent conditions were either offered telemedicine or were rescheduled, as appropriate. Our office was reconfigured and waiting room capacity was limited to promote physical distancing. Protective barriers were installed, and additional time was added for staff to clean surfaces. These enhanced safety protocols were on par with those enacted by other ophthalmology and retina clinics [8,9,10]. Finally, to decrease the amount of time that patients spent in the office, the eye exam was streamlined, and ancillary testing was either reduced or performed at a different time [1], similar to strategies implemented by other ophthalmology clinics in order to see patients in person only when critically necessary [11, 12].

This paper systematically evaluates the impact of these COVID-related operational changes on our retina clinic by evaluating electronic medical record (EMR) timestamps related to patient flow through the clinical encounter. This analysis was supplemented by a chart review of clinical events related to the delivery of retina care. We also assessed the impact of these operational changes on our patients’ satisfaction with their care by comparing patient experience surveys during the pre-COVID and COVID eras. Finally, we used these COVID-era metrics as variables in a model to simulate clinic flow with the objective of predicting operational performance under conditions of increasing patient and provider volumes. We specifically evaluated two related areas of clinic performance: patient safety, defined as the ability to maintain physical distancing and operate within clinic-defined capacity, and clinic performance, reflected by average patient wait time, total visit length, and duration of a clinic day required to care for all patients.

Methods

This research followed the tenets of the Declaration of Helsinki and was approved as a quality improvement initiative by the institutional review board of the Lahey Hospital & Medical Center. Information was gathered and secured in compliance with the Health Insurance Portability and Accountability Act. The requirement for informed consent was waived because of the retrospective nature of the study.

Electronic health record data extraction

An analysis was made of eight weeks’ worth of electronic medical records (EMR) from our academic multispecialty group practice facilities spanning the dates of March 15 to May 15 in 2019 and 2020. These dates were chosen to cover the period from the Massachusetts stay-at-home order, which took effect Monday, March 16, 2020 through the peak number of daily COVID-19 cases on April 17, 2020, as well as the period in May when cases in Massachusetts began to decline [5]. EMR-based events for all patients identified during this period were extracted using a customized report designed for use in our EMR (Epic Systems Software, Inc., Verona, Wisconsin, USA). Data collected from EMR generated reports included demographic information such as age, timestamps for patient-care events, such as check-in-to-technician and check-in-to-provider times, and visit completion and check-out timestamps. A detailed review of in-person retina encounters for the month of April 2020 was compared with 2019 (21 working clinic days) to ascertain overall process time. The focus of this case study on the retina clinic during this month was due to the subspecialty’s sufficiently high patient volume and more urgent case-mix. Finally, we performed a retrospective review of 200 randomly selected charts for patients seen in the retina clinic within each period to detail changes in procedure volume. Main outcome metrics included the number of measurements of intraocular pressure (IOP) and number of dilations performed, as well as ancillary procedure volume, including utilization of ophthalmic imaging and intravitreal administration of medications.

Timestamp calculations

Visit length was calculated by taking the time difference between the earliest and latest timestamps available for each patient’s visit. The earliest timestamp was typically “check-in time”. Of the two timestamps, “visit complete” and “visit complete, note pending,” the earlier was regarded as the end time of the visit. Check-in-to-technician time was used as a proxy for waiting room time for the simulation. Check-in-to-image-completion time was calculated by taking the time elapsed from a patient checking-in to the time of image completion. Time with technician was estimated by subtracting check-in-to-technician time from the earliest of the following timestamps: “In with technician,” “Done rooming,” “Dilating in waiting room,” or “Waiting for imaging.” For analysis, time spent in the provider phase of care was defined as the difference between check-in-to-provider time and total length of the visit. For the simulation, this number was halved to account for the fact that often two rooms were simultaneously assigned as “in with provider,” since no EMR-based timestamp delineates the actual location of the provider. Encounters with missing or erroneous timestamps were excluded from timestamp calculations but were still included in patient volume analyses.

Assessment of patient satisfaction

The 15-item Medical Practice Survey was used to assess patient satisfaction (Press Ganey Associates, LLC) [13]. Outpatient discharge records from retina patients seen in-person in the clinic were randomly selected for postal mailings. The period of the COVID-related public health emergency (March 2020 to May 2020) was compared with the same period one year prior. Completed questionnaires were collected by mail, Internet and phone. The survey response format was Likert-type, on a scale from 1 to 5 as follows: Very Poor (1), Poor (2), Fair (3), Good (4), Very Good (5). A patient was considered satisfied with their experience in each category if they gave it a very good rating of 5. Scores of 1 through 4 were considered low satisfaction.

Computer simulation model of clinic flow

To model clinic workflow, a discrete event simulation was utilized using Arena (Rockwell Automation, Wexford, Pennsylvania). EMR timestamp outcomes from retina visits in the month of April 2020 were used to set the variables in the simulation. Patients arrived on a schedule to simulate our retina clinic, which has 15-minute appointment blocks for 2-hours 45-minutes in the morning and afternoon, including a 1-hour 45-minute break without patient arrivals between sessions. The waiting room capacity was set at 10 people to reflect the need for spacing patients at least 6 feet apart. This was achieved by programming into the model a “hold queue” where patients would be placed before entering the simulated clinic, if the simulation detected a count of ten people within any of the wait-processes prior to being assigned a room. This count could include patient escorts or guests, which were estimated to arrive with one out of every fifteen patients. The guest limit per patient was set to a figure in-line with COVID-era guidelines [1]. There were two additional holds after the average waiting room time: one that scanned to make sure there was an empty room available, and another to make sure a technician was available. There was an additional three- to five-minute delay added to clean each exam room after it had been occupied. At a random point in the simulation, up to one “urgent” patient that skipped ahead in the arrival queues was included to simulate a disruption in the scheduled clinic flow.

The model was run for patient volumes of 10, 20, 25, 30, and 40 patients per provider, utilizing two different staff capacities: one provider with two technicians, and two providers with three technicians, reflecting the typical staffing of our retina clinic. These configurations will be referred to as the 1-provider and 2-provider models, respectively, each with 4 rooms available per provider. The following outcome measurements were generated by the model: simulated length of the clinic day, wait time experienced by patients before being placed in an exam room, total length of time each patient was in clinic, average length of time that the waiting room was at capacity, and maximum number of patients waiting to enter clinic during this period.

Statistical analysis

Data were recorded in Microsoft Excel 2010 (version 14.0, Microsoft Corporation, Redmond, Washington) and analyzed using SPSS for Windows version 21.0 (SPSS Inc., Chicago, Illinois) and R software version 4.0.2 (R Project for Statistical Computing, Vienna, Austria; R Foundation for Statistical Computing; 2019, Available at: https://www.r-project.org). A chi-square test with Bonferroni correction was used for between-group analysis. The Z-test was used to determine the difference between the mean top box response rates of respondents on patient experience surveys. Clinic-timing variables as well as averages generated in 1-provider and 2-provider models were compared using independent t-tests. All tests were 2-sided and p values below 0.05 were considered statistically significant.

Results

In comparing the period of the COVID-19 health emergency from March 15 to May 15 in 2020 to the same period one year before, the total number of in-person ophthalmology visits fell by 88 % from 14,486 to 1,774 visits. Similarly, visits to the retina clinic declined by 62 % from 2,647 to 1,015. Whereas retina visits previously accounted for only 18 % of total ophthalmology visits, within the COVID-period they increased to more than 57 % of in-person appointments (χ2 = 1363, p < 0.0001) (Fig. 1 a). Telemedicine visits, which were not previously utilized in our practice, accounted for 77 % of all ophthalmology encounters during this period in 2020, and represented 49 % of retina encounters (Fig. 1b).

Fig. 1
figure1

In-person and telehealth clinic encounters between March 15th and May 15th in 2019 and 2020. a In-person clinic encounters in 2019 and 2020. In 2019, 18 % of all in-person visits were retina encounters (n = 2,647) whereas in 2020 that proportion rose to 57 % (n = 1,015). b In-person and telemedicine encounters in 2020. In 2020, the retina service saw an increase in the proportion of telehealth visits relative to in-person encounters. Whereas in 2019 telehealth visits did not occur at all, during the peak pandemic period in Massachusetts, telehealth was being leveraged in over 50 % of all retina patient encounters on a weekly basis. Dashed vertical line represents the start of the Massachusetts stay-at-home order on March 16, 2020.

Although there was no difference in the average age of patients treated by the retina service during the period of the COVID-19 health emergency in 2020 compared with the same period in 2019 (74.7 ± 13.6 years versus 74.3 ± 13.6 years, p = 0.490), patients seen in-person in the retina clinic in 2020 were older compared with those seen in-person in 2019 (76.8 ± 13.4 years versus 72.5 ± 13.5 years, p < 0.0001) (Supplemental Fig. 1A). In contrast, retina patients treated by telehealth in 2020 were younger than those patients seen in clinic during the same period in 2020 (72.5 ± 13.5 years versus 74.7 ± 13.6 years versus, p < 0.0001) (Supplemental Fig. 1A). Additional patient demographic and clinical characteristics are presented in Table 1.

Table 1 Patient demographics and average timestamp values in 2019 and 2020

Retina clinic analysis

During the peak of the COVID-19 health emergency in April 2020 retina clinic in-person patient volume declined by 66 % compared with one year prior (425 visits versus 1,245 visits). Imaging fell by more than 90 %, occurring in only 24 % of visits in 2020, compared with 83 % of retina visits in 2019 (p < 0.0001). The average number of patients seen per retina provider per day also declined by more than 50 % to 11.8 ± 1.9 patients (median 12, range of 9 to 14 patients) in 2020, compared with 26.6 ± 3.2 patients (median 28, range of 21 to 29 patients) in 2019 (p < 0.0001).

The retina service also experienced changes in both procedural volume and visit length during the COVID-19 health emergency. The percentage of encounters with a documented IOP measurement declined from 97 to 56 % (p < 0.0001). Whereas dilation occurred in 87 % of encounters in 2019, dilation was performed in only 19 % of eyes during the same period in 2020 (p < 0.0001). Both patients’ eyes were dilated for 62 % of encounters in 2019, while in 2020 this occurred in fewer than 6 % of visits (p < 0.0001). The percentage of visits with one eye dilated declined from 25 to 3 % (p < 0.0001). In contrast, the percentage of retina patients undergoing intravitreal injection of medications rose from 49 % of all encounters in 2019 to 86 % of clinic visits in 2020 (p < 0.0001), and the percent of patients undergoing bilateral injections rose from 9 to 17 % (p = 0.09). The proportion of patients undergoing OCT imaging declined from 71 to 19 % (p < 0.0001), and the proportion of encounters with imaging other than OCT remained similar, occurring in 4 % of retina visits in 2019 compared with 5 % in 2020 (p = 0.733). The average length of the visit to the clinic declined by 46 %, to 47 ± 33 min in 2020 compared with 87 ± 35 min in 2019 (p < 0.001). Time spent in the physician phase of care declined 53 % from 38 ± 33 min to 18 ± 22 min (p < 0.0001). Similarly, retina service check-in to technician, check-in to provider, and check-in to image completion times declined by 79 %, 52 %, and 31 %, respectively (Fig. 2; Table 1).

Fig. 2
figure2

Average in-person encounter task times. Average visit length declined by 46 % from 87 to 47 min. Check-in to provider time declined by 52 % from 69 to 33 min. Check-in to technician time declined by 79 % from 14 to 3 min. Check-in to image completion time declined by 31 % from 45 to 31 min.

Impact of COVID-19 on patient satisfaction

Although significantly fewer outpatient in-person visits took place during the months encompassing the COVID-related health emergency, there was no difference in patient experience survey return rate compared with the same period one year prior (23.8 % versus 26.1 %, p = 0.418). The impact of COVID-related changes to clinic flow had noteworthy impacts on patient satisfaction related to patient access, moving through the visit, and interpersonal communication between patient and nursing assistant and/or provider (Table 2). Patients rated the ease of contact (email, phone, and web portal) at 84.6 % during the COVID-related health emergency compared with 61.0 % in the same months one year earlier (p = 0.001). Satisfaction with the degree to which patients were informed about delays (71.8 % versus 33.3 %, p < 0.001) and wait time at the clinic (68.4 % versus 27.6 %, p < 0.001) also more than doubled during the COVID-period compared with the same period one year earlier. Ratings related to the provider’s concern for questions or worries, explanations about problems or conditions, and effort to include the patient in decisions also all rose, albeit to a lesser extent compared with the pre-COVID period (Table 2). Finally, patient overall rating of “How well the staff worked together to care for you” rose to 95.1 % from 75.0 % one year prior (p = 0.008), paralleling nearly as steep of a rise in the “likelihood of patients to recommend the retina practice overall” (78.3 % versus 93.2 %, p = 0.038).

Table 2 Patient satisfaction results for patients seen in the retina clinic

Several aspects of the patient experience were largely unaffected by the COVID-related changes to care. Concern for both privacy and safety remained highly rated by patients in both periods, and although a slightly higher number of patients expressed confidence in how these aspects of their visits were handled during the COVID-19 health emergency, the change did not reach statistical significance (Table 2). Recommendation of the retina care providers was also largely unaffected by the COVID-related changes (87.2 % versus 80.4 %, p = 0.347).

Discrete‐event simulation modeling

The EMR timestamp results were used to set the following variables in our model: waiting room time (4 ± 5 min), time with technician (19 ± 14 min), examination time (8 ± 6 min), percent of patients requiring dilation (19 %), and percent of patients receiving imaging (24 %). The simulated length of the clinic day increased with increasing patient volumes, reaching a maximum of nearly 12 h at 40-patients-per-provider in both the 1- and 2-provider models (11.5 ± 0.45 h and 11.9 ± 0.36 h, respectively). There was no significant difference in average clinic day length between the 1-provider and 2-provider models at any of the patient volumes tested. At 25-patients-per-provider, the wait time before an arriving patient was assigned to a room became significantly longer in the 2-provider model, compared with the 1-provider model (50.4 ± 20.1 min versus 74.7 ± 28.6 min, p = 0.008). Above this patient volume, it took longer for patients to be assigned to a room in the 1-provider model, compared with the 2-provider model, reaching a significant difference at a patient volume of 40 patients (190.7 ± 48.5 min versus 136.9 ± 25.8 min, p < 0.001) (Fig. 3 a). The average total length of the visit per patient increased with increasing patient volumes and was significantly higher at a patient volume of 25-patients-per-provider in the 1-provider model (108.1 ± 20.8 min versus 90.1 ± 16.7 min, p = 0.005) and 30-patients-per-provider (143.4 ± 23.7 min versus 120.9 ± 18.3 min, p = 0.002) in the 2-provider model (Fig. 3b). At patient volumes above 25-patients-per-provider, the waiting room was above clinic-defined capacity for significantly longer periods of time in the 2-provider model, compared with the 1-provider model (Table 3).

Table 3 Average time the waiting room was at capacity and maximum number of people waiting across increasing patient volumes and provider capacities
Fig. 3
figure3

Simulated patient time in clinic. a Average total wait time before each patient was roomed in both the 1-provider and 2-provider models. b Average visit length in both the 1 provider, 2 technician and 2 provider, 3 technician models. Error bars present 95 % confidence intervals. (** p < 0.01).

Discussion

In response to the COVID-19 pandemic, public health recommendations and national guidelines resulted in a nationwide, temporary reduction in in-person eye care [4, 9]. Within the first six weeks after the announcement of the state-wide stay-at-home order in Massachusetts [4], total ophthalmology volume at our academic multispecialty group practice fell significantly [1]. This decrease in clinical volume made better physical distancing possible between patients and staff, and it freed up much needed physical space to implement updated patient care protocols. Our study reports the impact of these changes in the retina clinic during the period when this service provided uninterrupted, essential care to a large number of patients despite the public health emergency. Our results show that eye care can be delivered while maintaining COVID-era precautions and that patient satisfaction was enhanced by these measures. Our study is also among the first to use EMR-timestamped reports to study the clinical transformation, efficiency, and patient flow changes to outpatient systems during the COVID-era. While other fields of medicine such as gastroenterology, cardiology and nephrology utilized discrete-event simulation to study changes in operations during the COVID-19 period [12,13,14,15,16], the present study is the first to do so in ophthalmology. Using EMR-derived timestamps from actual patient encounters, we present a discrete-event simulation model capable of extrapolating from clinic-level data the impact of COVID-era changes on patient flow with the aim of planning for future patient volume increases. This is relevant because many of the measures instituted in response to the pandemic are likely to remain in place for the foreseeable future. Furthermore, the dynamic nature of the model allows for emerging changes to be simulated before actually being deployed in the clinic.

The retina service underwent a major transformation in the way care was delivered during the early stages of the pandemic compared with the same time period in 2019. As a result of intentionally reducing patient and procedural volume, the retina service achieved significant reductions in the average length of visit, check-in to technician time, image completion time, and time in the provider phase of care. Although other studies also demonstrated similar changes, including decreased patient and procedural volume, they did not study the impact of such changes on clinic time [17]. The improved patient experience was also reflected by increased satisfaction with being provided information about delays and total length of the visit. By contrast, the largely unaffected likelihood of patients being willing to recommend their care providers reflects the enduring nature of the patient-physician relationship, which was unaffected by these changes made in response to COVID-19. However, only patients with relatively urgent or emergent conditions were granted access to the clinic during the period of the COVID-19 health emergency, and this restriction to a subset of patients may bias our patient satisfaction results. Other studies have demonstrated that pandemic-related postponement in patient care was significantly associated with worse short-term outcomes [18, 19]. We note that nearly half of retina visits were conducted by telehealth during this period and that patient satisfaction with both access and the provider were rated at similarly high levels for telehealth encounters compared with in-person encounters (data not shown). We did not study clinical outcomes. However, our findings suggest that outcomes were likely favorable for the patients who used telehealth services compared to in-person services because patients were similarly satisfied with the care and access they received. No patient satisfaction data exist to compare patient experience with telehealth during the COVID-19 health emergency with the period before, as telehealth visits did not occur in 2019. Other studies have reported findings on the acceptance of telemedicine by patients in ophthalmology [20,21,22]. These studies found that a wide range of acute and routine diagnoses were managed via video visits and patients rated their experiences highly and would consider using such consultations into the future [20,21,22]. The fact that COVID-era changes are likely to remain, especially those pertaining to telehealth, supports the use of a dynamic modeling system, such as the one we used in this study, to further help predict and support clinic operational changes into the future.

To increase clinic throughput during the outbreak of COVID-19, the eye exam was streamlined, and ancillary testing was either reduced or performed at different times [1]. These changes decreased the amount of time that each patient spent in the clinic, facilitating physical distancing. Previously, it was common to dilate both eyes (62 %), whereas during this period, one or both eyes were dilated only 19 % of the time. This reflects the relatively greater proportion of problem-focused visits conducted during the COVID-period, since routine retinal exams were postponed [1, 7]. The proportion of encounters with imaging also declined significantly between the two periods. Although the number of OCT studies declined precipitously during the COVID-period, the rate of other types of imaging studies, e.g., fluorescein angiograms, performed relative to the number of visits remained unchanged compared with the same period one year earlier. This finding indicates that when diagnostic imaging was required, the service was still able to be delivered, similar to findings suggested by other studies of ophthalmology practice changes during the COVID-era where imaging was only done when diagnostically critical during the COVID-19 period [11].

In parallel, the proportion of patients undergoing intravitreal injections of medications significantly increased during the COVID-era in this study. This increased proportion, given the decline in other eye care services noted above, suggests the need for treatment with an intravitreal agent was the most likely reason for a patient to be seen in the retina clinic during the peak of the COVID-19 public health emergency in our region. Interestingly, other studies also revealed decreases in the overall number of intravitreal injections performed during the COVID-19 period compared to one year prior. One study found a 50 % reduction in injections using a chart review [23], while another found only a 9.9 % absolute reduction [24]. In our study, a greater proportion of patients undergoing treatment with intravitreal injections increased the productivity per retina visit during this period, thereby at least partially offsetting some of the loss of clinic volume. A larger proportion of patients also underwent bilateral injections in 2020 compared with 2019 in our study. One of the likely reasons is that patients elected to accept bilateral injections in an effort to make fewer visits to the clinic. Whether this change was driven by patient or provider preference cannot be determined from the evidence available, but the net effect was a further reduction in the number of in-person appointments. The impact of these changes on the effectiveness of care delivered and patient outcomes, if any, remains to be determined.

The mean age of patients attending the retina clinic in 2020 was older than in 2019 and higher compared with 2020 telemedicine encounters. This is likely because older patients more often have serious eye conditions that require immediate, in-person attention compared to younger patients. Further, older patients are more likely to have conditions that require treatment with intravitreal injections of medication. Older patients may also be generally less comfortable with telemedicine visits, preferring in-person care. However, providers played a leading role in deciding which patients were to have in-person visits. Another large study assessing the effectiveness of their virtual ophthalmic care found the median age of their virtual visit patients to be 32 years old, which was significantly younger than that of face-to-face patients at 45 years old [25]. This study found that retinopathy was the most common reason for in-person visits, similar to our study, while ocular surface diseases were most cited in virtual appointments [25]. This suggests that telemedicine visits may be used to effectively triage patient complaints.

Discrete-event simulation provides a model of the operations of a system in sequence, and it has been used extensively as a tool for analyzing healthcare systems with a goal of quality improvement [26]. In ophthalmology, discrete-event simulation has been effectively employed to model outpatient clinic flow and capacity restraints [27,28,29,30], and results have been validated by the inclusion of time-stamped EMR data [31]. Our model sought to assess the impact of measures adopted after the outbreak of COVID-19 on retina clinic operations. Some of the changes included limiting the waiting room capacity to maintain at least a six-foot separation between patients, restricting patients to a single exam room rather than moving patients between multiple technician and provider rooms, and ensuring that check-in and check-out processes allow for adequate social distancing. Our model provides insight into potential bottlenecks created by these changes. Finding alternative spaces to stage patients will be an important strategy, especially during months of the year when it is less practical to recommend that patients wait outside of the clinic, for example, in their personal vehicles [1]. The model also found that at the highest patient volumes, patients waited less time to be roomed in the 2-provider model but spent the same amount of total time in the clinic compared with the 1-provider model. This reveals that the bottleneck is not attributable to the COVID-19 rooming system per se, but rather to provider availability at higher patient volumes. In the future, it may even be possible to run the model in conjunction with actual operation of the clinic to adjust dynamically resource allocation, e.g., reassigning rooms or technical staff among providers. This information could also enable the scheduling staff to communicate projected delays to patients who have yet to arrive, move patients to another provider’s schedule, or even preemptively reschedule visits in response to the volume of patients and wait times. Finally, the model can also be used to assess utilization of clinic capacity, thereby informing scheduling decisions related to the ratio of patients to providers or other types of staff in order to optimize clinical productivity.

There are limitations to this type of study. Foremost is that time-stamped data depends on user input for consistency, and in our study, in-person observation or video recording was not available to validate our internal timestamps for accuracy [31]. Out of 1670 patient encounters examined, all but one had one or more time-stamped phases of care. However, in total, 36 % of visits had at least one missing timestamp, and the omission rate was substantially greater in the COVID-19 period compared with the period one year prior (82 % versus 21 % of encounters, p < 0.001). As the recording of these events happen behind the scenes in the EMR, this suggests that staff may have been less accurate in documenting changes in the phases of care during the period of COVID-19 as they moved through the visit. Another limitation inherent to our model is that it uses assumptions based on EMR timestamps and event rates derived from a period when the clinic was far below historical utilization rates and seeing far fewer patients. Although this period reflects the completely redesigned patient flow and practice measures that reflect COVID-era restrictions, it remains to be seen if all of the assumptions made in the model hold as the clinic returns to full capacity. When more patients start to return for routine visits, the proportion of eyes being dilated, receiving IOP checks, and/or being imaged is likely to increase. In keeping with changes in these rates, it will be necessary to re-run the model, which will increase the accuracy of predictions for clinic operations. Finally, it will be important to assess if patient satisfaction levels are maintained as the number of patients increases and more routine care is provided.

Conclusions

During the COVID-19 stay-at-home order, ophthalmology and, specifically, the retina service saw major shifts in the way eye care was delivered. Maintaining these improvements in performance, enhanced cleaning, and social distancing policies will pose a significant challenge when the demand for eye care increases. Discrete-event simulation models are an important strategy to allow practices to plan for volume increases while maintaining a safe clinical environment. Our findings support the conclusion that an ophthalmology practice is highly adaptable. The changes implemented enhanced the effective delivery of eye care and improved the patient’s sense of well-being, thus potentially becoming a new standard of care.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to the use of internal and confidential patient medical record data stored on internal, confidential and protected hard drives, but de-identified and redacted data are available from the corresponding author on reasonable request.

Abbreviations

COVID-19:

coronavirus disease 2019

SARS‐CoV‐2:

severe acute respiratory syndrome coronavirus 2

IOP:

intraocular pressure

EMR:

electronic medical record

OCT:

optical coherence tomography

References

  1. 1.

    Goldblatt J, Roh S, Sethi K, Ramsey DJ. Enhancing safety and performance in the COVID-19 era: strategies and modifications that may become the standard of care in ophthalmology practice. Ophthalmol Manag. 2020;24:E2.

    Google Scholar 

  2. 2.

    Chodosh J, Holland GN, Yeh S. Important coronavirus updates for ophthalmologists. American Academy of Ophthalmology. https://www.aao.org/headline/alert-important-coronavirus-context. Published May 11, 2020.Accessed 16 May 2020.

  3. 3.

    Liu J, Wang A, Ing EB. Efficacy of slit lamp breath shields. Am J Ophthalmol. 2020;218:120–7.

    CAS  Article  Google Scholar 

  4. 4.

    Release P. Baker-Polito administration announces emergency actions to address COVID-19. https://www.mass.gov/news/baker-polito-administration-announces-emergency-actions-to-address-covid-19. Published March 15, 2020. Accessed 28 April 2020.

  5. 5.

    COVID-19 Map. Johns Hopkins Coronavirus Resource Center https://coronavirus.jhu.edu/map.html. Updated June 7, 2020. Accessed 7 June 2020.

  6. 6.

    CDC COVID Data Tracker. United States COVID-19 Cases and deaths by state. Centers for Disease Control and Prevention. https://www.cdc.gov/covid-data-tracker/. Updated June 7, 2020. Accessed 7 June 2020.

  7. 7.

    Recommendations for urgent and nonurgent patient care. American Academy of Ophthalmology. https://www.aao.org/headline/new-recommendations-urgent-nonurgent-patient-care. Published March 18, 2020. Accessed 18 March 2020.

  8. 8.

    Gharebagi R, Desuatels J, Moshirfar M, Parvizi M, Daryabari SH, Heidary F. COVID-19: preliminary clinical guidelines for ophthalmology practices. Med Hypothesis Discov Innov Ophthalmol. 2020;9:149–58.

    Google Scholar 

  9. 9.

    Gupta PC, Kumar MP, Ram J. COVID-19 pandemic from an ophthalmology point of view. Indian J Med Res. 2020;151:411–8.

    CAS  Article  Google Scholar 

  10. 10.

    Korobelnik JF, Loewenstein A, Eldem B, Joussen AM, Koh A, Lambrou GN, et al. Guidance for anti-VEGF intravitreal injections during the COVID-19 pandemic. Graefes Arch Clin Exp Ophthalmol. 2020;258:1149–56.

    CAS  Article  Google Scholar 

  11. 11.

    Gupta V, Rajendran A, Narayanan R, Chawla S, Kumar A, Palanivelu MS, et al. Evolving consensus on managing vitreo-retina and uvea practice in post-COVID-19 pandemic era. Indian J Ophthalmol. 2020;68:962–73.

    Article  Google Scholar 

  12. 12.

    Corazza P, D’Alterio FM, Younis S. Proposed algorithm during COVID-19 pandemic for patient management in medical retina clinic. Int J Retina Vitreous. 2020;6:20.

    Article  Google Scholar 

  13. 13.

    Press G, Associates LLC. Medical Practice Survey Psychometrics Report. South Bend: Press Ganey Associates, LLC; 2019.

    Google Scholar 

  14. 14.

    Das A. Impact of the COVID-19 pandemic on the workflow of an ambulatory endoscopy center: an assessment by discrete event simulation. Gastrointest Endosc. 2020;92:914–24.

    Article  Google Scholar 

  15. 15.

    Nan J, Zhang T, Tian Y, Song K, Li Q, Fu Q, et al. Impact of the 2019 novel coronavirus disease pandemic on the performance of a cardiovascular department in a non-epidemic center in Beijing, China. Front Cardiovasc Med. 2021;8:630816.

    Article  Google Scholar 

  16. 16.

    Allen M, Bhanji A, Willemsen J, Dudfield S, Logan S, Monks T. A simulation modelling toolkit for organizing outpatient dialysis services during the COVID-19 pandemic. PLoS One. 2020;15:e0237628.

    CAS  Article  Google Scholar 

  17. 17.

    Breazzano MP, Nair AA, Arevalo JF, Barakat MR, Berrocal AM, Chang KS, et al. Frequency of urgent or emergent vitreoretinal surgical procedures in the United States during the COVID-19 pandemic. JAMA Ophthalmol. 2021;139(4):456–63. https://doi.org/10.1001/jamaophthalmol.2021.0036.

    Article  PubMed  Google Scholar 

  18. 18.

    Borrelli E, Grosso D, Vella G, Sacconi R, Battista M, Querques L, et al. Short-term outcomes of patient with neovascular exudative AMD: the effect of COVID-19 pandemic. Graefes Arch Clin Exp Ophthalmol. 2020;258:2621–8.

    Article  Google Scholar 

  19. 19.

    Song W, Singh RP, Rachitskaya AV. The effect of delay in care among patients requiring intravitreal injections. Ophthalmol Retina. 2021. S2468-6530(20)30507-8. https://doi.org/10.1016/j.oret.2020.12.020.

  20. 20.

    Kalra G, Williams AM, Commisky PW, Bowers EMR, Schempf T, Sahel JA, et al. Incorporating video visits into ophthalmology practice: a retrospective analysis and patient survey to assess initial experiences and patient acceptability at an academic eye center. Ophthalmol Ther. 2020;9:549–62.

    Article  Google Scholar 

  21. 21.

    Saleem SM, Pasquale LR, Sidoti PA, Tsai JC. Virtual ophthalmology: telemedicine in a COVID-19 Era. Am J Ophthalmol. 2020;216:237–42.

    CAS  Article  Google Scholar 

  22. 22.

    Lindeke-Myers A, Zhao PYC, Meyer BI, Liu EA, Levine DA, Bennett OM, et al. Patient perceptions of SARS-CoV-2 exposure risk and association with continuity of ophthalmic care. JAMA Ophthalmol. 2021. e210114. https://doi.org/10.1001/jamaophthalmol.2021.0114.

  23. 23.

    Borrelli E, Grosso D, Vella G, Sacconi R, Querques L, Zucchiatti I, et al. Impact of COVID-19 on outpatient visits and intravitreal treatment in a referral retina unit: let’s be ready for a plausible “rebound effect”. Graefes Arch Clin Exp Ophthalmol. 2020;258:2655–60.

    Article  Google Scholar 

  24. 24.

    Ashkenazy N, Goduni L, Smiddy WE. Short-term effects of COVID-19-related deferral of intravitreal injection visits. Clin Ophthalmol. 2021;15:413–7.

    Article  Google Scholar 

  25. 25.

    Wu X, Chen J, Yun D, Yuan M, Liu Z, Yan P, et al. Effectiveness of an ophthalmic hospital-based virtual service during the COVID-19 pandemic. Ophthalmology. 2020. S0161-6420(20)31010-1. https://doi.org/10.1016/j.ophtha.2020.10.012.

  26. 26.

    Zhang X. Application of discrete event simulation in health care: a systematic review. BMC Health Serv Res. 2018;18:687.

    Article  Google Scholar 

  27. 27.

    Demir E, Southern D, Verner A, Amoaku W. A simulation tool for better management of retinal services. BMC Health Serv Res. 2018;18:759.

    Article  Google Scholar 

  28. 28.

    Crane GJ, Kymes SM, Hiller JE, Casson R, Martin A, Karnon JD. Accounting for costs, QALYs, and capacity constraints: using discrete-event simulation to evaluate alternative service delivery and organizational scenarios for hospital-based glaucoma services. Med Decis Making. 2013;33:986–97.

    Article  Google Scholar 

  29. 29.

    Pan C, Zhang D, Kon AWM, Wai CSL, Ang WB. Patient flow improvement for an ophthalmic specialist outpatient clinic with aid of discrete event simulation and design of experiment. Health Care Manag Sci. 2015;18:137–55.

    Article  Google Scholar 

  30. 30.

    Kern C, König A, Fu DJ, Schworm B, Wolf A, Priglinger S, Kortuem KU. Big data simulations for capacity improvement in a general ophthalmology clinic. Graefes Arch Clin Exp Ophthalmol. 2021. https://doi.org/10.1007/s00417-020-05040-9 Online ahead of print.

    Article  PubMed  Google Scholar 

  31. 31.

    Hribar MR, Huang AE, Goldstein IH, Reznick LG, Kuo A, Loh AR, et al. Data-driven scheduling for improving patient efficiency in ophthalmology clinics. Ophthalmology. 2019;126:347–54.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Dr. Hannah K. Galvin, Sidrah Anjum, Karen Latulippe, Stacy Florentino, Ed Momplaisir, Rosemary Immig, Kira Szulborski, and Carol Spencer for research support. D.J. Ramsey is the Harry N. Lee Family Chair in Innovation at the Lahey Hospital & Medical Center, Beth Israel Lahey Health.

Funding

D.J.R.: Supported by the Harry N. Lee Family Chair in Innovation at the Lahey Hospital & Medical Center, Beth Israel Lahey Health.

Author information

Affiliations

Authors

Contributions

KS organized, analyzed and interpreted patient data, created figures and was a major contributor in writing the manuscript. ESL analyzed and interpreted patient data, performed the modeling, created figures and was a major contributor in writing the manuscript. SR interpreted patient data, provided overall guidance and support and was a major contributor in writing the manuscript. JLM interpreted patient data, provided overall guidance and support, and was a contributor in revising the manuscript. DJR was responsible for the study hypothesis and design, the development of electronic medical record reporting tools, reviewing and analyzing the data, interpreting the results, provided and interpreted patient data, provided overall guidance and support, and writing of the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to David J. Ramsey.

Ethics declarations

Ethics approval and consent to participate

The research followed the tenets of the Declaration of Helsinki and was approved as a quality improvement initiative by the institutional review board of the Lahey Hospital & Medical Center, Burlington, MA. Information was gathered and secured in compliance with the Health Insurance Portability and Accountability Act. The requirement for informed consent was waived because of the retrospective nature of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1:

Figure S1. Density plot of the age distribution of retina patients. aPatients seen in-person in the retina clinic in 2020 were older compared with those seen in-person in 2019. b In contrast, retina patients treated by telehealth in 2020 were younger than those patients seen in clinic during the same period in 2020.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sethi, K., Levine, E.S., Roh, S. et al. Modeling the impact of COVID-19 on Retina Clinic Performance. BMC Ophthalmol 21, 206 (2021). https://doi.org/10.1186/s12886-021-01955-x

Download citation

Keywords

  • COVID-19
  • Health Services Research
  • Discrete‐event simulation
  • Patient flow
  • Outpatient