- Open Access
- Open Peer Review
Patient-centred measurement in ophthalmology – a paradigm shift
BMC Ophthalmologyvolume 6, Article number: 25 (2006)
Ophthalmologists and researchers in ophthalmology understand what a rapidly evolving field ophthalmology is, and that to conduct good research it is essential to use the latest and best methods. In outcomes research, one modern initiative has been to conduct holistic measurement of outcomes inclusive of the patient's point of view; patient-centred outcome. This, of course, means including a questionnaire. However, the irony of trying to improve outcomes research by being inclusive of many measures is that the researcher may not be expert in all measures used. Certainly, few people conducting outcomes research in ophthalmology would claim to be questionnaire experts. Most tend to be experts in their ophthalmic subspecialty and probably simply choose a popular questionnaire that appears to fit their needs and think little more about it. Perhaps, unlike our own field, we assume that the field of questionnaire research is relatively stable. This is far from the case. The measurement of patient-centred outcomes with questionnaires is a rapidly evolving field. Indeed, over the last few years a paradigm shift has occurred in patient-centred measurement.
Measurement of patient-centred health outcomes began in the 1950s, and more than 70 measures of functional status, 2 dozen generic quality of life instruments, and hundreds of disease-specific measures now exist [1–4]. Much of this research has relied on the theory and methods of Classical Test Theory (CTT). Underpinning CTT is simple scoring of responses to questionnaires; for multi-category response scales this is usually summary scoring where response categories are assigned ordinal numbers which are summed across questions to arrive at a total score. This score is assumed to represent measurement of the underlying trait (e.g. quality of life). An alternative approach is Item Response Theory (IRT) whereby items and persons can be scaled according to a series of responses to items. Implicit is that not only can people have different ability, but that items can have different difficulty and both can be estimated. The foundations of IRT were laid down in the 1920s, and great advances were made after 1950, especially with the contribution of Georg Rasch [5, 6]. Rasch analysis is a special case of IRT similar to a one-parameter model, but importantly, Rasch models meet the conditions of non-interactive conjoint structures so, unlike IRT models, they are valid measurement models . (See Massof's excellent background paper for further discussion of these methods as applied to ophthalmology ). It has only been in the last 25 years that Rasch analysis has been applied in studies of health status. Early applications included mental health and physical rehabilitation in the 1980s [9, 10], including low vision rehabilitation [11, 12]. Through the 1990s, and into the 21st century, Rasch analysis has penetrated many health outcome fields and is becoming prevalent in ophthalmology.
However, many popular ophthalmic questionnaires use traditional summary scoring . Summary scoring assumes the value of each item represents equal difficulty and therefore scores them equally. In addition, the ordinal integer response scale used for each item assumes uniform changes between response categories. For example, in a summary scored vision disability instrument such as the 'Activities of Daily Vision Scale (ADVS)', a response of "a little difficulty" (score of 4) is used to represent twice the level of ability as "extreme difficulty" (score of 2) which is similarly two times as good as "unable to perform the activity due to vision" (score of 1) for all items. This appears illogical and Rasch analysis has been used to confirm that specific response category calibrations are required to provide a linear scale . Similarly, summary scoring assumes that all items are of equal difficulty. For example, with the ADVS instrument an answer of "a little difficulty" to the question regarding visual difficulties 'driving at night' scores the same as "a little difficulty" with 'driving during the day'. Again, this is illogical and Rasch analysis has been used to confirm that subjects report that 'driving at night' is a more difficult task than 'driving during the day'; Rasch analysis can provide the appropriate weighting for each item to enable linear measurement .
By resolving inequities in a scale arising from differential item difficulty, Rasch analysis provides a self-evident benefit in terms of accuracy of scoring. However, this process also removes noise from the measurement which in turn improves sensitivity to change and correlations with other variables [16, 17]. Clearly, these are important benefits for outcomes research with implications for sample sizes etc. The assumption of unidimensionality inherent in Rasch analysis also provides unparalleled insight into the dimensionality of a questionnaire. This can be used to advantage in questionnaire development. Therefore, beyond its simple application, Rasch analysis has been used in three important ways in ophthalmology: the development of new questionnaires, shortening or revising existing questionnaires, and test equating.
Across ophthalmology, a number of questionnaires have been developed using Rasch analysis. The field of low vision has led the way with the Veterans Affairs Low-Vision Visual Functioning Questionnaire, the Melbourne Low Vision ADL Index and the Visual Function Questionnaire [18–22]. Two Rasch scaled questionnaires have also been developed for refractive outcomes: one for spectacles, contact lenses and refractive surgery (The Quality of Life Impact of Refractive Correction, QIRC) and one for contact lenses only (Contact lens Impact on Quality of Life, CLIQ) [23–26]. While other sub-fields of ophthalmology are yet to gain questionnaires developed using Rasch analysis, these are likely not far away as a recent major review of quality of life in glaucoma called for the development of a Rasch scaled glaucoma-specific visual disability questionnaire .
Existing conventionally validated questionnaires can be rescaled using Rasch analysis. In this way, summary scoring can be converted to a truly linear scale using a simple formula. This has occurred for cataract surgery outcomes , refractive surgery outcomes , age-related macular degeneration outcomes , low vision care , and the measurement of ophthalmic pain [31, 32]. Some of these approaches have been to simply rescore an existing questionnaire, while others have taken the approach of completely re-engineering a questionnaire to optimize its performance. The latter process takes advantage of the insight into dimensionality of a questionnaire to refine measurement and improve targeting of content to the population. Notable examples of questionnaire re-engineering are the Impact of Visual Impairment (IVI) for low vision outcomes [33, 34], the Visual Disability Assessment (VDA) reinvented as the Cataract Outcomes Questionnaire [28, 35], and The Visual Functioning 14 (VF-14) [36–38]. Usually, re-engineering of questionnaires involves removal of poorly fitting items to make a shorter questionnaire, although Velozo et al actually added items to the VF-14 and yet were unable to completely satisfactorily improve the instrument . Rasch analysis has also been used to simply confirm the performance of, or detect deficiencies in questionnaires [39–42].
In ophthalmology, the most important patient-centred outcome measure is visual disability. Visual disability is the reason for performing cataract surgery, world wide the most common operation performed, and visual disability is a consequence of many ophthalmic diseases. Accordingly, there are many visual disability questionnaires e.g.: the Visual Activities Questionnaire (VAQ) , the Activities of Daily Vision Scale (ADVS) questionnaire , the VF-14 , the VDA , the National-Eye Institute Visual Functioning Questionnaire (NEI-VFQ) and the Catquest questionnaire . These questionnaires are widely used in different part of the world, for instance Catquest is widely used in Europe, the VF-14 in North America, and the VDA in Australia. While these questionnaires all measure the same concept, and have many of the same items, their scores cannot be simply compared. However, Rasch analysis provides a mechanism to equate scores from different questionnaires. Since all these questionnaires measure the same underlying trait, visual disability, they can all be modelled on the same latent variable. Massof has made an important first step in equating visual disability questionnaires implementing the ADVS, VAQ, VF-14 and the NEI-VFQ on a large population, conducting Rasch analysis and providing equations for conversion of summary scores to Rasch scores and conversion between questionnaires . While Rasch analysis is in itself fairly simple, its application is somewhat specialized. Massof's equations provide a mechanism to gain the benefits of Rasch scaling without having to perform Rasch analysis.
Today, the outcomes researcher is faced with many different published questionnaires to choose from; some summary scored, others Rasch scaled. A major recent review by De Boer et al has attempted to make sense of this choice by systematically classifying questionnaires by the quality of their development and validation . Notably, de Boer et al included Rasch scaling as a point of differentiation. A logical extension of this would be to rate highest questionnaires developed using Rasch analysis, followed by those conventionally developed and re-engineered using Rasch analysis and then those simply re-scored using Rasch analysis. There seems to be little advantage in using non-Rasch scaled questionnaires given this will increase noise and therefore reduce statistical power. Despite this, summary scored questionnaires remain popular , perhaps due to the simplicity of their scoring. But, many questionnaire developers, and other researchers, provide simple Rasch scale conversion obviating the need for questionnaire users to perform Rasch analysis [22, 23, 48]. This simple step would have added significant value to the paper by Owen et al published in this edition of BMC Ophthalmology . Hopefully, others will agree that the time has come to abandon summary scoring. Lets just hope that Max Planck's pessimism about scientific change: "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it" does not hold true for the Rasch analysis paradigm shift.
McHorney CA: Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann Intern Med. 1997, 127: 743-750.
McHorney CA: Health status assessment methods for adults: past accomplishments and future challenges. Annu Rev Public Health. 1999, 20: 309-335. 10.1146/annurev.publhealth.20.1.309.
McHorney CA: Use of item response theory to link 3 modules of functional status items from the Asset and Health Dynamics Among the Oldest Old study. Arch Phys Med Rehabil. 2002, 83: 383-394. 10.1053/apmr.2002.29610.
McHorney CA, Monahan PO: Postscript: Applications of Rasch analysis in health care. Med Care. 2004, 42: I73-8.
Bock R: A brief history of itern response theory. Educ Measure Issues Pract. 1997, 21-33. 10.1111/j.1745-3992.1997.tb00605.x.
Rasch BG: Probabilistic Models for Some Intelligence and Attaiment Tests. 1960, Copenhagen, Denmark, Danmarks Paedogogiske Institut
Fisher WPJ: The Rasch debate: Validity and revolution in educational measurement. Objective measurement: Theory into practice. Edited by: Wilson M. 1994, Norwood, NJ, Ablex, 2: 36-72.
Massof RW: The measurement of vision disability. Optom Vis Sci. 2002, 79: 516-552. 10.1097/00006324-200208000-00015.
Barrett PT, Kline P: A comparison between Rasch analysis and factor analysis of item in the EPQ. Pers Study Group Behav. 1981, 1: 11-28.
Wright BD, Linacre JM: Observations are always ordinal, measurements must be interval. Arch Phys Med Rehabil. 1989, 70: 857-860.
Schulz EM: Functional Assessment In Rehabilitation: An Example With The Visually Impaired. 1987, , Universirty of Chicago
Becker SW, Lambert RW, Schulz EM, Wright BD, Burnet DL: An instrument to measure the activity level of the blind. Int J Rehabil Res. 1985, 8: 415-424.
Likert R: A technique for the measurement of attitudes. Arch of Psychol. 1932, 140: 1-55.
Mangione CM, Phillips RS, Seddon JM, Lawrence MG, Cook EF, Dailey R, Goldman L: Development of the 'Activities of Daily Vision Scale'. A measure of visual functional status. Med Care. 1992, 30: 1111-1126.
Pesudovs K, Garamendi E, Keeves JP, Elliott DB: The Activities of Daily Vision Scale for cataract surgery outcomes: re-evaluating validity with Rasch analysis. Invest Ophthalmol Vis Sci. 2003, 44: 2892-2899. 10.1167/iovs.02-1075.
Norquist JM, Fitzpatrick R, Dawson J, Jenkinson C: Comparing alternative Rasch-based methods vs raw scores in measuring change in health. Med Care. 2004, 42: I25-36.
Garamendi E, Pesudovs K, Stevens MJ, Elliott DB: The Refractive Status and Vision Profile: Evaluation of psychometric properties and comparison of Rasch and summated Likert-scaling. Vision Res. 2006, 46: 1375-1383. 10.1016/j.visres.2005.07.007.
Haymes SA, Johnston AW, Heyes AD: The development of the Melbourne low-vision ADL index: a measure of vision disability. Invest Ophthalmol Vis Sci. 2001, 42: 1215-1225.
Nutheti R, Shamanna BR, Krishnaiah S, Gothwal VK, Thomas R, Rao GN: Perceived visual ability for functional vision performance among persons with low vision in the Indian state of Andhra Pradesh. Invest Ophthalmol Vis Sci. 2004, 45: 3458-3465. 10.1167/iovs.04-0243.
Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C: Visual disability variables. II: The difficulty of tasks for a sample of low-vision patients. Arch Phys Med Rehabil. 2005, 86: 954-967. 10.1016/j.apmr.2004.09.017.
Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C: Visual disability variables. I: the importance and difficulty of activity goals for a sample of low-vision patients. Arch Phys Med Rehabil. 2005, 86: 946-953. 10.1016/j.apmr.2004.09.016.
Stelmack JA, Szlyk JP, Stelmack TR, Demers-Turco P, Williams RT, Moran D, Massof RW: Psychometric properties of the Veterans Affairs Low-Vision Visual Functioning Questionnaire. Invest Ophthalmol Vis Sci. 2004, 45: 3919-3928. 10.1167/iovs.04-0208.
Pesudovs K, Garamendi E, Elliott DB: The Quality of Life Impact of Refractive Correction (QIRC) Questionnaire: development and validation. Optom Vis Sci. 2004, 81: 769-777. 10.1097/00006324-200410000-00009.
Pesudovs K, Garamendi E, Elliott DB: A quality of life comparison of people wearing spectacles or contact lenses or having undergone refractive surgery. J Refract Surg. 2006, 22: 19-27.
Garamendi E, Pesudovs K, Elliott DB: Changes in quality of life after laser in situ keratomileusis for myopia. J Cataract Refract Surg. 2005, 31: 1537-1543. 10.1016/j.jcrs.2004.12.059.
Pesudovs K, Garamendi E, Elliott DB: The Contact Lens Impact on Quality of Life (CLIQ) Questionnaire: Development and Validation. Invest Ophthalmol Vis Sci. 2006, 47: 2789-96. 10.1167/iovs.05-0933.
Spaeth G, Walt J, Keener J: Evaluation of quality of life for patients with glaucoma. Am J Ophthalmol. 2006, 141: S3-14. 10.1016/j.ajo.2005.07.075.
Pesudovs K, Elliott DB, Coster DJ: The Cataract Outcomes Questionnaire - A Rasch Scaled Measure of Visual Disability. Invest Ophthalmol Vis Sci. 2005, 45: ARVO e-abstract 3844.
Hewitt AW, Jeganathan VS, Kidd JE, Pesudovs K, Verma N: Influence of photodynamic therapy for age related macular degeneration upon subjective vision related quality of life. Graefes Arch Clin Exp Ophthalmol. 2006, 1-6. 10.1007/s00417-005-0218-x.
Smith HJ, Dickinson CM, Cacho I, Reeves BC, Harper RA: A randomized controlled trial to determine the effectiveness of prism spectacles for patients with age-related macular degeneration. Arch Ophthalmol. 2005, 123: 1042-1050. 10.1001/archopht.123.8.1042.
Noble BA, Loh RS, MacLennan S, Pesudovs K, Reynolds A, Bridges LR, Burr J, Stewart O, Quereshi S: Comparison of autologous serum eye drops with conventional therapy in a randomised controlled crossover trial for ocular surface disease. Br J Ophthalmol. 2004, 88: 647-652. 10.1136/bjo.2003.026211.
Pesudovs K, Craigie MJ, Roberton G: The visual analogue scale for the measurement of pain is not linear. Anaesth Intensive Care. 2005, 33: 686-7; author reply 687.
Hassell JB, Weih LM, Keeffe JE: A measure of handicap for low vision rehabilitation: the impact of vision impairment profile. Clin Experiment Ophthalmol. 2000, 28: 156-161. 10.1046/j.1442-9071.2000.00312.x.
Lamoureux EL, Pallant JF, Pesudovs K, Hassell JB, Keeffe JE: The Impact of Vision Impairment Questionnaire: an evaluation of its measurement properties using Rasch analysis. Invest Ophthalmol Vis Sci. 2006.
Pesudovs K, Coster DJ: An instrument for assessment of subjective visual disability in cataract patients. Br J Ophthalmol. 1998, 82: 617-624.
Velozo CA, Lai JS, Mallinson T, Hauselman E: Maintaining instrument quality while reducing items: application of Rasch analysis to a self-report of visual function. J Outcome Meas. 2000, 4: 667-680.
Valderas JM, Rue M, Guyatt G, Alonso J: The impact of the VF-14 index, a perceived visual function measure, in the routine management of cataract patients. Qual Life Res. 2005, 14: 1743-1753. 10.1007/s11136-005-1745-y.
Mallinson T, Stelmack J, Velozo C: A comparison of the separation ratio and coefficient alpha in the creation of minimum item sets. Med Care. 2004, 42: I17-24.
Massof RW, Fletcher DC: Evaluation of the NEI visual functioning questionnaire as an interval measure of visual ability in low vision. Vision Res. 2001, 41: 397-413. 10.1016/S0042-6989(00)00249-2.
Babcock-Parziale J, McKnight PE, Head DN: Evaluating psychometric properties of a clinical and a self-report blind rehabilitation outcome measure. J Rehabil Res Dev. 2005, 42: 487-498. 10.1682/JRRD.2004.07.0080.
Hart PM, Stevenson MR, Montgomery AM, Muldrew KA, Chakravarthy U: Further validation of the Daily Living Tasks Dependent on Vision: identification of domains. Br J Ophthalmol. 2005, 89: 1127-1130. 10.1136/bjo.2004.059683.
Patel I, Munoz B, Burke AG, Kayongoya A, Mchiwa W, Schwarzwalder AW, West SK: Impact of Presbyopia on Quality of Life in a Rural African Setting. Ophthalmology. 2006, 113: 728-734. 10.1016/j.ophtha.2006.01.028.
Sloane ME, Ball K, Owsley C, Bruni SR, Roenkar DL: The visual activities questionnaire: developing an instrument for assessing problems in everyday visual tasks. Tech Dig Noninvas Assess Vis Sys. 1992, 1: 26-29.
Mangione CM, Phillips RS, Seddon JM, G LM, Cook EF, Dailey R, Goldman L: Development of the "Activities of Daily Vision Scale". A measure of visual functional status. Med Care. 1992, 30: 1111-1126.
Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, Legro MW, Diener-West M, Bass EB, Damiano AM, Steinwachs DM, Sommer A: The VF-14: An index of functional impairment in patients with cataract. Arch Ophthalmol. 1994, 112: 630-638.
Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD: Development of the 25-item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol. 2001, 119: 1050-1058.
Lundström M, Roos P, Jensen S, Fregell G: Catquest questionnaire for use in cataract surgery care: description, validity, and reliability. J Cataract Refract Surg. 1997, 23: 1226-1236.
Massof RW: Application of stochastic measurement models to visual function rating scale questionnaires. Ophthalmic Epidemiol. 2005, 12: 103-124. 10.1080/09286580590932789.
de Boer MR, Moll AC, de Vet HC, Terwee CB, Volker-Dieben HJ, van Rens GH: Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004, 24: 257-273. 10.1111/j.1475-1313.2004.00187.x.
Owen CG, Rudnicka AR, Smeeth L, Evans JR, Wormald RPL, Fletcher AE: Is the NEI-VFQ-25 a useful tool in identifying visual impairment in an elderly population?. BMC Ophthalmol. 2006.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2415/6/25/prepub