Introduction

The role of retinal carotenoids lutein and (meso-)zeaxanthin, together forming the macular pigment (MP) in the human retina, has been a topic of interest in ophthalmologic research for many years.1, 2, 3, 4 The yellow MP is mainly located in the ganglion cell layers and inner plexiform layers of the retina.5 Typically, the concentration of the MP is maximal at, or near, the fovea and rapidly decreases with eccentricity.6, 7, 8 Because of its pre-receptorial location, MP is thought to shield the retina from deleterious effects of high-energy blue light (λ ∼320 to 450 nm), by partly absorbing it.9, 10, 11 As it functions as an anti-oxidant, MP may protect the retina by scavenging of free radicals formed by oxidative stress.12, 13, 14 Consequently, MP might protect against degenerative eye diseases, such as age-related macular degeneration. Carotenoids cannot be synthesized by the human body and are only available through diet.15 Hence, the amount of MP depends on the amount of lutein- and (meso)zeaxanthin-rich foods we ingest, and can be further influenced by the intake of nutritional supplements containing lutein and (meso)zeaxanthin.2, 16, 17

Accurate assessment of the amount of MP, expressed as MP optical density (MPOD), is therefore necessary to investigate the role of carotenoids and their assumed protective functions. High repeatability and reliability are especially important to monitor patients in studies investigating the influence of diet and/or nutritional (lutein and (meso)zeaxanthin) supplements, or disease processes on MPOD. Methods to determine MPOD should yield reproducible results accompanied by reliable uncertainty estimates, even when operated by non-professional or untrained staff. Reproducibility is hampered when the method is applied in older patients, who may suffer from hazy ocular media, compromised physical skills, and early degeneration of the retina.

The most frequently used technique to asses MPOD is heterochromatic flicker photometry (HFP). Different devices using this technique have been validated to produce reproducible and reliable results in healthy research populations of different ages and patients with signs of early AMD.18, 19, 20 HFP setups are relatively inexpensive and easy to use, even for untrained staff. Furthermore, measurements can be performed through an undilated pupil, and are generally not influenced by changes in the ocular media, such as cataracts.

This was performed to evaluate the repeatability and reliability of a recently introduced device using HFP; Macuscope (Macuvision Europe Ltd., Lapworth, Solihull, UK). To this end, we tested the Macuscope against a second device based on HFP (Quantifeye, Tinsley Precision Instruments Ltd., Croydon, Essex, UK) and fundus reflectometry (MP reflectometer (MPR)), two established techniques for measuring MPOD. Special emphasis was put on the agreement between measurements using the Macuscope and the two established devices, and on repeatability between Macuscope measurements within our population. Separate studies showed weak repeatability, however, Macuscope was never tested against other HFP devices or methods.21, 22

Materials and methods

We investigated the right eyes of 23 healthy subjects without ocular pathology, clear ocular media and a BCVA of at least 1.0. Mean age of the subjects was 34±15 years. To evaluate the ability of Macuscope to determine the MPOD, we compared the results with another commercially available device that uses psychophysical testing based on HFP, that is, QuantifEye.18, 23 The third method used to determine MPOD was a fundus reflectance technique, used in the MPR.24

MPOD was measured in healthy subjects without any ocular pathology, recruited from the University of Maastricht and the University Eye Clinic Maastricht. Study participants underwent all measurements consecutively on one day following the same protocol, that is, first measurement on the Macuscope, followed by the QuantifEye, followed by the Macuscope (repeatability measurement). The MPR method was the final measurement, because the high intensity of the used light could cause a temporal saturation of the photoreceptors that could influence the result of the other tests. The measurements were scheduled on one day between 0900 and 1500 hours, and were always performed by the same trained operator. When a subject was done with the first Macuscope test, another subject started his Macuscope measurement. Test series did not take longer than 30 min per subject to avoid loss of concentration due to fatigue. The test results were only taken into account after a subset of successful pre-test measurements, and were performed conform the manual. Repeatability measurements for QuantifEye were done on another day at the Academic Medical Center on 20 healthy subjects. Mean age of these subjects was 32±7 years. Repeatability data for MPR have already been published by van de Kraats et al,24 and are used in this study. All successful measurements were included for statistical analysis. All research adhered to the tenets of the Declaration of Helsinki. We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during this research.

Heterochromatic flicker photometry

Both the QuantifEye and Macuscope are based on the principle of HFP for measuring MPOD. Flicker is generated by alternating light of two different wavelengths, blue light, which is absorbed by MP, and green light, which is not absorbed by the MP. Because of the pre-receptorial location of the MP in the retina, incident light first passes through and is attenuated by the MP (peak absorption at λ ∼460 nm) before reaching the photoreceptors. MP has its peak concentration foveally or just parafoveally, and decreases rapidly with eccentricity. Therefore, these setups always use a central (peak MPOD) and peripheral (low or zero MPOD) measurement point, to subtract these values and obtain an individual MPOD value for each different subject. The peripheral measurement is hereby used as a reference point for the central measurement, to correct for the ‘baseline’ flicker sensitivity of each individual. Using HFP, subjects have to be instructed to fixate a stimulus and indicate when flicker is minimized (Macuscope) or when flicker is first observed (QuantifEye).

The MPOD value is derived by taking the logarithm of the ratio between blue and green luminance measured centrally and peripherally, as is displayed inequation (1):

Here Lbc and Lgc are the luminances of the blue and green light, respectively, during the central measurement and Lbp and Lgp the luminances of blue and green light, respectively, during the peripheral measurements.

MPOD values were determined after training of the subject. For the Macuscope, one or two pre-tests were completed depending on how fast the subject comprehended the course of action. For the QuantifEye, the pre-test function in the software was used for training of the subject.

Macuscope

Macuscope uses a conventional HFP approach, in which an operator adjusts a (green−) blue luminance ratio until no or minimal flicker is observed by the subject. Minimal flicker represents a matching of the brightness of both wavelengths. During a measurement, as also described by Hagen et al,21 firstly the subject has to fixate at a disc-shaped stimulus of 1.5 mm in order to measure foveally. A crosshair centred on the stimulus should facilitate central fixation. The presented stimulus alternates between the blue (λ ∼465 nm) and the green (λ ∼530 nm) light, at a fixed frequency of 20 Hz. After that, the subject has to fixate one of the crosshairs placed 8° on either side of the central stimulus. Flicker frequency here is changed to 30 Hz, and is also fixed. As the crosshair in the centre needs to be in focus, the operator is able to adjust for spherical corrections. The operator then adjusts the luminance of the blue light, and the subject has to verbally indicate when minimal flicker is observed at the centre of the blue disc. The luminescence ratio is then stored. For parafoveal testing of the right eye, the right crosshair is fixated. Before the testing, subjects received verbal instructions on how to perform the test and are given time to make themselves familiar with the task and the machine. When the point of minimal flicker could not be clearly indicated, foveal and parafoveal measurements were repeated. Subjects were reminded of the instructions of the task throughout the test.

QuantifEye

QuantifEye uses a method, in which the user indicates when flicker is first observed. The test consists of two stages. The overall sensitivity of the user to flicker is determined first, and the luminance contrast of the two lights (green and blue) is normalized for that particular subject. Subsequently, during the actual measurement, the subject starts by fixating the central stimulus. The frequency of the blue (λ ∼465 nm) and green (λ ∼530 nm) light of the stimulus is ramped down from above the critical flicker fusion frequency (CFF), for a series of different luminance ratios of the two lights. The subject views the stimulus and presses a button when flicker appears. Each luminance ratio, therefore, generates a certain temporal frequency at which flicker is detected, and directly creates a point on the graph in the software of QuantifEye, as measurements go on. During this sequence, the trained operator can immediately determine if the measurement has been performed well. Each individual will show a luminance ratio for which they are least perceptive, at the lowest detectable flicker frequency. This represents the minimum in the distinct V-shaped curve created and easily allows the operator to determine what the luminance ratio is at the minimum flicker frequency. The same sequence is then repeated for eccentric fixation (6° eccentricity). The difference between the minima of determined luminance ratios obtained from central and peripheral viewing determines the MPOD, equation (1). For a detailed description of this technique see van der Veen et al.18

MP reflectometer

The essentials of MPR24 are summarized as follows. The image of the filament of a 30-W halogen lamp is relayed to the pupil plane of the eye. The intensity of the light entering the eye is 1.04 × 107 Troland. A spot with a diameter of 1° centred on the fovea is illuminated, and the light that reflects from this spot is measured. An image of the retinal spot is focused on an optical fibre that has a mask on its tip to define a diameter spot of 1° at the retinal plane. The fibre is the receiving part of a spectrometer with a range of 400–800 nm and an optical resolution of 5.8 nm (FWHM). To keep instrument stray light to a minimum, the detection channel does not overlap with the illumination system. Chin rest and temple pads are used to help maintain head position. MPOD is determined by a full spectral analysis of the reflected light. In brief, the incoming light is assumed to reflect at the inner limiting membrane, at the infoldings/disks in cone/rod outer segments and at the sclera. Using known spectral characteristics of the different absorbers within the eye (lens, MP, melanin, blood), the densities of the pigments and percent reflectance at the interfaces are optimized to fit the measured data at all wavelengths.25, 26 For a detailed discussion of this analysis see Berendschot et al.27, 28

Statistics

The SPSS statistical software package (Version 15.0.1.1, Release 15.0.1; SPSS Inc., Chicago, IL, USA) was used for data analysis. To evaluate the repeatability, we generated a Bland–Altman plot, in which the difference between two measurements in one individual is plotted vs the average of these two measurements, for all individuals. The same analysis was used to evaluate the measurements between the two HFP methods. We also determined relative differences, that is, the differences of two values divided by their mean value. Pearson's correlation tests were used to quantify the linear association of determining MPOD between HFP methods and MPR measurement. MPR determines an MPOD averaged over a 1° field using the reflectance method, whereas with HFP the MPOD is assessed using psycho-physical testing. Consequently, as these methods use different analysis methods, a Bland–Altman graph is not suitable, and it is more appropriate to use Pearson's correlation tests.

Results

Mean MPOD and standard deviation obtained by the first Macuscope measurements was 0.38±0.16. The mean MPOD value and standard deviation determined with the QuantifEye was 0.39±0.17. The mean MPOD value and standard deviation obtained by the MPR was 0.58±0.17.

Figure 1 shows test-retest data for the Macuscope, QuantifEye and MPR. Low agreement was found between the test-retest measurements done with Macuscope. Using Bland–Altman graphs, we determined the average difference between two measurements with Macuscope to be −0.041 (Figure 1a). The limits of agreement, defined as the average difference plus or minus two times the standard deviation of the differences, were −0.041±0.32. Mean relative difference was 32.2%. In this figure two, data points were outside the limits of agreement. We found no reason, however, to exclude these two data points from our analysis.

Figure 1
figure 1

Repeatability of macular pigment optical density (MPOD) measurements displayed in Bland–Altman graph. The difference between the first measurement and the second measurement was plotted vs the average of both measurements. Mean of all differences and limits of agreement are displayed. (a) Macuscope. (b) QuantifEye. (c) MPR.

Full size image

Figure 1 also shows test-retest data for QuantifEye and MPR. The limits of agreement for QuantifEye were −0.02±0.18 (Figure 1b). The mean relative difference was 18.1%. The limits of agreement for MPR were −0.04±0.18 (Figure 1c).24 The mean relative difference of MPR was 11.3%.

Figure 2 shows the agreement between MPOD data obtained by the first Macuscope measurement and the QuantifEye measurement. MPOD data obtained by the first Macuscope measurement and the QuantifEye showed a poor agreement with limits of agreement of −0.017±0.44. Second Macuscope measurement, not shown in Figure 2, agreed also poorly with QuantifEye with limits of agreement of −0.004±0.39.

Figure 2
figure 2

Agreement of macular pigment optical density (MPOD) data obtained by QuantifEye and Macuscope, using Bland–Altman graphs.

Full size image

Figure 3 displays correlation coefficients between Macuscope and MPR measurements, and between QuantifEye and MPR measurements. For first Macuscope measurements and MPR measurements the correlation coefficient was r=0.05 (P=0.83), displayed in the upper panel. A significant correlation of r=0.87 (P<0.001) was found between the QuantifEye and the MPR, displayed in the lower panel.

Figure 3
figure 3

Correlation of MPOD data obtained by macular pigment reflectometer (MPR; horizontal axis) and Macuscope (vertical axis) in Figure 3a. Correlation of MPOD data obtained by MPR (horizontal axis) and QuantifEye (vertical axis) in Figure 3b. Linear fit and 95% confidence intervals are displayed.

Full size image

Discussion

Repeatability measurements performed with the Macuscope showed poor results. We found low agreement between test-retest measurements. Using Bland–Altman graphs, we determined an average difference between the two measurements with the Macuscope of −0.041. This indicates that there is negligible offset between the measurements, as expected. However, the calculated limits of agreement were −0.041±0.32. Concerning a device that determines values between 0 and 1, the limits of agreement are relatively wide. The Macuscope has been designed for middle-aged and elderly people, and uses a fixed flicker frequency designed for users of this age class. Other HFP techniques for MPOD determination enable adjustment of the flicker frequency for each subject.4, 19, 29 Customizing of flicker frequency for each subject can be crucial, because of inter-individual differences in flicker sensitivity. Without adjusting for individual flicker sensitivities, subjects could encounter either of two problems. When the set flicker frequency is too high for the subject, he will have a large null flicker range, or, when set flicker frequency is too low for the subject, he will be unable to stop the flicker in the target completely. It may be argued that, our sampling population imposes a limitation on the interpretation of the results, because it may not be representative in terms of age. Early studies have shown that CFF decreases with increasing age.30, 31 This means that the fixed flicker frequency of the Macuscope is probably too low for the subject when measuring in relatively young subjects. In this case, as explained, flickering will never disappear entirely. However, this does not influence the accuracy of the measurements since the subject had to indicate the point of minimal flickering. Also, It has been shown that MPOD does not alter with age.32, 33 Furthermore, devices described in above mentioned studies allowed subjects to adjust luminance themselves. In this, however, for the Macuscope, the minimum flicker point had to be verbally indicated to the operator, which could introduce a larger margin of error.

We believe another major issue in the current protocol of the Macuscope was Troxler's fading. Subjects had to fixate at the stimulus for a couple of minutes, which may have caused fading of the stimulus in some subjects.34 This hampers the determination of the minimal flicker frequency, which leads to repeated measurements. Even after repeated measurements, it remained difficult to determine the point of minimal flicker during a Macuscope measurement. Concentration problems are less likely to have caused variation in determination of MPOD, because a measurement with Macuscope does not take longer than a couple of minutes for each measurement. Recognizing the mentioned problems, we only used the MPOD value when the experienced operator, after carefully instructing the subject, considered that the subject fully understood the principle of the test.

The other HFP device (QuantifEye) used in this study has a higher repeatability (limits of agreement: −0.02±0.18). Repeatability of QuantifEye is determined on a comparable group of healthy subjects. A previous study showed that QuantifEye has a correlation coefficient between test and retest of r=0.97 (P<0.001) and a mean test-retest variability of 11.7%.18 QuantifEye uses a different technique to determine a point of minimal flicker sensitivity. It approaches the minimum flicker point from above the critical flicker threshold, with equiluminant stimuli for each part of the measurement. Subjects are therefore not exposed to flicker for more than ∼0.5 s when they press the button to indicate flicker has been detected, which will avoid Troxler's fading.

The fundus reflectance device (MPR) used in this study has also been tested for repeatability in an earlier study, and showed limits of agreement of −0.04±0.18 between test and retest measurements.24 In this study, test and retest showed a correlation coefficient of r=0.94 (P<0.001) and a mean within subjects variation of 7%. As MPR is a well-established technique to determine the MPOD value and is frequently used in the clinic, we used repeatability values from an earlier study.24

Mean MPOD values obtained by Macuscope (0.38±0.16) were similar to those obtained by the QuantifEye (0.39±0.17). These findings corroborate with other studies using HFP for MPOD determination, finding MPOD values of 0.21–0.49.35, 36, 37, 38 Although both Macuscope and QuantifEye did not differ in the measured mean value of MPOD, agreement between the two was poor (see Figure 2).

The Macuscope measurements did not correlate well with MPR measurements either, as shown in Figure 3a. In contrast, data obtained by the QuantifEye did correlate significantly with MPR results (r=0.87, P<0.001), see Figure 3b, as was also found in a previous study.18 In another study, the MPR and a different HFP device also showed a correlation coefficient of r=0.56 (P=0.012).24 Delori et al39 showed an inter-method correlation coefficient of r=0.61 (P<0.001) between a reflectance-based method and an HFP method in their study as well. The MPOD value with MPR is determined using a fundus reflectance method, in which known spectral characteristics of the different absorbers within the eye (lens, MP, melanin, blood), the densities of the pigments and percent reflectance at the interfaces are optimized to fit the measured data at all wavelengths. HFP, on the other hand, determines the MPOD value using psychophysical testing. These different analysis methods can introduce an offset between the determined MPOD values.

In line with previous studies, we observed an offset when comparing MPOD values obtained by HFP and fundus reflectance, as can be seen by the values shown in Figure 3b.18, 32, 39 This offset was probably due to subjects setting flicker frequencies using the edge of the stimulus.40, 41 Although absolute values differ, it does not influence the strength of the correlation. The inclusion of a correction factor for this phenomenon by Hammond et al16 did not affect their data. Therefore, current data sets were not corrected for this discrepancy.

Different prospective studies have shown manipulation of MPOD levels with respect to dietary and environmental factors, such as green leafy vegetables, lutein supplementation, and smoking.2, 4, 42 These MPOD augmentations could go up to almost 20%.16 Given our study results, the MPOD alterations are still within current mean relative difference, and will therefore not be picked up using the current Macuscope protocol. It may be noted that mean relative difference is discussed in this study and individual differences could have to be even larger to be picked up by the current device.

To summarize, this study showed low repeatability of the Macuscope. Comparison with an established HFP method, Quantifeye, and with a fundus reflectance technique, MPR, yielded low agreement and correlation. This leads to the conclusion that this device creates unreliable data and does not meet the requirements of high repeatability and reliability, needed for studying MPOD.