Introduction

Electroretinography (ERG) is a procedure that assesses the function of the retina and is used worldwide to aid ophthalmic diagnoses. It is a well-established technique with recognised standards published by the International Society for Clinical Electrophysiology of Vision (ISCEV) [1]. The gold standard ERG protocols usually require the use of corneal electrodes, pupil dilation and a significant time of dark (20 min) and light (10 min) adaptation. The patient is exposed to different full field flash intensities and the full range of testing which includes dark-adapted 0.01 cd s m−2, 3.0 cd s m−2, 3.0 cd s m−2 oscillatory potentials and light adapted 3.0 cd s m−2 and 3.0 cd s m−2 30 Hz flicker. The light-adapted ERGs predominantly reflect the cone system, while dark-adapted ERGs predominantly reflect the rod system. ERG waveforms comprise of ‘a’ and ‘b’ waves: The corneal electronegative ‘a’ wave represents photoreceptor function whereas the positive ‘b’ wave reflects inner retinal function.

Such methodology is understandably difficult to apply to the paediatric population due to compliance. To overcome this, some centres use sedation or anaesthesia to perform these studies. ISCEV [1] allows the modification of the protocol for paediatric patients; one such method developed by Tony Kriss at Great Ormond Street Children’s Hospital, London (GOSH), uses a hand held Grass photic stimulator [2, 3] and enables ERG recording without the need for sedation, mydriasis, extended dark adaptation or use of corneal electrodes. By using non-invasive skin electrodes, adjusting the recording environment and the strength and colour of the flash stimuli, it is possible to preferentially differentiate between rod and cone systems. Although this method produces lower amplitude ERGs [4], they still provide robust and repeatable diagnostic information. This paediatric protocol has been recognised internationally [1], is used routinely in several centres in the UK and, has formed the basis of numerous publications [3, 5].

The drawbacks of both conventional ISCEV ERG and photic stimulator paediatric techniques are the requirement of expensive electrodiagnostic equipment, significant time costs and a high level of training for the health professional completing the studies. This has resulted in these tests only being performed at specialist centres, which typically have long waiting times. The lack of accessibility is exacerbated by the equipment being laboratory based and so not easily transportable. Recent developments in technology (RETeval LKC technologies, Gaithersburg, MD, USA) have tried to overcome these limitations by developing a handheld ERG device with a step-by-step, user friendly interface. Furthermore, it eliminates mydriasis and is configured to be used with non-corneal electrodes that consist of sensor strips that adhere below the lower eyelids. This device can perform the full range of ISCEV procedures. The majority of published studies only utilised the ISCEV 3.0 cd s m−2 30 Hz flicker, which primarily assesses the cone-mediated system. Originally used as a screening tool in diabetic retinopathy [6,7,8], flicker ERGs have been reported to be promising in the assessment of general retinal function in adults and children [9,10,11,12,13,14]. However, these studies confined the RETeval recording to flicker ERGs thus only providing information about cone function and not the rod-mediated system or the mixed cone/rod waveforms. This device has been proposed to be child friendly [15], although reports in the literature are mixed with regards to compliance in children [14, 16].

This paper presents the experience from the Visual Electrodiagnostic Department at Southampton General Hospital in adapting the paediatric protocol photic stimulator (Grass PS33 plus) which is routinely used in the paediatric clinic, to the handheld RETeval device. The study presents data collected using both recordings systems from healthy adult volunteers and from paediatric patients aged 4 months to 14 years with a range of clinical conditions. The RETeval is evaluated and the efficacy of its use in an everyday paediatric ophthalmic clinic is discussed.

Method

Ethics

The data collected in the study were approved by the University Hospital Southampton Foundation Trust Governance Committee after advice was sought from Research and Development.

Participants

Forty healthy subjects (80 eyes) (age 19–75 years; mean 39 years) volunteered for this study. They were all staff from the Eye Unit from whom fully informed consent was obtained. All reported to have no known ophthalmic conditions or symptoms with the exception of refractive error. This sample size, although relatively small, is statistically sufficient (>29 needed to detect at least moderate agreement between devices (α—0.05, β—0.2 and r = 0.5) [17].

Thirty-seven paediatric patients were also recruited (aged 4 months–14 Years; mean: 5 years). All patients were attending the Visual Electrodiagnostic Department as part of their routine ophthalmic appointment.

ERG recordings

ERGs were recorded with no formal dark or light adaptation using the standard and RETeval systems for each participant. The order was randomised to limit the effects of confounders. ERGs were performed in the order described in Table 1. Paediatric ERGs were carried out by specialists in visual electrodiagnostics whereas those in healthy adult volunteers were performed by a health care assistant following training.

Table 1 Stimulation settings for the RETeval and Grass photic stimulator protocols.
Full size table

Standard system

The standard paediatric protocol for ERG recording was carried out using disposable, adhesive skin electrodes, placed centrally below the pupil, close to the lower eyelid margin (within 1 cm) and, referred to a common mid-frontal electrode (Fz 10−20 system) [5]. An Espion 300 desktop system (Diagnosys llc, Cambridge, UK) was used. This set up enables the simultaneous recording of the right and left eye. A 13 cm diameter Grass photic stimulator, with manufacturer’s supplied diffuser, Model PS33 Plus, (Grass Instruments, Quincy, MA) was used to evoke rod-mediated ERGs (Grass strength setting 1: white and with blue filter), and mixed cone/rod ERGs (Grass strength setting 4) in darkened laboratory conditions (room lights off). Cone-mediated ERGs were elicited under photopic conditions (Grass strength 4) and to 30 Hz flicker stimulus (room lights on). The photic stimulator is held around 15 cm in front of the face for all recording conditions with the exception of the rod ERG recording with dim white where the lamp is held further away at around 50 cm. The filter settings used are 0.625–300 Hz and comply with international standards [1]. Flash ERGs for all parameters were averaged until a clear ERG was obtained (5–10 averages).

RETeval system (firmware 2.9.3)

ERGs were recorded using the sensor strips provided with the RETeval system. These strips incorporate the active, reference and ground electrodes in a single adhesive strip measuring ~7 cm in length. Sensor strips were positioned under the lower eyelid margin, with care taken in placing the end of the strip under the centre of the eye. It is recommended by the suppliers that the eyecup of the device is pressed around the patient’s eye, minimising any gaps between the face and the eyecup, however in practice we found that contact between the eyecup and electrode strip caused electrical interference. To overcome this, the eyecup was held between thumb and forefinger which then rests around the eye. Instructions state that ‘patient should not talk, smile or grimace’. Each eye was recorded (randomly left or right followed by the fellow eye) using a bespoke protocol under varying environmental lighting conditions (scotopic and photopic). This protocol was developed using ISCEV flash intensities to reflect the optimal settings for recording rod- and cone-mediated ERGs. Formal dark and light adaptation was not performed and additional testing with a blue flash (0.01 cd s m−2) under scotopic conditions was incorporated to correspond to testing performed using the standard system. The number of averages was adjusted to 5, 10 or 15 depending on the compliance of the subject. The RETeval is supplied with a standard filter setting of 1–100 Hz with smoothing. The standard protocol provides trolands correction, which adjusts stimulus light strength during testing depending on pupil size, measured via the inbuilt pupilometer [18]. There is a fixed candela option to be used with subjects who have unsteady fixation or erratic eye movements. As this protocol is bespoke, the inbuilt reference ranges could not be used. The ERGs in this study were not performed under strict ISCEV standards, conventional descriptors will be replaced with more descriptive terminology; scotopic (dim white, dim blue, bright white) and photopic (bright white, 30 Hz flicker).

Analysis

Establishing reference data

Left and right eye ERGs were recorded from the healthy adult population using both systems across all paradigms. The peak times and amplitudes of each wave were manually measured and cross-checked by two separate specialists in electrodiagnostics. The left and right eye measurements were averaged (40 data points) and presented as descriptive statistics; mean, 2.5th and 97.5th centile and as box plots.

Agreement of the RETeval and standard system in the healthy adult group

The averaged ERG amplitudes and peak times for each paradigm were assessed for agreement between the RETeval (trolands) and standard system using the inter-class coefficient (absolute agreement) and Bland-Altman plots.

Agreement in detecting normal and abnormal results in the paediatric patient group

Each paediatric patient that underwent ERG testing with the RETeval (fixed candela) and standard system, had their results determined by two specialists in electrodiagnostics, as normal or abnormal based on the overall ERG picture and adult reference range. These findings were then cross-checked. Cohen’s Kappa was used to determine the agreement between the devices in detecting normal and abnormal results.

Participant and user survey

A number of validity measures were performed during this study, including; participants views, compliance, timing of test and user feedback. The views of the healthy adult subjects were obtained by asking “what is their preferred method: RETeval or standard?” and the reason “why?”. The time was measured from start of the test to the end for each participant across each device and will be presented as a range. Compliance was measured by recording if the test was completed and will be presented as a percentage. User feedback is summarised in the discussion.

Results

Establishing reference data

The adult reference range is presented in Fig. 1; this can only be used for children over the age of 1 year as in younger infants the amplitudes are not comparable to those of adult ERGs [3]. The descriptive statistics showed consistently smaller amplitudes obtained with the RETeval compared to the standard system, with the exception of flicker ERG amplitudes. The enhanced flicker amplitudes with the RETeval may be due to the reconstructed response from a frequency domain extraction. The variation in the data was less or similar for the RETeval system across different stimulation conditions when compared to the standard system. The peak times showed less variation but still demonstrated a difference between the systems. It is important to note that the individual and mean peak times obtained by both devices were within the ranges expected for the rod- and cone-mediated systems being tested [2].

Fig. 1: Box plots and reference ranges using the RETeval and standard system.
figure 1

The boxplots demonstrate the larger variation in amplitude on the standard system compared to the RETeval, reflecting the need for individual reference ranges. The peak times show less variation with values that reflect the rod/cone mediated system being tested.

Full size image

Agreement of the RETeval and standard system in the healthy adult group

The amplitude and peak time data of each wave across each paradigm met the assumptions for the statistical tests used (inter-class coefficient and Bland–Altman plots), showing normal distribution (Shapiro-Wilk, Q–Q charts) and similar variances (F-test) as raw data, or after a log or square root transformation. In relation to Bland–Altman, the mean difference group also showed normal distribution.

The inter-class coefficient for the amplitude and peak time of each wave demonstrated lack of absolute agreement between the two techniques (<0.5) across all paradigms. This was replicated on Bland–Altman plots (see Fig. 2), with the one-way t-test showing significant differences between the two systems across all measurements. The plots themselves showed large limits of agreement (the exception being the photopic ‘a’ wave peak time), which if used interchangeably would result in a normal value on one device being abnormal on another. Bias was also present across all paradigms, with greater differences between the systems from small to large amplitude and peak time values (with the exception of the photopic ‘a’ wave peak time).

Fig. 2: Bland–Altman plot of the photic ‘b’ wave amplitude difference between the two systems.
figure 2

Demonstrating a 4 μV bias between the devices and a limit of agreement of 37 μV, which is too large for clinical use as it is larger than 90% of the amplitude recorded in adult healthy population.

Full size image

Agreement in detecting normal and abnormal results in paediatric patient group

34 patients had their ERGs recorded on both devices and had a range of pathologies, some of which included post-retinal conditions with normal ERGs as summarised in Table 2.

Table 2 Table summarising clinical classification of the paediatric patient group.
Full size table

The diagnostic agreement of the RETeval with the standard system was tested using Cohen’s Kappa (one patient was excluded as they were under the age of one year). This analysis revealed a high level of agreement on the capability of detecting normal and abnormal ERGs between both devices (k = 0.80), with a relative sensitivity and specificity of the RETeval system of 1.0 and 0.91, respectively. This specificity is reduced due to five false positives recorded on the RETeval system; these false positives were patients referred for post-retinal dysfunction and were confirmed clinically as not having a retinal disorder. The RETeval was able to identify all participants who were recorded as abnormal on the standard system. Figure 3 illustrates some representative ERG waveforms from participants with normal ERG as well as from those with different retinal pathologies.

Fig. 3: Examples of ERGs from multiple conditions using the RETeval (Blue) and Grass (Red) stimulators.
figure 3

The normal waveforms demonstrate the variation in amplitude between the two recording methods, although peak times remain similar. There is reproducibility of the pathognomonic ERG findings in the range of conditions including the electro-negative configuration of the mixed response in CSNB.

Full size image

Participant and user survey

All healthy participants responded to the user survey: 30% preferred the RETeval, 67% the standard system and, 3% were indifferent. The reasons for this are considered in the discussion section.

The time taken for each test was 5–15 min for the RETeval and 3–10 min for the standard system Of the 37 patients, three (aged under 2 years) failed to complete recordings using the RETeval due to poor compliance, although they were able to complete testing using the standard system.

Discussion

Results from the patient group using the RETeval system showed full agreement in detecting abnormal ERGs with those obtained using the standard system. Isolated cone and rod dysfunction, including alterations in waveform configurations that result in negative ERGs (as found in retinoschisis and CSNB), were clearly demonstrated using the RETeval system and were identical to those from our standard system (Fig. 3). This was obtained by both recording methods without mydriasis or prolonged dark/light adaptation. The suitability of this device for use in children, using the modified protocol, shows promise with results demonstrating over 90% success rate; this is in contrast to other studies which demonstrated only 51% [15] and 31% [13] success rates. Our results are consistent with the in-depth studies by Liu et al., which compared standard full-field ISCEV ERGs to the RETeval system in adults and children and concluded that there was a strong agreement and accuracy between the two systems [16]. Similarly, the data from this study showed very strong clinical agreement between the RETeval system, but in this case, with the standard paediatric set up. The high accuracy in the Liu et al. study was also repeatable in this study with respect to relative sensitivity (1.0) and specificity (0.91). The poor inter-class correlation coefficient and significant differences seen on the Bland–Altman plots demonstrated a lack of absolute agreement between the devices. Thus, the devices were not interchangeable and individual reference ranges need to be used for the RETeval device. It is important to stress that this lack of agreement was between the protocols used on each device and not the device itself; each protocol used different stimulus parameters which in itself could account for the lack of agreement. It is also worth noting that each device delivers a different type of stimulus with the RETeval delivering a full field flash and the standard system delivering a focal stimulus which may explain the reduced variability in the responses from the RETeval.

Interestingly, only 30% of the healthy adult volunteers preferred the RETeval system over the standard system. Those that did not, mentioned concerns regarding claustrophobia and proximity of the device to the eye. Furthermore, a substantial number of these subjects (67%) preferred the standard system as testing was quicker with both eyes being recorded simultaneously. Obtaining results efficiently and rapidly is a fundamental necessity when dealing with the young paediatric patient.

The fixation spot in the RETeval system was a positive feature and participants stated that this made the test easier. Some of the paediatric patients also responded positively to having a fixation target. The standard system does not have a fixation target, but the patient is encouraged to look at the bulb. A very important aspect of the standard system is that it is usually hand held and so enables the examiner to follow the subject’s face if they move. This is more difficult with the RETeval as the eyecup needs to be placed over the eye with recommended minimum gap between it and the face. Moreover, the pupil needs to be centred within a target circle by the operator. The advantage of a non-contact large photic stimulator is the freedom for the operator to physically move the stimulator as the subject moves around: an especially useful asset when testing the wriggly toddler, the bottle- or breast-feeding baby as well as the crying/sleeping infant or the developmentally/ neurologically impaired patient that may have poor fixation, erratic eye movements or nystagmus. In the case of recording the crying or sleeping patient, our experience shows that skin electrode placement can be modified to account for eye position and the strength of the stimulus can be adjusted whilst recording. In such cases, it may not be possible to get truly isolated cone and rod ERGs, but a bright flash mixed cone/rod ERG can be achieved and thus provide a useful screening and baseline results.

A particular problem which became apparent when we tested young patients and infants, was the size of the sensor strip that is supplied with the RETeval system. The relatively long strip meant the electrode often could not be placed optimally in relation to the pupil, or it would lose adhesion as the electrode placement reached the hair line on smaller faces. The manufacturer is currently developing a paediatric size sensor and there is always the option to use individual electrodes with the device, however this would require additional user training.

Observational data using the RETeval system, noted high variability in the rod-mediated, dim white stimulation paradigm, with multiple repeats required to obtain reasonable ERGs. This could be due to limited dark adaptation. It is preferable that the patient is kept in a mesopic environment prior to testing and if suboptimal results are obtained, then this adaptation time should be extended, as recommended by the paediatric approved protocol [2, 3, 5]. Exact and different timings of dark adaptation could be an area of further research to address the optimal recording conditions for paediatric patients. In addition, erratic and random eye movements in paediatric patients caused difficulty in pupil monitoring, hence the trolands programme needed to be replaced with the fixed candela protocol (values documented in Table 1). Photopic ERGs were also difficult to obtain as the flash stimulation starts before data acquisition resulting in some subjects having problems maintaining fixation.

The RETeval system has an intuitive step-by-step user interface and has the capability of direct feedback regarding factory aged matched or bespoke reference ranges. The company currently has inbuilt reference ranges for adults, but there is an option to have local reference data applied via custom protocols. However, this linear software interface does not allow the user to review the results or repeat any previous recordings once one has progressed to the next step. A bidirectional interface rectifying this could be a consideration in future software development. Finally, variability was seen due to electrode placement and adhesion, which can be a result of user error, which has been described in previous work [19].

Table 3 summarises the main advantages and disadvantages of both recording systems according to the data and experience from this study. Further research would be needed to produce age-matched controls and to evaluate its sensitivity and specificity as a tool in retinal disease.

Table 3 Table presenting the advantages and disadvantages of the two devices in the context of the ideal characteristics for a paediatric ERG recording system.
Full size table

The RETeval system produced five false positives across the normal and patient subjects, which the standard system demonstrated as being normal. However, the RETeval device did not produce any false negatives. This highlights the need for further testing of abnormal patients.

This study demonstrates the possible use of the RETeval system as a point-of-care triaging tool by healthcare professionals not specialised in electrodiagnostics. There are several patient groups that would benefit from this type of testing to identify normal or abnormal retinal function prior to a full visual electrodiagnostic referral. These include young patients with unexplained myopia, nystagmus and abnormal eye movements, possible genetic conditions such as Usher’s (especially when referred purely due to sensorineural hearing loss), Bardet-Biedl, Batten’s and Cohen’s disease. It would also be of use prior to referral for the gross evaluation of the integrity of the retina when it cannot be viewed due to corneal opacities or cataracts. Under specialist hands, the RETeval could be interchangeable with ERGs recorded with standard systems and in our experience has proved useful in assessing patients on the ward who are unable to attend clinic. Recording ERGs with the RETeval is relatively easy. It is envisaged that its use in clinic by the clinician, orthoptist or optometrist would help provide fast diagnostic information and highlight patients that need further investigations. It should be stressed that the RETeval cannot replace full visual electrodiagnostic assessment where other tests such as visual evoked potentials, pattern ERGs and electro-oculograms may be necessary to build a clinical picture.

Conclusions

This study has shown that the RETeval system elicits ERGs that are similar to those obtained with our standard system in young paediatric patients. The devices did not reveal an absolute agreement and highlights the need for individual reference data. Comparing the two techniques, which both provide fast, immediate ERG results without mydriasis, there was a high level of agreement in detecting abnormal and normal results (Cohen’s Kappa k = 0.80). The relative sensitivity and specificity of the RETeval was 1.0 and 0.91, respectively. These findings indicate that the RETeval is a promising device for assessing retinal conditions, although there are certain technical issues that need to be addressed to enhance its use in paediatric patients. Recording cone- and rod-mediated ERGs in young children needs to be adaptable to ensure that the maximum amount of data is obtained in the shortest time.

To the best of our knowledge, this is the first study comparing the RETeval system with ERGs obtained using a photic stimulator and the first to assess the efficacy of its use in a paediatric clinical setting. It is envisaged that the RETeval device will become a valuable tool in the paediatric ophthalmology clinic, and may result in savings for clinical services in terms of time, cost, staffing and training.

Summary

What was known before

  • The RETeval has been used to record ERGs on older children and adults and has compared the device to results obtained using ISCEV standard recording methodologies.

  • Previous studies have been confined mostly to the use of cone mediated flicker stimulation to assess retinal function.

  • Previous studies on children have reported variable levels of compliance.

What this study adds

  • This is the first study to apply standard paediatric ERG parameters, currently used in specialist centres to the RETeval portable device.

  • It compares the two methodologies in regards to repeatability, diagnostic capability, ease of use and patient experience. The RETeval with the modified protocol shows high levels of agreement with current paediatric methods that do not require sedation or mydriasis.

  • The advantages and disadvantages of the systems are discussed. The study demonstrates that the handheld RETeval device with this modified protocol has the potential as a triaging tool for retinal conditions, that can be used in the paediatric ophthalmology clinic.