Reduced False Positives and Improved Reporting of Loop-Mediated Isothermal Amplification using Quenched Fluorescent Primers

Abstract

Loop-mediated isothermal amplification (LAMP) is increasingly used in molecular diagnostics as an alternative to PCR based methods. There are numerous reported techniques to detect the LAMP amplification including turbidity, bioluminescence and intercalating fluorescent dyes. In this report we show that quenched fluorescent labels on various LAMP primers can be used to quantify and detect target DNA molecules down to single copy numbers. By selecting different fluorophores, this method can be simply multiplexed. Moreover this highly specific LAMP detection technique can reduce the incidence of false positives originating from mispriming events. Attribution of these events to particular primers will help inform and improve LAMP primer design.

Introduction

In molecular diagnostics, isothermal nucleic acid amplification methods are an attractive alternative to benchmark polymerase chain reaction (PCR)1 strategies because of the low cost equipment required to initiate and maintain the reaction. This is of particular relevance to low resource settings and point-of-care applications. Numerous methods have been described2,3,4 for isothermal amplification of which loop-mediated amplification (LAMP)5 is the most widely reported. LAMP based assays have been used for numerous applications including the detection of pathogens such as malaria6 and salmonella7, viral RNA rapid detection for HIV with reverse transcriptase LAMP (RT-LAMP)8, GM crop contamination9 and in forensic science to specifically detect human DNA10.

The LAMP reaction is initiated by strand invasion of the DNA template by hairpin-forming LAMP primers which anneal and extend catalysed by a strand displacing DNA polymerase. These annealed LAMP primers are in turn displaced by displacement primers in the initiation of amplification, and lead to the formation of a dumbbell-like structure. This structure forms the basis of cycle amplification and elongation, with only LAMP primers, into cauliflower-like formations of single stranded DNA loops. The LAMP primers FIP and BIP (forward and backward inner primer), are designed to hybridise to the complementary and reverse complementary target sequences of F2/B2 and F1/B1 (Fig. 1). Displacement primers are referred to as F3 and B3. The addition to the reaction of Loop primers11 and STEM primers12 accelerates the DNA amplification by hybridising and extending from the hairpin loops or the region between loops respectively. Faster reaction times have been shown with the addition of swarm primers13. LAMP amplification is typically achieved at 60 to 65 degrees C for a time period dependent on the concentration of the template. Highly desirable characteristics of LAMP includes high sensitivity and specificity with rapid reaction times and there is also evidence that LAMP amplification will proceed in the presence of PCR inhibitors9,14,15 permitting less stringent DNA extraction procedures.

Figure 1
figure1

LAMP amplification initiation, cycling and elongation. Strand invasion of double stranded DNA with LAMP and displacement primers initiates the LAMP reaction. Positions of primer recognition sites are shown in red for displacement primers, green for LAMP primers and blue for Loop primers. LAMP initiation leads to the formation of dumbbell-like structures with cycling between the two forms and elongation into cauliflower-like structures with multiple loops. The fluorophore is shown unquenched attached to the FIP primer and quenched by proximity to guanine bases due to fluorescence resonance energy transfer (FRET).

PCR has been developed to increase the specificity of the two primer amplification method. Real time PCR16 developments have included amplicon specific TaqMan probes17 which are quenched when unbound, but the fluorophore is unquenched by release from the probe because of the 5′ to 3′ exonuclease activity of Taq polymerase. Molecular beacons18 are self-quenching oligonucleotides with a template specific loop structure which when bound to the target sequence unquenches the fluorophore, unbound molecular beacons remain quenched. The increased specificity of these example methods, combats the false positive results that can affect indirect methods of amplification detection such as intercalating dyes and gel electrophoresis.

Although LAMP isothermal amplification with Loop acceleration utilises six primers targeting eight target DNA sequences, the incidence of false positive results has been reported19. LAMP assays may use indirect methods of amplification detection and these, whether real-time or end-point, rely on careful primer design, optimal reaction conditions and robustness testing to negate false positives. Such indirect detection strategies include the addition of intercalating dyes for fluorescence or colorimetric determination, turbidity20 from pyrophosphate precipitation, observations of LAMP ladder patterns with agarose gel electrophoresis5, quenched calcein21, hydroxy-naphthol blue22 as well as a number of fluorescent probe based techniques15,23,24. Real-time methods include bioluminescent reporting (BART)25 whereby the increase in inorganic pyrophosphate during nucleic acid amplification is converted to ATP which is converted into detectable light using a thermostable luciferase and luciferin substrate. The incorporation of intercalating dyes such as SYBR Green26,27 and the SYTO family28,29 generate increased fluorescence with increasing double-stranded DNA propagation. BART and SYTO9 real time LAMP amplification detection are used in this report. Other real time methods include turbidimetry30, colour change with pH sensitive dyes31 and amplifying with the nucleic acid specific dye berberine32.

To improve the specificity of LAMP, a number of solutions involving fluorescent labelled probes and primers have been proposed. Fluorescent probes used for quantitative PCR such as for example molecular beacons33, have been adapted to LAMP, however the displacement activity of the polymerase and lack of thermal cycling make such adaptations problematic. Fluorescently labelled oligonucleotide probes targeting the LAMP loop structures34 offer increased specificity when visualised after precipitation with polyethylenimine (PEI) which forms an insoluble fluorescent complex. Proximity to guanine nucleotides had been previously shown to quench certain fluorophores35 and forms the basis of a number of specific LAMP detection methods. Alternately binding quenching probe competitive LAMP (ABC-LAMP)15 is another oligonucleotide probe method for increased specificity, whereby the AB-Q probe with a 5′ fluorophore hybridises to the target loop and specially designed competitor DNA which has a number of guanines replaced with cytosines in proximity to the fluorophore. Real time LAMP amplification will therefore show a reduction in fluorescence in the presence of specific loop sequences. Fluorescent labelling of the loopF primers for fluorescence quenching during LAMP amplification by proximity to guanine36,37 enabled different target LAMP amplifications to be detected. Further multiplex detections were shown with melt LAMP (mLAMP)38,39 designed to use fluorescent labelled FIP primers for end point detection following real time turbidimetry of multiple targets.

Fluorescence resonance energy transfer (FRET) methods such as duplex LAMP23 utilise an acceptor intercalating dye and 5′ end FAM labelled BIP primer for one gene target sequence and no fluorophore for another. Increase or decrease in fluorescence is indicative of the gene present. A FRET LAMP assay19 utilises a combination of donor probe, acceptor probe and a hybridisation probe to detect white spot syndrome virus by fluorescence. Another probe method for increased specificity is the detection of amplification by release of quenching technique (DARQ)40 whereby a quencher linked to the 5′ end of a FIP primer quenches a hybridised fluorescent probe which is released during amplification. Assimilating probe24 employs a similar methodology. One step strand displacement LAMP amplification (LAMP-OSD)41 utilises a quenched reporter consisting of Reporter F and Reporter Q probes. Reporter F has an additional 11 nucleotide toehold sequence and the probe hybridises to a specific target loop of the amplification. The probe is unquenched and fluorescence can be detected. Graphene Oxide based FRET LAMP42 is another approach that uses a fluorescent probe, but with graphene oxide quenching the DNA fluorophore complex. Quenching of unincorporated amplification signal reporters (QUASR)43 offers an alternative approach for increased specificity using temperature to determine the presence of target. Labelling FIP and BIP primers with various fluorophores and dot-ELISA44 was used in a further specific multiplex LAMP assay and recently fluorescence of Loop primer upon self dequenching (FLOS LAMP)45 used modified thymine residues towards the 3′ end of primers with linked fluorophores so that during LAMP amplification the probes were unquenched.

In our work we have combined the 5′ fluorescent labelling of LAMP primers with quenching by LAMP amplification to provide a simple and specific method of DNA quantification and detection. The LAMP primers are ideally suited to this purpose because they are both essential and lead to the formation of complex looped structures. The attachment of a fluorophore does not appear to compromise LAMP target sensitivity and detection times remain rapid when compared to qPCR. We further show how the labelling of different primers, combined with non-specific detection of amplification with SYTO9 can be used to inform the primer optimisation process in designing LAMP assays. Furthermore we show that multiplex assays are possible with this method.

Results

The ‘cauliflower-like’ structures which are rapidly generated in LAMP amplification potentially bring into proximity the 5′ end of a Loop primer with the 5′ end of a LAMP primer (Fig. 1). We investigated using a FAM labelled LoopF and a JOE labelled FIP to explore whether fluorescence resonance energy transfer (FRET) could be observed between these two fluorophores by excitation of FAM transferring energy to JOE for increased fluorescence (Fig. 2). However we observed a decrease in fluorescence with time from JOE and upon further investigation it was clear that the fluorescence of both of these fluorophores was quenched during the LAMP amplification. Furthermore the time at which quenching occurred was proportional to the original concentration of the DNA template, suggesting it was linked to the increasing amount of LAMP amplicon.

Figure 2
figure2

Quenching of fluorescence. LAMP amplification with 35S promoter primers substituted with a FIP primer labelled with JOE and LoopF primer labelled with FAM. Dilution series of a linear plasmid template and with additional no template control. Quenching of the FAM signal from the FAM-LoopF primer (A) and quenching of the JOE signal from the JOE-FIP primer (B). Four replicates for each dilution of template and for NTCs. Average Ct (cycle threshold set at −0.05 or −0.07) against logarithmic scale of the template dilutions for JOE-FIP (C) and FAM-LoopF assays (D). Three out of four samples were positive at 50 copies, one out of four samples were positive at 5 copies. The R2 values with the highest concentration of template removed were 0.9868 and 0.9795 for JOE-FIP and FAM-LoopF respectively.

The results in Fig. 2 show that the labeling of either the FIP primer or the LoopF primer produced similar results in terms of degree of signal quenching and the timing of the quenching. The non-template controls remained constant throughout the assay time of 120 minutes. All four replicates of samples with 500 copies or more of linear plasmid template were detected, but only 75 percent of the samples with 50 copies and 25 percent of those with 5 copies. However the positive results at these low concentrations of template were closely aligned with quantitative semi-logarithmic trend lines (Fig. 2C,D).

The concentration of primers and the ratio of primer combinations which contain the fluorescently labelled FIP were then optimised and shown in Fig. 3. Best results were obtained with 0.4 to 0.8 micromolar JOE-FIP (Fig. 3A,E).

Figure 3
figure3

LAMP primer combinations with JOE labelled FIP. LAMP amplification and fluorescent detection of 50000 copies per 5 microlitres linear plasmid DNA using 35Sp primers; (A) 0.8 micromolar JOE-FIP with 1x other primers (0.8 micromolar BIP, 0.4 micromolar LoopF and LoopB, 0.2 micromolar F3 and B3), (B) 0.4 micromolar JOE-FIP, 0.4 micromolar FIP, 1x other primers, (C) 0.2 micromolar JOE-FIP, 0.6 micromolar FIP, 1x other primers, (D) no fluorescently labelled primers, 1x primers, (E) 0.4 micromolar JOE-FIP, 1x other primers, (F) 1.2 micromolar JOE-FIP, 1x other primers, (G) 1.6 micromolar JOE-FIP, 1x other primers, (H) 1.6 micromolar JOE-FIP, 1.6 micromolar BIP, 1x other primers, (I) 1.6 micromolar JOE-FIP, 2x other primers. Red lines: no template control; blue lines: samples with 50000 copies of target.

The clearest separation between the detection of amplified template and the negative template controls (NTC) coupled with low variation between the replicates was observed with 0.8 micromolar JOE-FIP and the standard concentration of the other primers (Fig. 3A). The concentration of 0.4 micromolar JOE-FIP with standard concentration of the other primers (Fig. 3E) was better separated with less replicate variation than 0.4 micromolar JOE-FIP with 0.4 micromolar FIP (Fig. 3B). However 0.2 micromolar JOE-FIP with 0.6 micromolar FIP (Fig. 3C) was also better than 0.4 micromolar JOE-FIP and FIP. The higher concentration of JOE-FIP of 1.6 micromolar (Fig. 3G,H,J) all showed reduced initial quenching when compared to the lower concentration of JOE-FIP with a further reduction in quenching towards the end of the assay.

Using the optimised ratio and concentration of primers in Fig. 3A the quenching LAMP method was compared to LAMP detection with the intercalating dye SYTO9. The SYTO9 was used in the presence or absence of JOE-FIP primer to gauge the impact of the labelled primer on the assay kinetics. In Fig. 4,1, the assay with the JOE-FIP primer in the presence of SYTO9 was acquired in the yellow channel to detect the fluorescence signal from JOE, and the assay with 50000 template copies had an average Ct of 11.2 (SD 0.6) and for 50 copies an average Ct of 18.6 (SD 0.9), at a defined cycle threshold of −0.03. In Fig. 4,2, the JOE-FIP primer with SYTO9 assay was acquired via the green channel to detect the fluorescent signal from SYTO9. The 50000 template copies assay had an average Ct of 11.7 (SD 0.4) and for 50 copies this figure was 17.2 (SD 1.4) at a cycle threshold of 0.4. The replicate assay with unlabelled FIP and SYTO dye had more rapid Ct values of 8.1 (SD 0.1) for the 50000 copies and 12.7 (SD 0.5) for the 50 copies at the cycle threshold of 0.4 (Fig. 4,3).

Figure 4
figure4

Detection with JOE-FIP and SYTO9. LAMP amplification with 35S promoter primers of a linearised plasmid template (1) quenched fluorescence in presence of JOE-FIP primer with SYTO9 using yellow channel detection of JOE, (2) fluorescence of same samples with green channel detection of SYTO9, (3) fluorescence with unmodified primers with SYTO9 only (green channel). (A) Normalised fluorescence truncated from 60 minute assay, (B) melt curve analysis, (C) average Ct plotted against template concentration on a log scale and (D) summary of replicate Ct values, average Ct and standard deviation (SD) for each dilution. NTCs (red lines) were negative. Light blue lines: 500000 copies; orange: 50000 copies; dark blue: 5000 copies; brown: 500 copies; green: 50 copies.

The linearity for the dilution range to a semi-log model with each method was high (R2 0.9799 to 0.9912), however it is apparent that the variation between replicates increases with decreasing concentration as previously observed46. The variability between replicates at each dilution for the SYTO9 detection method was lower than that of the quenched fluorophore technique and increased variation and slower detection times were shown with SYTO9 detection using the primer combination that included the fluorophore labelled FIP. Melt curve analysis showed almost identical profiles for the two assays with SYTO9 detection and the melt curve using quenched fluorophore detection clearly separates the positive from the NTC replicates.

The quenched fluorophore method using labelled LoopF or FIP primers could provide an enhanced specificity when compared to detection methods that rely solely on increases in double stranded DNA or pyrophosphate production, because only the products of LAMP amplification will be detected and not ‘false positives’ from non-specific primer interactions. To investigate this possibility LAMP primers with minimal post-synthesis purification were subjected to extended assay times of 200 minutes to encourage amplification from non-specific primer interactions (Fig. 5,2A–C).

Figure 5
figure5

Detection with SYTO9 and JOE-FIP positive and negative samples. (1) LAMP amplification of 500000 copies linear plasmid template and NTCs with detection by both (1A) and JOE-FIP quenched fluorescence (1B) in separate tubes. Four replicates of template in blue and NTCs in red. Melt curve analysis (1C,D) showing amplification of template only in the 90 minute assay. (2) Amplification with LAMP primers of 72 replicate NTCs for 200 minutes. (2A) SYTO9 detection of amplification between 150 and 200 minutes showing 5 of 72 giving a positive signal. (2B) JOE-FIP detection of the same samples using the yellow channel (excitation at 530 +/− 5 nm with emission at 555 nm +/− 5 nm). (2C) FAM-LoopF detection using the green channel (excitation at 470 nm +/− 10 nm with emission at 510 nm +/− 5 nm). (2D–F) Melt curve analyses.

Using SYTO9 for detection, five of the seventy two NTC replicates gave a positive signal between 150 and 200 minutes (Fig. 5,2A). The melt curve analysis (Fig. 5,2D) showed a higher Tm than for the true LAMP products formed from template with the same primers (Fig. 5,1C) with SYTO9 detection. The LAMP assay with JOE labelled FIP supplemented for the basic purification FIP (Fig. 5,2B) showed no positive melt curve results from the same seventy two NTCs over the 200 minute assay time, when compared with the corresponding LAMP assay with template and NTCs (Fig. 5,1B). The supplementing of LoopF with FAM labelled LoopF showed one of the seventy two NTCs as a positive (Fig. 5,2C). We conclude that combining SYTO9 detection with quenched fluorophore detection provides two methods of detection in one tube at the same time increasing overall specificity.

We expanded this concept to see if we could observe simultaneous non-specific detection with SYTO9 and specific detection with the quenched fluorophore attached to the FIP LAMP primer. Using JOE to label the FIP primer, fluorescence acquisition can be achieved with excitation and emission of SYTO9 at 470 and 510 nanometers separated from JOE excitation and emission at 520 and 555 nanometers. Assay times were extended to 180 minutes to facilitate non-specific primer interactions.

The detection of the fluorescence from the JOE labelled FIP primer in the LAMP amplification showed three positive results from the seventy two NTCs. These positives were detected after 100 minutes (Fig. 6C). Detection of the same LAMP amplification from the seventy two NTCs with SYTO9 (Fig. 6A) showed a higher number of positives with the three positives detected by the quenched fluorophore method also positive with SYTO9 detection (highlighted in blue). Melt curve analysis showed a spread of melt temperatures for these amplicons between 85 and 90 degrees C. However agarose gel electrophoresis (Fig. S1) showed LAMP ladder patterns for the three positives detected using the JOE labelled FIP primer in contrast to the other putative positives detected only using SYTO9. The detection of the three ‘false positives’ which produced a ladder pattern similar to that seen from LAMP amplicons suggest that the JOE labelled FIP primer was one of the primers involved in the non-specific primer interaction.

Figure 6
figure6

Non-specific primer interactions detected with SYTO9 and JOE-FIP. Panel (A) shows the normalised fluorescence from SYTO9 detection and (C) shows the consecutive JOE-FIP fluorescence for NTCs. Corresponding melt curve analysis shown in panels (B,D). Negative results and SYTO9 detected positives are highlighted in red, JOE-FIP detected positives are highlighted in blue. One negative and all positives were examined on an agarose gel (E) for amplicons and the JOE-FIP detected positives showed a ladder pattern associated with LAMP amplification.

At higher concentrations of template the importance of non-specific primer interactions is likely to be less significant due to the domination of ‘true’ LAMP amplification. However at very low concentrations of template, and for example the template dilutions used in digital quantification47,48, the prevalence of ‘false positives’ will be detrimental to accurate quantifications. We therefore investigated dual detection of dilute samples of denatured maize genomic DNA with a low proportion of transgenic targets to compare sensitivity to bioluminescent detection of pyrophosphate production (BART) and to identify incidence of non-specific amplification (Fig. 7).

Figure 7
figure7

Detection of LAMP amplification with SYTO9 and JOE-FIP reporting compared to LAMP-BART results. Panels (1A) to (1F) represent amplification with the 35S promoter primers of a low concentration of denatured maize genomic DNA (0.1 percent maize GM event Bt11). Panels (2A) to (2F) represent amplification of the same sample with the NOS terminator primers. Panels (1A and 2A) show LAMP amplification with BART detection with corresponding frequency distribution of positive results (1B and 2B). Panels (1C and 2C) show detection using SYTO9 and (1E and 2E) show the simultaneous results from JOE-FIP detection. The melt curve analysis for these results are shown in panels (1D,1F,2D,2F).

LAMP BART assays of a denatured maize genomic DNA sample showed five positive results between 10 and 20 minutes and a further two positive results after 40 minutes within the 90 minute assay period, with two different primer sets targeted to the 35S promoter and NOS terminator. The dual detection using 35Sp primers and JOE-FIP highlighted six positives before 90 minutes with the majority between 10 and 40 minutes. One of the positives was detected by SYTO9 only and is therefore likely to be a false positive. The sensitivity between the LAMP BART and the dual detection methods appeared to be similar. With the NOSt LAMP dual detection the quenched fluorescence detected four positive results all between 10 and 40 minutes. The SYTO9 method detected numerous other positives but the melt curve (Fig. 7,2D) indicated that these products have a higher melt temperature than the ones identified with the quenched fluorescence method.

Carrier DNA is often used in LAMP assays to improve results9, however the mechanism by which this happens is unclear. We therefore used the dual detection method to investigate whether the addition of carrier DNA increases the incidence of non-specific primer interactions, thereby only apparently increasing the amplification frequency.

The amplification frequency of both LAMP primer interrogations of the same template showed the frequency increased using JOE-FIP quenched fluorescence detection from 18 percent without carrier DNA to approximately 25 percent with 100 nanograms per partition of salmon sperm carrier DNA (Fig. 8A). There was a general trend downwards of ‘false positives’ (in red) with increasing concentrations of carrier DNA indicating that non-specific primer interactions are not contributing to the improved amplification frequency seen with carrier DNA inclusion. For both primer sets with this template the optimum concentration of carrier DNA is 100 nanograms per partition. However, the average Ct values for both primer sets with JOE-FIP detection are faster without carrier or with 50 nanograms per partition of carrier DNA than the 100 nanogram level. The fastest Ct value at each concentration of carrier DNA with both LAMP primer sets remains fairly constant below the 20 minutes level. This result taken in conjunction with the average Ct values suggests that carrier DNA produces greater variability between replicates.

Figure 8
figure8

Carrier DNA in LAMP amplification with SYTO9 and JOE-FIP detection. Panels (1A to C) represent amplification with the 35S promoter primers of a low concentration of denatured maize genomic DNA (5.0 percent maize GM event Bt11). Panels (2A to C) represent amplification of the same sample with the NOS terminator primers. Panels (A) show the amplification frequency for increasing concentrations of carrier DNA. Cycle threshold set for all assays at −0.005. Panels (B) show the average Ct values and panels (C) shows the fastest Ct values with increasing concentrations of salmon sperm carrier DNA. Green indicates SYTO9 derived values, blue indicates JOE-FIP derived values and red indicates ‘false positive’ results derived from the difference between the SYTO9 and JOE-FIP values. In panels 1C and 2C the SYTO9 and JOE-FIP values are identical.

With non-specific LAMP detection techniques that report on concentration changes of DNA or pyrophosphate production, there is limited potential for multiplexing. The specific detection of quenched fluorescence through labelling of FIP primers was investigated further to see if the technique can be multiplexed effectively. The FIP primer in the 35S promoter primer set was replaced with a TAMRA labelled FIP and ADH1 with FAM. Denatured maize genomic DNA with a transgenic proportion was assayed with each LAMP reaction individually and as a duplex. Displacement and Loop primers were not included to reduce non-specific primer interactions (Fig. 9).

Figure 9
figure9

Duplexed LAMP reaction with ADH1 and 35Sp detection. LAMP amplification and detection of 20 copies of denatured Bt11 maize genomic DNA with ADH1 LAMP primers with FAM labelled FIP and 35Sp LAMP primers with TAMRA labelled FIP. Blue indicates template amplification and orange denotes the NTCs. The (DUPLEX) column shows the detection of two LAMP reactions in each tube. Other columns show the individual LAMP reactions and the detection channels of the thermocycler; FAM-FIP detected on green channel only and TAMRA-FIP by the orange channel.

In the duplex reaction ADH1 and 35Sp LAMP amplifications from the template are successfully detected by excitation and detection of the various fluorophores attached to the FIP primer in each set. The signal from FAM-FIP in the ADH1 LAMP amplification showed the least quenching of template amplification compared to the NTCs. The strongest signal was derived from the TAMRA-FIP used in the 35Sp LAMP amplification. Singleplex LAMP amplifications showed the correct identification in the green channel of FAM-FIP in the 35Sp LAMP amplification and appropriate negative results for the ADH1 LAMP amplifications. The orange channel selected the TAMRA labelled FIP in the 35Sp LAMP amplification and not the other LAMP amplifications.

Discussion

We show here that FIP and LoopF labelled with JOE and FAM respectively can be successfully incorporated into LAMP reactions replacing their unlabelled counterparts and are quenched as a result of amplification. The mechanism by which the fluorescently labelled FIP and LoopF primers are quenched is unclear. One theory is that the formation of cauliflower-like concatamers in the elongation stage of LAMP amplification tightly packs the fluorophores to nucleotide residues in sufficiently close proximity for fluorescence resonance energy transfer (FRET) to quench the fluorescence as previously observed35. This phenomenon was observed with labelling of both the LoopF and FIP primers and with three different fluorophores; FAM, JOE and TAMRA. The FIP and BIP primers are the most fundamental to LAMP amplification because no other primers are required during LAMP cycling and elongation phases. We therefore used the FIP primers in our investigations, seeking to make use of the phenomenon.

The 10-fold dilutions of linearised plasmid template in Fig. 2 demonstrated the potential of the quenching primer method for high sensitivity and quantification by detecting 5 copies per microlitre in approximately 40 minutes and 5 million copies in approximately 20 minutes. However, at the lower concentration of DNA template the amplification frequencies were low for the two assays. Also variation of the quenched signals was evident for some of the template concentrations and the separation between non-template fluorescence and quenched fluorescence in the presence of template amplification was poor for some concentrations. Optimisation of the primer combinations and concentrations of the JOE labelled FIP in LAMP amplification was therefore carried out (Fig. 3), highlighting the optimal concentration from these experiments of 0.8 micromolar. At this concentration the variation between replicates was low and the difference between quenched and unquenched signals was clear.

We compared the quenching of the JOE-FIP primer to detection of LAMP amplification with SYTO9 including JOE-FIP in the primer set or with unlabelled FIP substituted (Fig. 4). All three methods showed high linearity to the semi logarithmic trendline, but it was observed that moderately faster times and lower variation were evident using the unlabelled FIP primer compared to the JOE-FIP assay. Nevertheless the observation that LAMP amplification with a fluorescently labelled FIP primer was successful with SYTO9 detection suggests that amplification could be monitored in real time on the green channel for SYTO9 and yellow channel for JOE of a qPCR thermocycler as a dual detection method.

Theoretically, the labelling of a FIP or BIP primer will have absolute specificity for the LAMP amplification. In the absence of template there would be no LAMP amplification and therefore no quenching of the signal from the fluorophore labelled LAMP primer. We sought to produce false positive results with a combination of a low purity LAMP primer set with extended reaction times. Detection of increases in double stranded DNA using the intercalating dye SYTO9 showed five positive partitions out of the 72 tested. The same assay monitored with JOE-FIP did not show quenching from any of the partitions. All false positives occurred after 150 minutes at 60 degrees C and are highly unlikely to be caused by contamination; positive template results were rapid and reaction times were similar for the two assays (Fig. 5).

By combining SYTO9 detection with JOE-FIP quenching in the same tube (Fig. 6) over an extended time period we could characterise the false positives generated in the absence of template by melt curve analysis and gel electrophoresis of the amplicons. Three false positives were evident in both the green channel for SYTO9 detection and the yellow channel for JOE-FIP quenching which indicated that the FIP primer was involved in the generation of the false positives. This was supported by a shift in melt temperature for these three and a ladder pattern on the agarose gel typical for LAMP amplification indicating concatamers of varying size. The closed tube format of this dual testing restricts amplicon contamination which is a risk when opening tubes for gel electrophoresis. The results also show the synchronicity of the false positives between the two methods reporting on a single partition.

We then explored the uses of quenching as a tool to optimise LAMP assays. Bioluminescent reporting of LAMP amplification with BART25 used to detect low concentrations of transgenes in genomic DNA samples shows a spread of positive results. Our investigation of the samples with quenching dual detection showed that the fastest groups of positive results are likely to be ‘true’ positives, but later results are likely to be false positives (Fig. 7). This was also supported with the melt curve analysis which showed a shift in melt temperatures between the ‘true’ positives and the false positives. This indicates that particular sets of LAMP primers are therefore susceptible to false positives at low copy number and would benefit from further investigation and possible redesign of certain primers. Displacement, LAMP and Loop primers can be designed using Primer Explorer (Eiken, Japan), commercial software and bioinformatic programmes49,50. However, the interactions between primers, target sequences, non-target sequences and amplicons are difficult to predict in silico. This demonstrates how quenched-based dual detection of amplification can serve as an optimisation tool in the development of LAMP assays.

Carrier DNA has previously been shown to enhance sensitivity, reduce variation between replicates, and increase reaction times and amplification frequency9 when added at 100 nanograms per reaction. We used the dual detection method with 35S promoter and NOS terminator LAMP amplification of a transgenic maize DNA sample to demonstrate that amplification frequency increased at 100 nanograms per reaction without an increase in false positive detection (Fig. 8). The fastest partition at each concentration of carrier DNA were of similar values whereas amplification frequency was highest at 100 nanograms reaction, possibly indicating that carrier DNA has a positive impact on initiating LAMP amplification from available target copies, but does not affect subsequent amplification.

Many fluorescent probe and modified primer technologies with enhanced specificity also describe adaptations for multiplexing. We have shown a duplex system here with JOE labelled LAMP primers and SYTO9, and we also investigated a duplex approach with the detection of reference gene ADH1 and transgene 35S promoter (Fig. 9). The FIP primers of each of these LAMP primer sets was linked to FAM and TAMRA respectively. Our results showed that two LAMP amplifications could occur in the same tube with quenching detection of the two labelled FIP primers. The excitation:emission for FAM is 495:520 and 544:576 for TAMRA. This duplex reaction shows that multiplexing with this technique could be developed.

We conclude that the simplicity of using quenching of a suitable 5′ labelled LAMP primer is an attractive proposition for specific LAMP detection and can be used with an intercalating dye to assess the extent to which false positives may affect a new LAMP primer assay. This will be an important factor in developing LAMP primers for digital LAMP assays due to the impact a false positive result could have on quantification calculations, and will enable existing indirect LAMP amplification detection methods to be employed with well designed, high quality primers.

Methods

DNA templates

A linearised plasmid pART751, of defined copy number from calculations based on initial quantification by NanoDrop spectrophotometry and Agilent Bioanalyzer, was used as a DNA template. The plasmid contains a variant of the cauliflower mosaic virus 35S promoter sequence (35Sp). Genomic DNA was extracted from 0.1 percent w/w maize event Bt11 certified reference material (CRM) supplied by Fluka GmbH (Buchs, Switzerland). The transgenes in Bt11 are regulated by the 35S promoter from the cauliflower mosaic virus and the nopaline synthase terminator (NOSt) from Agrobacterium tumefaciens. The genomic maize DNA contains the alcohol dehydrogenase 1 gene sequence (ADH1). The Promega Wizard genomic DNA purification kit (Madison, United States) was used to extract the DNA from maize event Bt11 according to the manufacturer’s instructions for plant tissue. The final pellet was hydrated with 50 microlitres of rehydration buffer and stored at 4 degrees C. A further 5.0 percent Bt11 CRM, 5.0 percent NK603 CRM and 100 percent Mon810 maize sample were extracted using the Promega Wizard kit and quantified using the Qubit dsDNA BR assay kit (Thermofisher, Massachusetts USA) in accordance with the instructions for the Qubit 2.0 fluorometer.

Copy number calculations

The initial quantification values from NanoDrop, Agilent Bioanalyzer and Qubit were converted from nanogram per microlitre to copies per microlitre using the following formula:

$${\rm{Copies}}\,{\rm{of}}\,{\rm{target}}=\frac{{\rm{ng}}\,{\rm{of}}\,{\rm{double}}\,{\rm{stranded}}\,{\rm{DNA}}\times {\rm{Avogadro}}\mbox{'}{\rm{s}}\,{\rm{constant}}\,({\rm{6.022}}\times {10}^{23})}{{\rm{Length}}\,{\rm{in}}\,{\rm{base}}\,{\rm{pairs}}\times {{\rm{10}}}^{{\rm{9}}}\times {\rm{650}}\,{\rm{Daltons}}}$$

The size of the linearised plasmid pART7 was assumed to be 4900 base pairs and 2.4 × 10(9) base pairs was used in calculations for the maize genome.

The online calculator was used at http://cels.uri.edu/gsc/cndna.html.

The calculations of copy number for the CRM materials for maize events Bt11 and NK603, and the seed stock of Mon810 are made in accordance with Hardinge et al.46 by adjusting transgenic copy numbers by 160/385. Futhermore Bt11 has two copies of the 35Sp promoter which doubles the available target sequence.

Primer design and synthesis

Oligonucleotide primers for LAMP DNA amplification (Table 1) were synthesized and HPLC purified by Sigma Aldrich (Poole, UK). Primers were hydrated with molecular grade water to 100 micromolar and stored at minus 20 degrees C. The unmodified LAMP primers used to target 35Sp, NOSt and ADH1 sequences have previously been described and optimised9,46,52. An additional set of LAMP primers to target the 35S promoter was designed using Primer Explorer v4 from Eiken (Japan) for displacement, LAMP and loop primers. The stem primers were designed using the Primer 3 online software at http://primer3.ut.ee/. The fluorophores attached without additional modification to the 5′ end of FIP or LoopF primers were selected based on published evidence of nucleotide base quenching35. FAM, JOE and TAMRA were selected for LAMP quenching. Modifications to the 5′ terminal nucleotides were not made due to the NOSt JOE labelled FIP adjacent to a guanine base remained functional in amplification and LAMP quenching.

Table 1 Oligonucleotide primers for LAMP. Primers targetting the 35S promoter, NOS terminator and the ADH1 reference gene have been described previously and are attributed to Guy Kiddle (GK)9 and David Lee (DL)52.

Fluorescent LAMP amplification

DNA samples were amplified using LAMP and detected using SYTO9 in real time on a Qiagen (Hilden, Germany) RotorGene thermal cycler acquiring to the green channel unless otherwise specified. All reagents were supplied by Sigma Aldrich (Poole, UK) unless otherwise stated. The reaction chemistry for LAMP and amplification detection was 1X isothermal buffer (New England Biolabs Inc, Massachusetts, United States), 300 micromolar each deoxynucleotide triphosphate (dNTP), 0.8 micromolar Betaine, 0.32 units per microlitre Bst polymerase v2.0 warm start (NEB), 0.5 micromolar SYTO9 Green, 0.8 micromolar each LAMP primer, 0.4 micromolar each Loop primer, 0.2 micromolar each displacement primer and molecular grade water for a reaction volume of 20 microlitres. The parameters were set for 60 cycles of 60 seconds at 60 degrees C unless otherwise stated. Temperature melt analysis provided data between 60 and 95 degrees C. Results were analysed on RotorGene 6000 software v1.7 and Microsoft Excel. The threshold was set at the mid point between the background fluorescence and quenched fluorescence from the positive samples. The default threshold value was −0.05 on the normalised fluorescence scale. Standard normalisation was used in the RotorGene software to take the average background from the first five cycles to provide a value by which sample data points are divided.

Quenched fluorescence LAMP amplification

Oligonucleotide LAMP primers with JOE, FAM or TAMRA fluorophores attached to the 5 prime end were incorporated into the fluorescent LAMP amplification method without the inclusion of SYTO9. For dual detection SYTO9 was included with JOE labelled FIP primers. The LAMP reaction was monitored in real time on the RotorGene thermocycler according the emission wavelength of the fluorophore. The multiplex and dual detection reactions monitored 6′FAM or SYTO9 using the green channel (source 470 +/− 10 nanometers, detection 510 +/− 5 nanometers), JOE on the yellow channel (source 530 +/− 5 nanometers, detection 557 +/− 5 nanometers) and TAMRA with the orange channel (excitation at 585 +/− 5 nanometers, detection aat 610 +/− 5 nanometers) in the same reaction. Melt temperature analysis followed amplification between 60 and 95 degrees C. Results were analysed RotorGene 6000 software v1.7 and Microsoft Excel.

Real-time LAMP-BART detection

LAMP-BART chemistry was previously described by Gandelman et al.25 and Kiddle et al.9. Salmon sperm carrier DNA at 100 nanograms per partition was used in partitions of 5 microlitres. The LAMP-BART reaction mix contained 1x Thermopol buffer (NEB), 10 millimolar dithiothrietol (DTT), 0.4 milligrams per millilitre polyvinylpyrrolidone (PVP), 60 millimolar potassium chloride (KCl), 300 micromolar each dNTP, 100 micrograms per millilitre D-luciferin (Europa, Ipswich, UK), 250 micromolar adenosine-5′-O-phosphosulfate (APS; Biolog, Bremen, Germany), 5.5 micrograms per millilitre Ultra-Glo luciferase (Promega, Madison, United States), 0.375 units per microlitre ATP sulfurylase (NEB), 0.32 units per microlitre Bst polymerase v1.0, 0.8 micromolar each LAMP primer, 0.4 micromolar each Loop primer, 0.2 micromolar each displacement primer and molecular grade water. Each partition was layered with mineral oil and sealed with a clear adhesive film. The bioluminescent signal was detected and analysed using React IVD software (Synoptics, Cambridge, UK) in a ‘Lucy’ device developed by ERBA MDX (Ely, UK). The LAMP-BART assays were set for 90 minutes at 60 degrees C.

Data Availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. 1.

    Saiki, R. K. et al. Enzymatic amplification of beta-globin genomic sequences and restriction site analysis for diagnosis of sickle cell anemia. Science 230, 1350–1354 (1985).

  2. 2.

    Gill, P. & Ghaemi, A. Nucleic acid isothermal amplification technologies: a review. Nucleosides Nucleotides Nucleic Acids 27, 224–243 (2008).

  3. 3.

    Asiello, P. J. & Baeumner, A. J. Miniaturized isothermal nucleic acid amplification, a review. Lab Chip 11, 1420–1430 (2011).

  4. 4.

    Haleyur Giri Setty, M. K. & Hewlett, I. K. Point of care technologies for HIV (2014).

  5. 5.

    Notomi, T. et al. Loop-mediated isothermal amplification of DNA. Nucleic Acids Res 28, E63 (2000).

  6. 6.

    Poon, L. L. et al. Sensitive and inexpensive molecular test for falciparum malaria: Defecting Plasmodium falciparum DNA directly from heat-treated blood by loop-mediated isothermal amplification. Clinical Chemistry (2006).

  7. 7.

    Ohtsuka, K., Yanagawa, K., Takatori, K. & Hara-Kudo, Y. Detection of Salmonella enterica in naturally contaminated liquid eggs by loop-mediated isothermal amplification, and characterization of Salmonella isolates. Applied and Environmental Microbiology (2005).

  8. 8.

    Curtis, K. A., Rudolph, D. L. & Owen, S. M. Rapid detection of HIV - 1 by reverse - transcription, loop – mediated isothermal amplification (RT - LAMP). Journal of Virological Methods (2008).

  9. 9.

    Kiddle, G. et al. GMO detection using a bioluminescent real time reporter (BART) of loop mediated isothermal amplification (LAMP) suitable for field use. BMC Biotechnology 12, 15 (2012).

  10. 10.

    Watthanapanpituck, K., Kiatpathomchai,W., Chu, E. & Panvisavas, N. Identification of human DNA in forensic evidence by loop-mediated isothermal amplification combined with a colorimetric gold nanoparticle hybridization probe. International Journal of Legal Medicine (2014).

  11. 11.

    Nagamine, K., Hase, T. & Notomi, T. Accelerated reaction by loop-mediated isothermal amplification using loop primers. Mol Cell Probes 16, 223–229 (2002).

  12. 12.

    Gandelman, O., Jackson, R., Kiddle, G. & Tisi, L. Loop-mediated amplification accelerated by stem primers. Int J Mol Sci 12, 9108–9124 (2011).

  13. 13.

    Martineau, R. L. et al. Improved Performance of Loop-Mediated Isothermal Amplification Assays via Swarm Priming. Analytical Chemistry (2017).

  14. 14.

    Kaneko, H., Kawana, T., Fukushima, E. & Suzutani, T. Tolerance of loop-mediated isothermal amplification to a culture medium and biological substances. J Biochem Biophys Methods 70, 499–501 (2007).

  15. 15.

    Tani, H. et al. Technique for quantitative detection of specific DNA sequences using alternately binding quenching probe competitive assay combined with loop-mediated isothermal amplification. Anal Chem 79, 5608–5613 (2007).

  16. 16.

    Kubista, M. et al. The real-time polymerase chain reaction (2006).

  17. 17.

    Holland, P. M., Abramson, R. D., Watson, R. & Gelfand, D. H. Detection of specific polymerase chain reaction product by utilizing the 5′-3′ exonuclease activity of Thermus aquaticus DNA polymerase. Proc Natl Acad Sci USA 88, 7276–7280 (1991).

  18. 18.

    Tyagi, S. & Kramer, F. R. Molecular Beacons: Probes that Fluoresce Upon Hybridization. Nature Biotechnology 14, 303–308 (1996).

  19. 19.

    Chou, P. H., Lin, Y. C., Teng, P. H., Chen, C. L. & Lee, P. Y. Real-time target-specific detection of loop-mediated isothermal amplification for white spot syndrome virus using fluorescence energy transfer-based probes. J Virol Methods 173, 67–74 (2011).

  20. 20.

    Mori, Y., Nagamine, K., Tomita, N. & Notomi, T. Detection of loop-mediated isothermal amplification reaction by turbidity derived from magnesium pyrophosphate formation. Biochem Biophys Res Commun 289, 150–154 (2001).

  21. 21.

    Tomita, N., Mori, Y., Kanda, H. & Notomi, T. Loop-mediated isothermal amplification (LAMP) of gene sequences and simple visual detection of products. Nat Protoc 3, 877–882 (2008).

  22. 22.

    Goto, M., Honda, E., Ogura, A., Nomoto, A. & Hanaki, K. Colorimetric detection of loop-mediated isothermal amplification reaction by using hydroxy naphthol blue. Biotechniques 46, 167–172 (2009).

  23. 23.

    Kouguchi, Y., Fujiwara, T., Teramoto, M. & Kuramoto, M. Homogenous, real-time duplex loop-mediated isothermal amplification using a single fluorophore-labeled primer and an intercalator dye: Its application to the simultaneous detection of Shiga toxin genes 1 and 2 in Shiga toxigenic Escherichia coli isolates. Mol Cell Probes 24, 190–195 (2010).

  24. 24.

    Kubota, R. et al. Molecular diagnostics in a teacup: Non-Instrumented Nucleic Acid Amplification (NINA) for rapid, low cost detection of Salmonella enterica. Chin Sci Bull 58, 1162–1168 (2013).

  25. 25.

    Gandelman, O. A. et al. Novel bioluminescent quantitative detection of nucleic acid amplification in real-time. PLoS ONE 5 (2010).

  26. 26.

    Iwamoto, T., Sonobe, T. & Hayashi, K. Loop-mediated isothermal amplification for direct detection of Mycobacterium tuberculosis complex, M. avium, and M. intracellulare in sputum samples. J Clin Microbiol 41, 2616–2622 (2003).

  27. 27.

    Parida, M. M. Rapid and real-time detection technologies for emerging viruses of biomedical importance. J Biosci 33, 617–628 (2008).

  28. 28.

    Seyrig, G. et al. Selection of fluorescent DNA dyes for real-time LAMP with portable and simple optics. J Microbiol Methods 119, 223–227 (2015).

  29. 29.

    Njiru, Z. K. et al. Loop-mediated isothermal amplification (LAMP) method for rapid detection of Trypanosoma brucei rhodesiense. PLoS Negl Trop Dis 2, e147 (2008).

  30. 30.

    Mori, Y., Kitao, M., Tomita, N. & Notomi, T. Real-time turbidimetry of LAMP reaction for quantifying template DNA. J Biochem Biophys Methods 59, 145–157 (2004).

  31. 31.

    Tanner, N. A., Zhang, Y. & Evans, T. C. Visual detection of isothermal nucleic acid amplification using pH-sensitive dyes. Biotechniques 58, 59–68 (2015).

  32. 32.

    Fischbach, J., Xander, N. C., Frohme, M. & Glökler, J. F. Shining a light on LAMP assays–a comparison of LAMP visualization methods including the novel use of berberine. Biotechniques 58, 189–194 (2015).

  33. 33.

    Liu, W. et al. Establishment of an accurate and fast detection method using molecular beacons in loop-mediated isothermal amplification assay. Scientific Reports 7 (2017).

  34. 34.

    Mori, Y., Hirano, T. & Notomi, T. Sequence specific visual detection of LAMP reactions by addition of cationic polymers. BMC Biotechnol 6, 3 (2006).

  35. 35.

    Torimura, M. et al. Fluorescence-quenching phenomenon by photoinduced electron transfer between a fluorescent dye and a nucleotide base. Anal Sci 17, 155–160 (2001).

  36. 36.

    Zerilli, F. et al. Methylation-specific loop-mediated isothermal amplification for detecting hypermethylated DNA in simplex and multiplex formats. Clin Chem 56, 1287–1296 (2010).

  37. 37.

    Spinelli, O. et al. Simple, rapid and accurate molecular diagnosis of acute promyelocytic leukemia by loop mediated amplification technology. Oncoscience 2, 50–58 (2015).

  38. 38.

    Aonuma, H. et al. A single fluorescence-based LAMP reaction for identifying multiple parasites in mosquitoes. Experimental Parasitology (2010).

  39. 39.

    Stratakos, A. C., Linton, M., Millington, S. & Grant, I. R. A loop-mediated isothermal amplification method for rapid direct detection and differentiation of nonpathogenic and verocytotoxigenic Escherichia coli in beef and bovine faeces. Journal of Applied Microbiology (2017).

  40. 40.

    Tanner, N. A., Zhang, Y. & Evans, T. C. Simultaneous multiple target detection in real-time loop-mediated isothermal amplification. Biotechniques 53, 81–89 (2012).

  41. 41.

    Jiang, Y. S. et al. Robust Strand Exchange Reactions for the Sequence-Specific, Real-Time Detection of Nucleic Acid Amplicons. Analytical Chemistry (2015).

  42. 42.

    Waiwijit, U. et al. Graphene oxide based fluorescence resonance energy transfer and loop-mediated isothermal amplification for white spot syndrome virus detection. J Biotechnol 212, 44–49 (2015).

  43. 43.

    Ball, C. S. et al. Quenching of Unincorporated Amplification Signal Reporters in Reverse-Transcription Loop-Mediated Isothermal Amplification Enabling Bright, Single-Step, Closed-Tube, and Multiplexed Detection of RNA Viruses. Anal Chem 88, 3562–3568 (2016).

  44. 44.

    Nkouawa, A., Sako, Y., Okamoto, M. & Ito, A. Simple identification of human taenia species by multiplex loop-mediated isothermal amplification in combination with dot enzyme-linked immunosorbent assay. American Journal of Tropical Medicine and Hygiene (2016).

  45. 45.

    Gadkar, V. J., Goldfarb, D. M., Gantt, S. & Tilley, P. A. Real-time Detection and Monitoring of Loop Mediated Amplification (LAMP) Reaction Using Self-quenching and De-quenching Fluorogenic Probes. Scientific Reports (2018).

  46. 46.

    Hardinge, P., Kiddle, G., Tisi, L. & Murray, J. A. H. Optimised LAMP allows single copy detection of 35Sp and NOSt in transgenic maize using Bioluminescent Assay in Real Time (BART). Scientific Reports (2018).

  47. 47.

    Vogelstein, B. & Kinzler, K. W. Digital PCR. Proc Natl Acad Sci USA 96, 9236–9241 (1999).

  48. 48.

    Gansen, A., Herrick, A. M., Dimov, I. K., Lee, L. P. & Chiu, D. T. Digital LAMP in a sample self-digitization (SD) chip. Lab Chip 12, 2247–2254 (2012).

  49. 49.

    Torres, C. et al. LAVA: an open-source approach to designing LAMP (loop-mediated isothermal amplification) DNA signatures. BMC Bioinformatics 12, 240 (2011).

  50. 50.

    Salinas, N. R. & Little, D. P. Electric LAMP: Virtual Loop-Mediated Isothermal AMPlification. ISRN Bioinformatics (2012).

  51. 51.

    Gleave, A. P. A versatile binary vector system with a T-DNA organisational structure conducive to efficient integration of cloned DNA into the plant genome. Plant Mol Biol 20, 1203–1207 (1992).

  52. 52.

    Lee, D., La Mura, M., Allnutt, T. R. & Powell, W. Detection of genetically modified organisms (GMOs) using isothermal amplification of target DNA sequences. BMC Biotechnol 9, 7 (2009).

Download references

Acknowledgements

This work was supported by BBSRC Stand Alone LINK Grant BB/L022346/1, a BBSRC Industrial CASE Award, and part funded by the European Regional Development Fund.

Author information

P.H. and J.A.H.M. conceived the experiments, P.H. conducted the experiments, P.H. and J.A.H.M. analysed the results and wrote the manuscript. Both authors reviewed the manuscript.

Correspondence to Patrick Hardinge.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.