Introduction

In recent years, continuous glucose monitoring (CGM) systems have positioned themselves as a very useful tool to improve metabolic control in patients with diabetes. These devices allow obtaining more complete information on glycemic behavior than through traditional measurement methods (capillary blood glucose), allowing the patient and the health professional to make more complex therapeutic decisions that have an impact on improving metabolic control1. We currently have a growing number of product options for using CGM which we can classify into (1) real-time CGM systems (rtCGM) (2) intermittently scanned CGM systems (isCGM) and (3) professional CGM systems. All these devices measure interstitial glucose levels and provide numerical and graphical information about glucose profiles however there are differences with respect to handling and clinical use. rtCGM system send glucose information continuously from the sensor to the user’s receiver, smartphone, or insulin pump. isCGM systems provide and store the information when the user approaches the receiver or smartphone to the sensor. Professional CGM refers to use of devices that are owned by the clinic and used to retrospectively analyze glucose data. These devices can be used in the “blinded” mode to capture information about what patients are doing without influencing their behavior2.

Since their introduction, these devices have revolutionized3 our understanding of glycemic behaviour, enabling more precise monitoring than ever before. With their impressive technical capabilities, CGM systems hold tremendous potential for both clinical and research applications. One of the most significant advantages of CGM systems is their ability to enhance glycemic control. By providing real-time glucose data, these devices may help a larger number of patients achieve and maintain glycemic target glycated hemoglobin and time in range values4 while minimizing the risk of hypoglucemia5. Furthermore, the reliability and accuracy of CGM systems have paved the way for seamless integration with subcutaneous insulin infusion systems. This integration allows insulin delivery to be dynamically adjusted based on CGM information, offering an automated and efficient approach to manage blood glucose levels6,7,8.

CGM have shown promise in not only monitoring glucose levels in patients with diabetes but also in epidemiological studies evolving healthy volunteers and the general population9,10,11. These studies have provided valuable insights into glycemic behaviour of health individuals under real-life conditions. Understanding glucose profiles in non-diabetic populations has significant clinical implications, ranging from detecting early dysglycemia to preventing or delaying the onset of diabetes. Additionally, investigating post-prandial responses to nutrient combinations is of great interest at improving the overall health of the general population12.

To ensure proper utilizations of CGMs, it is crucial to assess the reliability and reproducibility of these devices over multiple days, considering the biological variations across individuals and time (see an example in another domain13). This evaluation is necessary to understand how perform in different populations and to stablich effective recommendations for clinical decisions derived from CGM procedures.

In a near future, with the development in the material, power supply, and data transmission area, the wearable point-of-care glucose sensors will be more miniaturized, accurate, and self-powered14. The success of CGM technology will depend on the quality of the monitoring information it provides. It is essential that the validity and reliability of the measurements taken be known. Traditionally, validity has been determined by comparing the values obtained by monitoring devices to standard reference values obtained by the self‐monitoring of capillary blood glucose6. Reliability or reproducibility refers to the consistency of measurements across repeated tests and/or varying conditions15; with respect to CGM systems, this has not been addressed.

Subcutaneous CGM sensors measure the glucose concentration of the interstitial space and use algorithms to extrapolate blood glucose levels from the values recorded16. In devices that display glucose values every 5 min, 288 values are reported each day. However, understanding glycaemic behaviour requires a knowledge of the magnitude of fluctuations in blood sugar concentration, and an overall comprehension of its variability. The latter can be appreciated in the shape of blood glucose curves17.

Recent advances in statistical methods, such as functional data analysis (FDA)18, 19, have opened up valuable opportunities to gain deeper insights into the complexities of glucose time series data obtained from CGM systems. FDA represents a powerful extension of traditional multivariate analysis which allows to treat mass data as dynamic curves that evolve over time. By treating the data as a collection of latent temporal processes. FDA facilitates the investigation of dynamic changes over time, providing a richer understanding of glucose dynamics. This novel approach holds significant potential for enhancing various statistical modeling techniques, including hypothesis testing and regression19. The utilization of FDA can introduce greater accuracy and reliability in predicting clinical outcomes or detecting relevant clinical and statistical differences related to glucose fluctuations.

To date, no study has used FDA to examine the reproducibility of results provided by CGM systems. The present work uses an FDA method to examine those thus obtained in a random sample of a free-living, adult population, with respect to its members' glycaemic status.

Materials and methods

Study design

The study subjects in this cross-sectional investigation were a subset of those enrolled in the A Estrada Glycation and Inflammation Study (AEGIS), trial NCT01796184 at www.clinicaltrials.gov. A detailed description of the latter study has been published elsewhere10. AEGIS was a cross-sectional study conducted in the municipality of A-Estrada, in the northwest of Spain. An age-stratified random sample of the population aged 18 years and older was drawn from Spain’s National Health System Registry.

Patients

Of the total of 1516 participants recruited a subsample of 622 individuals participated in the Glycation project, which included continuous glucose monitoring procedures. From March 2013 until March 2015, subjects were successively convened at the A Estrada Primary Care Centre (A Estrada, Galicia, Spain) where they (i) completed an interviewer-administered structured questionnaire that collected demographic and anthropometric data, (ii) provided a description of their lifestyle, including information on diet, physical activity, alcohol consumption and smoking, (iii) provided a fasting venous blood sample, and (iv) were prepared for CGM. Of the 622 subjects who consented to undergo a 6-day period of CGM (361 women, 220 men), 581 completed at least 2 days of monitoring; the data of these latter subjects were used in analyses. The remaining 41 subjects were excluded due to non-compliance with the demands of the protocol (n = 4), or difficulties in handling the device (n = 37).

Calculation of body mass index. Body mass index (BMI) was calculated as body weight (kg)/height (m)2.

Assessment of physical activity. Subjects completed the International Physical Activity Questionnaire (short form), and the metabolic equivalents of the hours per week engaged in vigorous and moderate activities and walking were calculated as described by Craig20. Subjects were then classified as either (i) "inactive" (n = 209, 36%), (ii) "minimally active" (n = 223, 38%), or (iii) HEPA (health-enhancing physical activity)-active" (n = 149, 26%).

Classification of alcohol consumption and smoking. Alcohol consumption was recorded in terms of standard drinking units, summing the number of glasses of wine (~ 10 g alcohol per glass), bottles of beer (~ 10 g per 200 ml), and units of spirits (~ 20 g alcohol per measure) regularly consumed per week, as previously described21. Smoking habits were recorded as the number of cigarettes regularly consumed per day. Consumers of at least one cigarette per day were considered smokers. Individuals who had quit smoking during the preceding year were regarded as smokers.

Baseline and final laboratory determinations. Glucose was determined in plasma samples from fasting participants via the glucose oxidase peroxidase method. Glycated haemoglobin (A1C) was determined by high-performance liquid chromatography using a Menarini Diagnostics HA-8160 analyser; all A1C values were converted to DCCT-aligned values22. Serum insulin was determined using the ADVIA Centaur XP immunoassay system (Siemens, Erlangen, Germany). Insulin resistance was estimated using the homeostasis model assessment method (HOMA-IR) as the fasting concentration of plasma insulin (µ units/mL) × plasma glucose (mg/dL)/40523.

Baseline glycaemic status. Individuals were deemed to be diabetic if they had been previously diagnosed as such, or had an A1C level of > 6.4%, and/or a fasting plasma glucose (FPG) concentration of > 125 mg/dL (n = 70, 12%). Subjects with prediabetes were defined as those with an A1C range of 5.7–6.4% or a FPG range of 100–125 mg/dL (n = 121, 21%). Normoglycaemic subjects were defined as those with an A1C of < 5.7% and an FPG of < 100 mg/dL (n = 390, 67%). Baseline glycemic status (normoglycemia, prediabetes, and diabetes) were defined according to the American Diabetes Association criteria24.

Dietary variables

During the CGM assessment period (see below), participants monitored their food and drink intakes. They were instructed to record the weight or portion size of all items consumed, and to provide a detailed description of each at the time of consumption. At the end of each subjects' assessment period, a research dietician reviewed the intake records and asked for additional data when records appeared incomplete or implausible. Total energy intake (kcal), and intakes (g and %) of fat, protein, carbohydrate, sugar and fibre were determined using Dietowin® 8.0 software (BioLogica Tecnologica Medica SL, Barcelona, Spain, http://www.bl-biologica.es/dietowin.htm), and mean daily totals calculated for each complete 24 h (midnight to midnight) period.

CGM data collection

At the start of the patient monitoring period, a research nurse inserted an Enlite™ sensor (Medtronic Inc., Northridge, CA, USA) subcutaneously in the subject's abdomen, and instructed the subject in the care of the connected iPro™ model CGM device (Medtronic Inc, Northridge, CA, USA). iPro™ is a professional blind CGM system which provides retrospective information on glucose profiles. As the system does not provide data on glucose values during the use of the device, the behavior of the patients is not affected by the information from the CGM. The sensor continuously measures the glucose concentration of the interstitial space in the subcutaneous tissue, recording values (range 40–400 mg/dL) every 5 min, and storing them in the CGM device. On the 7th day the sensor was removed and the data downloaded for analysis, disregarding the first day's results. In addition, if data-acquisition failure totalled more than 2 h in a day, the entire day's data were discarded.

Subjects were also provided with a conventional OneTouchR VerioR Pro glucometer (LifeScan, Milpitas, CA, USA) and compatible lancets and test strips. To guarantee the reliability and quality of the monitoring data, the participants were instructed to perform at least 3 capillary blood glucose measurements per day (usually before the main meals). The capillary blood glucose readings were used to calibrate the iPro™ CGM system. Data from monitoring that could not be calibrated with at least 3 capillary blood glucose controls per day were excluded from the analysis.

For all 583 subjects, there were a total of 9980 paired points (median 18 samples per patient) with both capillary BG measurements and CGM system interstitial fluid glucose measurements. Overall sensor accuracy expressed as Mean Absolute Relative Difference (MARD) was 7.9%. The sensor accuracy was highest during periods of euglycemia and hyperglycemia (> 180 mg/dL) and lowest during hypoglycemia. The mean ARD was 7.8% when blood glucose was between 70 and 180 mg/dL; 9.5% when blood glucose was greater than 180 mg/dl; and 29.2% when blood glucose was less than 70 mg/dL. Eighty-seven percent of the device results were within 15 mg/dL of the capillary BG results (for results of less than 100 mg/dL), and 87% were within 15% of the capillary BG results (for results higher than 100 mg/dL). The performance of the system on the first day was different to that on the following days. MARDs for all capillary-sensor glucose paired points stratified by day (1–6) were 12.1%, 7.6%, 7.0%, 7.1%, 7.3% and 6.6%, respectively.

Thus, data from day 1 were excluded.

Ethical issues

Written informed consent was obtained from all participants. The study was approved by the Regional Ethics Committee (Comité Ético de Investigación Clínica de Galicia, registration code: 2012/025) and conformed to the current Helsinki Declaration.

Statistical analyses

To assess the inter-day reproducibility of the CGM results according to subject glycaemic status (normoglycaemic, prediabetic and diabetic), functional intraclass correlation coefficients (ICC) and their 95% confidence intervals (95% CI), were calculated. ICC coefficients range from 0 to 1, and can be interpreted as follows: < 0.4 poor agreement; 0.4–0.59 fair agreement; 0.60–0.74 good agreement; > 0.74 excellent agreement25. The functional iCC was computed using the instrumental methodology of a two-way ANOVA multilevel functional model, as introduced in reference26. To achieve this, we use a novel bootstrap methodology, which is elaborated on in a subsequent paper27. For those interested in ICC calculation for other study designs from a functional perspective, additional information can be found in reference19.

To provide an idea of the variability in glucose measurements recorded by the CGM, two measures of glycaemic variability were analyzed: the coefficient of variation (CV), a measure of intra-day glycaemic variability28 and the mean of the daily differences (MODD), a measure of inter-day glycaemic variability, as previously described27. Specifically, CV was calculated as 100 × standard deviation/mean glucose. And MODD was calculated as follows:

$$MODD=\frac{{\sum }_{t1}^{tk}\left|BGt-BGt-1440\right|}{k}$$

where BG is blood glucose at time t, and k is the number of observations where there is an observation at the same time 24 h (1440 min) ago.

For the normoglycemic subjects, ICC, CV and MODD were calculated stratifying by demographics (gender, age groups) and life-styles (tobacco and alcohol consumption, physical activity and diet). All statistical functional data analyses were conducted using the statistical software R. For the functional exploratory analysis, we employed the refund package. Visualizations of functional data were created using standard R plotting functions. Lastly, the estimation of the functional ICC calculation was performed using the R package I2C2.

The I2C2 procedure allows the determination of ICCs when working with complex datasets. A non-parametric bootstrap procedure was performed to determine the I2C2 confidence level27. The code for calculating functional ICC is available at http://www.biostat.jhsph.edu/~ccrainic/software.html, or https://neuroconductor.org/package/I2C2.

Results

The subjects had a mean age of 48 years (range 18–87); 62% were females. Seventy (12%) had previously been diagnosed with diabetes (100% type 2), and 121 (21%) fulfilled the criteria of prediabetes. Among those with diabetes, 69% took oral antidiabetic medication, 4% used insulin alone, 17% used insulin plus oral drugs, and 10% used neither insulin nor oral antidiabetic drugs. None of the participants was on treatment with GLP1 analogue.

The subjects with diabetes were older than those with prediabetes, and also older than the normoglycaemic subjects (61 ± 12, 57 ± 12, and 43 ± 13 years; respectively). The subjects with diabetes were also more commonly men (50%, 40% and 36%; respectively). Subjects with diabetes and prediabetes had higher BMIs than their normoglycaemic counterparts 31.2 ± 5.1 and 31.1 ± 4.8 vs. 26.7 ± 4.6 kg/m2). As expected, baseline PFG, A1C and the HOMA-IR value, as well as the glycaemic variability indices (CV and MODD), were highest among subjects with diabetes (Table 1).

Table 1 Clinical characteristics of the study participants according to glycaemic status.
Full size table

The inter-day reproducibility of the GCM measurements was greater in subjects with diabetes (ICC 0.46 [95% CI 0.39–0.55), than in either the normoglycaemic subjects (ICC 0.30 [95% CI 0.27–0.33]) or those with prediabetes (ICC 0.37 [95% CI 0.31–0.42]).

Table 2 shows the reproducibility (ICC) and glycaemic variability (CV and MODD) coefficients (and their 95% CI) for the normoglycaemic subjects in relation to demographic data and lifestyle.

Table 2 Reproducibility of continuous glucose monitoring records and glycaemic variability indices in normoglycaemic subjects according to demographic data and lifestyle.
Full size table

The inter-day reproducibility of the CGM data was better in the elderly than in the younger people, although age influenced neither intra-day nor inter-day glycaemic variability.

As can be seen in Table 2 inter-day glycaemic variability (MODD) was lower in heavy drinkers than in light drinkers. CV was also lower in heavy drinkers than in light drinkers but without reaching statistical significance. Ex-smokers showed higher reproducibility and lower inter-day glycaemic variability than non-smokers. No significant differences were found in reproducibility nor glycaemic variability with respect to gender, BMI, physical activity, the number of meals consumed per day, daily calorie intake or daily carbohydrate intake.

Discussion

The use of CGM systems in non-diabetic populations has witnessed a surge in recent years, offering valuable insights into glycemic behavior. These devices extend their clinical implications beyond the traditional monitoring and control of diabetes, encompassing applications in diabetes prevention and diagnosis, reinforcement of healthy habits, and evaluating treatments. Some of these applications are still in the developmental phase, showing promising potential, particularly in screening for diabetes-related diseases.

Given the significance of these diverse applications, it becomes imperative to comprehensively understand essential aspects of CGM system behavior across different populations and situations. Evidence-based recommendations for the optimal use of CGM devices rely heavily on grasping factors such as reproducibility. Ensuring the reliability and consistency of CGM data across days and various time scales is crucial in substantiating the validity of findings and recommendations.

This paper presents a pioneering perspective on exploring reproducibility in CGM data from a functional data analysis (FDA) standpoint. The methodology utilized in this study can prove instrumental in conducting similar essential analyses with other wearables and biosensors, paving the way for advancements in personalized healthcare and disease management3, 19.

The present results show the inter-day reproducibility of the GCM readings to be greater for the subjects with diabetes than for those with prediabetes or those who were normoglycaemic (ICCs = 0.46, 0.37 and 0.30, respectively). Overall, the inter-day reproducibility of the CGM readings was poor, although in patients with diabetes it might be considered fair. Among the normoglycaemic subjects, inter-day reproducibility became greater with age.

Recent studies have analyzed the influence of glycemic variability (GV) as an independent risk factor in the long-term development of diabetes-related complications29. In addition, the recommendations on clinical endpoints for the interpretation of continuous glucose monitoring data suggest that parameters that measure GV be included. Among the many metrics that exist to quantify GV, CV is the most appropriate measure to identify short-term glycemic variability and levels for patients with diabetes should be kept below 36%4, although some studies recommend lower targets (< 33%) for patients treated with insulin to reduce the risk of hypoglycemia30. In our study, patients with diabetes presented values of glycemic variability within the control objectives. Since our study, the evaluation of continuous glucose monitoring (CGM) metrics with healthy populations has seen significant progress. Recent studies in healthy populations in large-scale cohorts from Israel, such as the work by Keshet et al.11, have corroborated our glucose variability estimations. These findings have opened up new opportunities to assess the reliability of CGM metrics in conjunction with other biomarkers.

Figure 1 shows different daily glucose profiles for the normoglycaemic, prediabetic and diabetic subjects. For the diabetic subjects, the curves for consecutive days appear to be more similar to one another, the possible result of these subjects showing less functional adaptation. This less complex dynamic behaviour means that future CGM readings are more predictable from preceding readings. Similarly, reproducibility increased with age in the normoglycaemic subjects, suggesting age to be associated with subjects showing less functional adaptation capacity and thus more stable CGM readings. In addition, the clinical recommendations in diabetes, regardless of the type of treatment and the degree of control, are based on maintaining a healthy eating plan adapted to pharmacological treatment, as well as carrying out a physical exercise plan. These recommendations can cause people to have a more organized way of life in daily life conditions regarding issues that affect the glycemic response (such as maintaining the proportion of carbohydrates and the timing of meals). However, more research is needed in different populations of patients with diabetes (for example, in individuals with high glycemic variability) and especially in people with prediabetes and normoglycemia.

Figure 1
figure 1

Continuous glucose monitoring profiles for 5 days in normoglycaemic subjects, and in those with prediabetes and diabetes.

Full size image

The reproducibility of CGM readings was also greater among ex-smokers than non-smokers. This might be explained by the former being older. However, many smokers quitted because they had some smoking-related pathology which may have also reduced their functional adaptation capacity.

Different methods have been developed for analyzing the variability in physiological signals over time, and compared to non-diabetic subjects, reduced inter-day variability in blood glucose concentrations has been recorded in patients with diabetes31, 32, with a more pronounced loss of dynamic complexity in patients with type 1 diabetes33. Kohnert et al. also indicate beta-cell function to be an independent predictor of glucose time series dynamics as measured by detrended fluctuation analysis34, and report results compatible with increased glucose variability as determined from classical measures of standard deviation.

The AEGIS study database is one of the few population-based epidemiological studies using CGM technology in a random sample of a general population composed of normoglycemic, prediabetic, and diabetic patients. The use of continuous glucose monitoring in healthy populations11, 35,36,37,38,39 is topic of growing interest due to its multiple applications in epidemiological studies17, 38 or diet optimization40, 41. Therefore, our research is fundamental as it provides new information about the reproducibility of CGM with non-diabetic patients and uses the raw time series recorded by the CGM device as a richer piece of information.

The present results have important clinical implications. CGM monitoring over a single day may not be enough for solid conclusions on glucose homeostasis to be drawn; it may be necessary to monitor patients for longer. In February 2017, a panel of experts42 recommended a minimum of 14 consecutive days of CGM for optimal analysis and decision making a recommendation based on longitudinal CGM data from randomized trials undertaken by the The Juvenile Diabetes Research Foundation43. As expected, the fuller days of glucose data sampled, the stronger the correlation with 3 months' worth of data. The authors also suggested that a 12–15 day period of monitoring every 3 months may be needed to optimally assess overall glucose control. Unlike in the present study, the latter involved summary measures of glycaemic indices, and the correlation between different sample periods and the 3-month interval was determined via a coefficient based on ranks.

The amount and type of information provided by the CGM systems represent a challenge for its analysis, therefore it is necessary to develop new statistical methodologies44. Furthermore, we would like to emphasize that we are examining for the first time the reproducibility of CGM records with a raw analysis of temporal signals in the field of diabetes with techniques based on functional data analysis. This type of analysis provides a different and novel approach in the study of glycemic behavior through the information provided by glucose curves. The analysis of reliability with functional data analysis is a common technique with other devices such as Functional Magnetic Resonance Imaging (fMRI) or accelerometry devices45, 46. Traditional statistical methods may not adequately capture the dynamic nature of the CGM data and the underlying patterns. New statistical methods, such as functional data analysis (FDA), offer valuable insights into understanding the complexities of glucose time series data obtained from CGM systems. By analyzing the dynamic changes in glucose values over time, FDA provides a powerful extension of multivariate analysis. It allows us to explore the data as a set of latent temporal processes, represented, for example, by five glucose functions, from a functional perspective. This approach effectively exploits the underlying structure of the mathematical function in the data.

Functional data analysis may provide a potentially useful alternative approach47, 48. By considering the entire individual glucose trajectory as a functional unit and incorporating appropriate statistical models, functional data analysis may help researchers to characterize and compare glucose profiles, identify temporal patterns, and assess the impact of various factors on glucose dynamics49, 50. Incorporating FDA into the study of glucose time series data yields several modeling advantages, as highlighted in the biomechanics field45. For example, one hand, it enhances statistical power in hypothesis testing, enabling more robust and conclusive results. One another hand, FDA enables more accurate and precise regression modeling, providing a deeper understanding influence of the temporal dynamics in glucose levels.

This study suffers from the inherent limitations of its cross-sectional design, furthermore, the data were collected between March 2013 and March 2015 and CGM systems have advanced rapidly in the last 8 years. However, the sensor technology is the same (measurement of glucose oxidase in interstitial fluid) and the accuracy obtained (defined by its MARD) is similar to that presented in recent clinical studies51. The MARD is a parameter that expresses the average difference between the measurement of the system and the reference standard (in this case capillary blood glucose). Although the MARD is not as accurate when it is used to characterize the specific sensor alone, this parameter provides a number that reflects the total performance of a given CGM system in a clinical study51. Presentation of all MARD data available provide a better understanding of the CGM system performance and supports comparison of different CGM systems plus facilitates understanding the improvement seen with different generations of a given CGM system51.

In addition, no optimal number of monitoring days can be deduced. The strength of this work lies in its use of a new index to measure inter-day reproducibility that takes into account the dynamic nature of plasma glucose concentrations. The sample was also composed of subjects selected randomly from the general, free-living population, and included normoglycemic persons, those with prediabetes, and those with diabetes.

In conclusion, the present results show that CGM readings for diabetic subjects are more reproducible from one day to the next. This might indicate that these subjects have lost functional adaptational capacity compared to, for example, normoglycemic subjects. Given the promise of CGM systems as clinical tools, it would be of value to compile best practice guidelines aimed at increasing the reproducibility and validity of the results they provide.