Introduction

In Europe, regulation (EC 1169/2011) foresees labeling and other forms of visual information on the nutritional content of foods [1]. Adopting simplified nutrition labeling such as front-of-pack logos implies choosing a validated nutrient profiling system, able to rank foods according to nutritional composition, as defined by the World Health Organization (WHO) [2], but the European regulation doesn’t stipulate which nutrient profiling system should be used to implement labeling.

In this context, the simplified nutrient profiling system (SENS) was derived from the SAIN,LIM system [3] to provide an algorithm operational for simplified nutrition labeling in Europe [4]. The SENS algorithm was obtained by adapting the SAIN,LIM system [5], a transversal nutrient profiling system defining four classes based on two scores: a qualifying score named SAIN and a disqualifying score (or score of nutrients to limit) named LIM, and one threshold on each score [3]. The SAIN,LIM system was originally developed by the French Food Standard Agency in 2008, in response to the European regulation on nutrition and health claims [6]. Adaptation of the algorithm for labeling purpose followed a step-by-step process [5], as encouraged by WHO in its manual guiding the development of nutrient profiling systems [2]. The adaptation involved reducing the number of nutrients needed for calculation, integrating food-category specificities, ordering the four classes and introducing European reference intakes into the algorithm [5]. SENS allocates each food to one of four classes according to two nutritional scores: SAINSENS, and LIMSENS [4], respectively derived from SAIN and LIM scores.

WHO guidelines for validating nutrient profile systems include simple methods, such as testing whether classifications using nutrient profiling are in line with food‐based dietary guidelines, and more complex methods [2], such as testing whether healthy foods (as identified by the nutrient profile model) make healthy diets (and whether unhealthy foods make unhealthy diets) using observed diets and/or theoretical diets.

However, the healthy vs. unhealthy dichotomy creates hurdles when nutrient profiling systems for food labeling are based on more than two classification levels, so validation studies built from WHO guidelines need to be adapted and able to rank all foods into several different nutritional quality classes (e.g., four here for the SENS). It is thus necessary to introduce this hierarchy in order to validate any system featuring more than two classes.

The objective of this study was to validate the simplified nutrient profiling system (SENS) algorithm according to its ability to rank foods in the four SENS classes according to overall nutritional quality of diet using both observed and theoretical diets.

Materials and methods

Dietary data and food-composition database

Dietary data of 1719 normal-reporter adults (>19 years) from the previously-described French national INCA2 survey [7] was used in this study. All foods and beverages (including water and excluding alcohol) declared as consumed in 7-day food records were matched to 1256 foods in the French nutritional database, CIQUAL 2013 [8]. This database was completed with data on free sugars (g/100 g) following the methodology published elsewhere [9] and with percentage of fruit and vegetables (including dry fruit and nuts). Standard recipes were used to estimate percentages of cereals, dairy products (cheese or other dairy products), fish and eggs, required to calculate the SENS scores. Each food was categorized as either a ‘Beverage’, an ‘Added Fat’ or as an ‘Other Solid Food’. A portion size was defined for each food item based on standard portion sizes used in French and foreign epidemiological studies [10,11,12,13] and collective catering in France [14] (Table 1). Daily macro- and micro-nutrient intakes based on dietary intakes and nutritional composition of the food items were estimated for each subject.

Table 1 Portion sizes
Full size table

INCA2 survey was approved by the French authority of data protection (CNIL:“Commission Nationale Informatique et Libertés” No. 2003 × 727 AU) and the French national council for statistical information (“Conseil National de l’Information Statistique”). Verbal informed consent was obtained from all participants and formally recorded.

Food ranking with the SENS algorithm

SENS was used to rank 1256 foods from the CIQUAL table into one of the four SENS classes from most (Class-1) to least (Class-4) favorable profile according to thresholds applied to a nutrient density score (SAINSENS) and a limited nutrient score (LIMSENS). The SENS algorithm is described in supplementary information and detailed in full elsewhere [4].

Sorting observed diets into four groups of increasing overall nutritional quality

Individual diets were classified based on their overall nutritional quality using a previously-described method [15]. Briefly, three indicators were calculated for each diet: solid energy density (SED) [16], mean adequacy ratio (MAR, mean percent of French Recommended Dietary Allowance (RDA) values for nutrients whose intake is encouraged) [17], and mean excess ratio (MER, mean percent of maximal recommended values for nutrients whose intake should be limited) [15]. Individual diets were then ranked based on these indicators compared to the observed medians in the population. The “High” nutritional quality group was defined for observed diets meeting three properties, i.e., i) a SED below the median, ii) a MAR above the median, iii) a MER below the median. Diets meeting two, one or zero of these properties were allocated to “Intermediate+”, “Intermediate−” and “Low” nutritional quality groups, respectively.

Modeling each observed diet into nutritionally adequate optimized diet

For each individual diet of the adult sample, a new theoretical optimized iso-caloric diet was designed by linear programming using an upgraded version of the previously described Individual Diet models (ID models) [18]. Optimized diets were kept as close as possible to observed diets while simultaneously meeting all nutrient recommendations.

Linear programming models are typically defined by a list of decision variables, an objective function, and a set of constraints. The decision variables are the amounts of foods available for diet modeling. Here, food variables were divided into “repertoire foods” (foods declared as consumed by the individual) and “non-repertoire foods” (foods declared as consumed at least once in the survey but not by this individual). The objective function was defined to find the optimized diet that came as close as possible to the corresponding observed diet while fulfilling the constraints. Regarding the introduction of non-repertoire foods, the foods most frequently eaten by the French population were preferentially selected, and were introduced in the lowest possible amounts.

The constraints to be fulfilled (Table 2) were a set of nutritional constraints based on WHO recommendations for proteins [19], free sugars [20], total carbohydrates, total lipids, saturated fatty acids (SFA), cholesterol and essential fatty acids [21], on the Nordic recommendations for sodium [22], and on the French recommendations for fibers, 10 vitamins and 9 minerals [23].

Table 2 Nutritional constraints achieved in all optimized diets (n = 1719)
Full size table

Statistical analysis

Distribution of portion sizes in the food database

After removing duplicates and excluding water, tea and coffee, 1055 foods were considered and their portion-size distribution according to SENS class was graphically displayed using box-plots. The Kruskal–Wallis test was used to statistically compare distributions across SENS classes. Average energy density (kcal/100 g) of the foods was calculated by SENS class.

Frequency assessment

For each observed diet and each optimized diet, daily frequency (portions/day) of each food was assessed by dividing total weight (in g/d) declared as consumed or optimized by the corresponding portion size. For each individual, the cumulative daily frequency of each class was estimated as the sum of daily frequencies of food from that class.

Validation of food ranking based on observed diets

Contributions of the four SENS classes to dietary energy were estimated and statistically compared between the four observed-diet groups of increasing overall nutritional quality. The hypothesis was that contribution from foods from Class-1and Class-2 would increase (with foods from Class-1 increasing more than Class-2 foods) while contribution from foods from Class-3 and Class-4 would decrease (with foods from Class-4 decreasing more than Class-3 foods) with increasing overall nutritional quality of the diet. A similar analysis was conducted using the daily frequency of foods from each SENS class. Generalized Linear Models (GLM) adjusted for energy intake, age and gender were used to estimate P value and P for trend.

Validation of food ranking based on optimized diets

SENS classes were statistically compared among observed and optimized diets according to per-class contribution to dietary energy and daily frequency of consumption. A trend test was performed among optimized diets with the hypothesis that contributions and daily frequencies of SENS classes would decrease from most favorable to least favorable class.

The average variation from the observed to optimized daily frequency of foods from each SENS class was estimated and compared to zero using GLM adjusted for age and gender. The hypothesis on achievement of nutritional adequacy was that the optimization process would increase foods from Class-1 more than Class-2 and decrease foods from Class-4 more than Class-3. The linear trend of these variations was statistically tested. In addition, for each SENS class, the percentage of optimized diets showing an increase (or decrease) in daily frequency of foods compared to the corresponding observed diet was calculated.

All comparisons and linear trends between SENS classes were tested using a general linear mixed model. Each statistical model included a fixed effect for SENS classes, age and gender. A random effect for subjects was included by using an unstructured covariance matrix which accounted for the presence of the four SENS classes in each individual diet.

For all statistical tests an alpha level of 5% was used for significance. SAS software version 9.4 was used.

Results

Distribution of portion sizes of foods from each SENS class

Figure 1 shows the distribution of portion sizes (g/d) of CIQUAL foods (excluding water, tea, coffee and duplicates, n = 1055) by SENS class.

Fig. 1
figure 1

Distribution of portion sizes (g/d) of CIQUAL foods (excluding water, tea, coffee, and duplicates, n = 1055) by SENS class

Full size image

Class-1 and Class-2 foods had higher portion-sizes (median 100 g) than Class-3 and Class-4 foods (median 50 g), which is consistent with their lower energy density (on average 89 kcal/100 g in Class-1 and 170 kcal/100 g in Class-2 vs 271 kcal/100 g in Class-3 and 357 kcal/100 g in Class-4) (data not shown). There was high variability in portion-sizes within Class-2 and Class-3 (interquartile range of 70 g) compared to Class-1 and Class-4 (interquartile ranges of 20 g and 50 g, respectively).

Validation of food ranking based on observed diets

In observed diets, the energy contribution (Fig. 2a) and the daily frequency of consumption (Fig. 2b) of Class-1 foods were significantly higher when the overall nutritional quality of observed diets was higher. Conversely, the energy contribution (Fig. 2a) and the daily frequency of consumption (Fig. 2b) of Class-4 foods were significantly lower when the overall nutritional quality of observed diets was lower. Energy contribution and daily frequency of Class-2 and Class-3 foods followed the same gradient (p < 0.001), but in a lower amplitude, than of Class-1 and Class-4 foods, respectively.

Fig. 2
figure 2

Energy contribution (a) and consumption frequency (b) of foods by SENS class according to diet quality levels

Full size image

Validation of food ranking based on optimized diets

Energy contribution and cumulative daily frequency of each SENS class in observed vs optimized diets

In observed diets, the average contribution to dietary energy (Fig. 3a) and the cumulative daily frequency (Fig. 3b) were significantly different between SENS classes but without trend tendency. The lowest average contribution to dietary energy came from Class-1 foods in the observed diets (20.6%) but conversely from Class-4 (16.0%), after optimization. In optimized diets, cumulative daily frequency was significantly decreasing (p for trend <0.01) by increasing SENS classes (9.1, 6.5, 4.2, and 2.7 portions/day for Class-1, Class-2, Class-3, and Class-4 respectively). Thus, for both energy contribution and cumulative daily frequency, a decreasing trend was statistically found in the optimized (but not the observed) diets.

Fig. 3
figure 3

Energy contribution (a) and consumption frequency (b) of foods by SENS class in observed vs optimized diets

Full size image

Variation from the observed to optimized cumulative daily frequencies of each SENS class

The linear trend test on observed-optimized variations across SENS classes indicated that the optimization process increased foods from Class-1 more than Class-2 foods (+3.16 vs. +0.41 portions/d), and decreased foods from Class-4 more than Class-3 foods (−2.17 vs. −0.55 portions/d) (Table 3).

Table 3 For each SENS class, mean variations of daily frequencies (in portions/day) from observed to corresponding optimized diets and percentage of optimized diets (N = 1719) with an increase or a decrease in daily frequencies compared to observed diet
Full size table

Almost all optimized diets showed an increase in daily frequency of Class-1 foods (for 98.4% of diets) and a decrease of Class-4 foods (for 94.2% of diets), suggesting a strong relationship between the variation in daily frequencies of foods and nutritional adequacy of the diets (Table 3).

Discussion

This study used WHO guidelines for validating nutrient profile systems to assess the ability of the SENS to adequately rank individual foods. First, based on the analysis of observed diet, the results showed that all SENS classes were ranked from least favorable (Class-4) to most favorable (Class-1) in accordance with their contribution to diets of increasing overall nutritional quality. Then, using a diet optimization approach, the results showed that the dietary changes needed to reach nutritional adequacy were in agreement with the SENS ranking of individual foods. Foods were therefore adequately ranked by SENS, as the algorithm was able to correctly classify them in relation to their contribution to the nutritional quality of both observed and optimized diets.

As there is no gold standard for defining a healthy food, WHO recommends several methods for validating nutrient profiling systems [2]. One simple method described by WHO is to identify a small number of ‘indicator’ foods as either “healthy” or “unhealthy”, notably by reference to food-based dietary guidelines (FBDGs), and to assess whether classifications using the nutrient profiling system agree with the predetermined classifications of these foods. We had previously applied this simple method to establish the consistency of SENS with official FBDGs in Europe favouring nutrient-rich foods and limiting high-fat/sugar/salt foods [4, 24] .

WHO also describes more refined validation methods involving testing “whether healthy foods (as identified by the nutrient profile model) make healthy diets (as defined by an independent, preferably validated dietary quality index) and whether unhealthy foods make unhealthy diets” [2]. Arambepola et al. (2008) examined the relationship between the way the British Food Standard Agency (FSA)-Ofcom model categorized foods against healthiness of observed UK diets characterized according to the Diet Quality Index (DQI) [25]. Our results are in line with their findings of greater amounts of energy coming from ‘less healthy’ foods in the least healthy diets compared to the healthier diets. However, unlike our findings of increased contribution of food from the most favorable SENS Class-1 with increasing diet quality level, the contribution of healthier foods according to the FSA-Ofcom model did not differ across diet quality levels [25].

Studies have gone beyond the healthy-unhealthy dichotomy. In the US, Fulgoni et al. (2009) found that diet indices based on a family of nutrient-rich food (NRF) scores were significantly related to the Healthy Eating Index (HEI), an independent measure of a healthy diet [26]. In France, Julia developed the “FSA nutrient profiling system dietary index”, an aggregation of the FSA-Ofcom values of the foods consumed by a given individual [27, 28]. A more favorable “FSA nutrient profiling system dietary index” was characterized by higher amounts of fruit, vegetables and fish and lower amounts of sugary and savory snack foods [27] and associated with higher intakes of beneficial nutrients and higher adherence to French nutritional guidelines [28]. Both Fulgoni et al. [26] and Julia et al. [27, 28] used a diet index based on the sum of scores of each consumed food weighted by consumed quantities, which implies they did actually test diet rankability—not food rankability. Considering that nutrient profiling systems are intended to synthesize the nutrient quality of individual foods, applying a food profiling system to a whole diet is probably not the most appropriate method for validating the food ranking capacity of the system.

A major strength of the present study is that we were able to analyze the capacity of a nutrient profiling system, the SENS, to appropriately rank foods within classes according to their contribution to overall nutritional quality of diets. Not only were foods from the most favorable nutrient profiling class shown to contribute more to higher-quality diets and thus warrant increased intake to reach nutritional adequacy, but the expected hierarchy of the four SENS classes was actually validated. Moreover, the present analysis at food level (and not using an overall diet score derived from the nutrient profiling system) allowed to assess and confirm the relevance of the ranking in terms of food frequency. Daily food frequency (expressed as number of portions per day) is rarely used in validating a nutrient profiling system, but we believe that it is important to address portion sizes and daily food frequencies, given how both are needed to assess the food’s potential to affect overall dietary balance.

Substitution scenarios are also useful for validating nutrient profiling systems. Roodenburg et al. (2013) used a substitution scenario to evaluate the nutritional criteria of the Choices Program nutrient profiling system, and found that replacing non-compliant foods by Choices-compliant products would shift the nutrient intake distributions closer towards recommended intakes [29]. However, diet quality was sometimes only partially improved after substitutions and there was a need for validation studies against diets that actually meet a whole set of recommendations. Because nutritional adequacy is achievable with diet modeling using linear programming it has been proposed for validating nutrient profiling systems [2]. More specifically, linear programming has been used to test the capacity of the SAIN,LIM system [3] and other nutrient profiling systems [30] to design nutritionally adequate diets using only foods with a favorable profile. However, these modeled diets were unrelated to observed population intakes and thus remain highly theoretical. To address this gap, an individual diet optimization approach ensuring nutritional adequacy but also integrating individual food preferences was applied to test SAIN,LIM [31]. Dietary changes needed to reach nutritional adequacy were in accordance with the SAIN,LIM classification, with foods from the most favorable class increasing the most. Nevertheless, the hierarchy of classes was not formally tested [31], as we did here with the four SENS classes.

This study has limitations. Modeling studies are always open to criticism that they remain strongly theoretical [3]. Here, the optimization process integrated individual food preferences, which increases the possibility that the optimized diets may translate into real-world consumption, even though some quantitative changes might be difficult to follow. In addition, this criticism does not apply to the part of our results based on observed data from a national dietary survey. We used dietary data from generic foods that are not the best-adapted source for evaluating foods as purchased by consumers in real-life settings. Also, the INCA2-survey dietary data is now almost ten years old but was still, in mid-2017, the most recent open dataset available for the French population. Another limit with the approach presently used to validate the nutrient profiling algorithm (i.e., testing whether healthy foods make healthy diets) is some circularity in the argument, because healthy foods and healthy diets share common properties (such as having a low content of saturated fatty acids). This was acknowledged in the WHO report on nutrient profiling [2]. It is true that all the nutrients used in the SENS algorithm were also used to define the nutritional quality of observed diets and the nutritional adequacy of optimized diets. However, SENS is based on few key nutrients only (3 disqualifying nutrients and a maximum of 4 qualifying ones for each food) whereas nutritional adequacy of optimized diets was defined by more than 40 nutrients. In addition, even for those nutrients shared between the SENS algorithm and the nutritional constraints included in the models, the reference values used in the diet models were not the same as those used in the SENS algorithm, and they were different from one individual to another, because the constraints were individualized depending on observed nutrient intakes. Nevertheless, testing whether consuming foods defined as healthy according to the nutrient profile algorithm is able to protect against health outcomes provides a more reliable external reference [32, 33]. In France, the “FSA nutrient profiling system dietary index” was found to be associated with weight and BMI gain [34], risk of developing metabolic syndrome [35], cardiovascular disease risk [36], overall cancer risk [37] and antioxidant status [28] using data from the SUpplémentation en VItamines et Minéraux AntioXydants cohort (1994-2007). However, longitudinal data on dietary intakes and health indicators of a French cohort were not publicly available at the time of this study, making it difficult to conduct health impact studies. Even if we did not use health measures, our results showing a decrease in the least-favorable foods (Class-4) for 95.2% of individuals is totally aligned with the global recommendation to reduce the intake of energy-dense foods to prevent excessive weight and obesity [38, 39]. Finally, the daily frequencies of foods used here had to be based on portion-size references mainly from France but including some references from other countries, as the EU lacks a harmonized standard definition of portion size. The development of a set of accredited portion-size references, such as the reference amounts customarily consumed (RACC) servings in the USA [40], would be highly valuable to make greater use of the food frequency dimension, as portion-based profiles may represent an attractive option for consumers.

Conclusion

The SENS algorithm was validated according to WHO guidelines. It is able to adequately rank foods according to their contribution to the overall nutritional quality of diets, a pre-requisite to its use for simplified nutritional labeling in Europe.