Reliability and validity of a General Nutrition Knowledge Questionnaire for adults in a Romanian population

Background Nutritional knowledge assessment is an important component in nutrition research, and a prerequisite for the implementation of many policies and programs aimed at improving eating behavior. In order to generate objective results, validated tools for a given population must be employed. The aim of this study was to determine the validity and reliability of a nutrition knowledge questionnaire for Romanian adults. Methods Kleimann’s version of a General Nutrition Knowledge Questionnaire, was translated and adapted to Romanian language, culture, and cuisine. The final format was developed in several steps and used four components: internal and external reliability were assessed in a general population sample (n1 = 412), respectively in a subgroup (n2 = 46) from Component 1; Component 3 assessed construct validity (n3 = 96) using the “known-groups” method; Component 4 (convergent validity, n4 = 508) tested the association between socio-demographic characteristics and nutrition knowledge. Results The overall internal reliability was 0.878 and the external reliability was >0.880 in all sections, and overall. Specialists had higher scores than nonspecialists, with a very large effect size. In the general population, females scored higher than males, and middle-aged and older adults scored higher than young adults. Higher scores were associated with higher levels of education. The characteristics of individuals prone to giving wrong answers were: males (beta = 0.170), high school or less (beta = 0.167), and no training in nutrition (beta = 0.154). Conclusions The Romanian version of the General Nutrition Knowledge Questionnaire is a reliable and valid tool for measuring nutrition knowledge in adults.


Introduction
There is a very young history for nutrition and dietetics professionals in Romania, since this profession was first recognized only in 2015 [1], and the application norms of the Law of Dietitian were published early in 2019 [2]. Until the legal recognition of the profession of dietitian, the role of nutritional management related to food choices, dieting, and special diets for medical conditions, was performed by medical doctors in different medical specialties such as family doctors or specialists in diabetes, nutrition, and metabolic diseases [3].
In several studies on different populations, it was shown that higher nutrition knowledge was associated with an adoption of healthier lifestyles and food choices [4,5], albeit this association is still controversial or has limited value [6]. Other factors, including social-economic status of the family and traditions, taste preferences, and genetics, are strong drivers that influence individual dietary habits [6][7][8]. Previous research indicated that the food choices of Romanians are largely driven by economic reasons [9].
Currently, there are no validated tools to measure nutrition knowledge of the general adult public in Romania. The purpose of this study was to determine the validity and reliability of a General Nutrition Knowledge Questionnaire in Romanian population, in order for it to be further used in subsequent research.

Materials and methods
The most known questionnaire of nutrition knowledge is Parmenter's work [10], developed in 1990s and validated in English, and which is used as starting point for generating nutrition knowledge questionnaires in different languages [11][12][13][14] or for different purposes [15][16][17], ever since it was published. More recently, Kliemann et al. [18]  Step 1. The original questionnaire was translated to Romanian and then back to English by two independent translators. The English original version was compared with the backward translated version, with corresponding minor corrections being made in the Romanian version.

•
Step 2. A panel of three experts, with medical background and expertise in human nutrition and dietetics, nutritional epidemiology, and nutrigenomics/ nutrigenetics, critically reviewed each question from the Kliemann version for usability and adaptability to Romanian culture. Following this step, all questions were kept but some specific English foods were replaced with foods known and utilized in Romania.

•
Step 3. Pretesting of questionnaire was performed on 25 volunteers, working in panel. The main point of pretesting was to provide a clear and easy understanding for each question. The outcome of the pretesting panel consisted in minor improvements in the wording of some questions.

•
Step 4. A general population sample of 412 adults was used for determining the internal reliability of the questionnaire per each section and per total (Component 1).

•
Step 5. External reliability of the questionnaire was assessed with a subgroup of 46 adults from general sample population, who were retested after at least 30 days after the first round (Component 2).

•
Step 6. Construct validity was assessed with 48 students/ specialists in dietetics (last year undergraduates, master students, and recent-up to 3 years-master graduates) and with 48 mathematics/informatics undergraduate students, using the "known-groups" method (Component 3). The sample size was calculated in order to assure a large effect size, with a power of at least 80%, using one-sided tests.  Step 7. Combined participants in steps 4 and 6 were used for the assessment of convergent validity that explored associations between nutrition and dietetics knowledge and social and demographic characteristics (Component 4). The sample size provided by this component is ensuring a power of at least 80%, with a margin of error of 5%.
The questionnaire contained, besides specific nutritional knowledge questions, demographic questions such as: gender, self-evaluation of health status (with five possible ordinal categories from excellent to poor), marital status (with six possible nominal categories), number of children (ordinal with five categories, from zero to four or more children), presence of minors in household (dichotomic, yes/no), highest education level (ordinal, with eight possible answers from gymnasium school or less to PhD), if they had training in nutrition (dichotomic, yes/no), and a self-evaluation of nutritional status by self-reported weight and height.

Ethics approval
The Institutional Review Board at the Victor Babes University of Medicine and Pharmacy Timisoara, Romania approved this research protocol. All subjects included in the study provided informed consent before participation.

Data management and data analysis
Up to step 4 the paper-based version of the questionnaire was used. For the following steps an online questionnaire was employed. Since the possibility to enter bias by using the internet or communicate with others in order to get the right answers was big if posted freely online, the researchers invited individuals or groups, and opened the questionnaire then closed it after the completion was done. At least one member of the research team was always present in order to limit the access to sources of information and communication between participants, if approached in groups.
Databases were exported and IBM-SPSS version 21 was used to process the data. Scores for questions were calculated in two ways [17]: • Type A scoring system: for each correct answer one point was added. For wrong answers or if the responder chose "I do not know" option, no points were added.
• Type B scoring system: for each correct answer one point was added, but for wrong answers one point was subtracted. If the responder chose "I do not know" option, no points were added.
Type B of score computing was used for Component 4, while Type A was used in Components 2, 3, and 4.
For each section and for the entire questionnaire the achievement score was computed, taking into account the score obtained by each scoring system and the maximum score that could be obtained by sections and for the entire questionnaire. For the fourth component, the difference between type A and type B scoring was computed for the purpose of identifying those individuals who gave wrong answers. For demographical questions, some categories with few responses were collapsed, and in order to facilitate the better understanding of the meaning of such questions.
Body mass index was calculated using self-reported height and weight, and participants further classified into normal or overweight and obese.
Continuous data are presented as mean ± standard deviation or median (interquartile range (IQR)). Categorical data are presented as percentages. For comparison of proportions chi-square test was used. For comparing parametric data on a 2-category factor, t-test was used with appropriate decision for equality of variances with Levene's test, and when comparing nonparametric or ordinal data, the Mann-Whitney test was used. For three or more groups with data that did not assume normality, Kruskal-Wallis test was the choice. Bonferroni correction was applied for several comparisons following significant Kruskal-Wallis test. Paired comparisons between two different scoring systems were assessed with Wilcoxon signed-ranks test. A linear regression model was built, with the difference between type A and type B scoring system as a dependent variable, and demographic factors as independent variables.

Internal and external reliability
Internal reliability, which measures the degree of correlation between different items of the questionnaire, was assessed on a general population sample (n = 412) by Cronbach's alfa. Per the entire questionnaire, it had an overall alpha value of 0.879, larger than the threshold of 0.7. Sections 1 and 3 had an individual alfa lower than the threshold, and sections 2 and 4 had larger Cronbach's alfa values than the threshold (Table 1).
External consistency (or test-retest) was assessed on a subgroup of the population sample (n = 46). The interclass correlation coefficient was higher than 0.85 for all sections and overall ( Table 2).

Assessment of construct validity
For the construct validity, the two groups considered further (specialists and nonspecialists) were equal in size, n = 48/ group. The distribution of gender was similar, but specialists who had a higher age, a lower BMI, were more likely to be in a relationship and have children, and considered to have a better health status as compared with nonspecialists (Table 3).
Mean ± SD and medians (IQR) per sections and per total in categories of specialists and nonspecialists are presented in Table 4. With a very large effect size, the overall score and per sections scores were significantly higher for specialists when compared with nonspecialists ( Table 4). Table 5 presented the knowledge scores per sections in a sample of 508 responders by different demographic characteristics of the sample, using two different methods of score calculation, with a total possible score of 88 points.

Assessment of convergent validity
Type B scoring had significant lower median achievement scores in all sections and for the entire questionnaire, as compared with type A (p < 0.001). Table 6 contains demographic significant predictors of high difference between the two scoring systems, when controlling for age, BMI, marital status, health perception, and the presence of children. Males (2.6 times more likely), lower level of education (2.3 times more likely), and no previous studies in nutrition (3.2 times more likely) were the independent variables that contributed significantly to the model.  [10] Australian sample (n = 156) [19] Turkish sample (n = 195) [13] Uganda sample (n = 117) [12] UK sample (n = 266) [18] Japanese sample (n = 1182) [

Discussion
Since currently there is no validated tool to collect general nutrition knowledge in Romanian adults, the present study aimed to validate and adapt to Romanian language the General Nutrition Knowledge Questionnaire updated recently by Kliemann et al. [18] and developed by Parmenter and Wardle [10], which is exploring general nutrition knowledge with the following sections: dietary recommendation; food groups; healthy food choices; and diet, disease, and weight associations. The original version of the questionnaire is in English, but besides translation, in Romanian version, less common dishes/foods were replaced with similar, more common foods present in Romanian eating pattern/cuisine. The results from sections and for the entire questionnaire were used to assess overall internal reliability, external reliability, construct validity, and convergent validity of the questionnaire (Tables 1-4) These criteria were adequate and comparable to other published studies, as presented in    Tables 1 and 2 [11,13,[18][19][20][21][22]. Lower internal reliability for sections 1 (dietary recommendations) and 3 (healthy food choices) were explained by others due to the high heterogeneity in background of responders and possibly by different interpretations of the nutritional recommendations in the absence of adequate training [12,23]. The scores performed by specialists who had several training courses in nutrition were, with a very large effect size, significantly higher than those of nonspecialists, who did not have training in nutrition (Table 3), and were comparable to other similar studies [10,11,13,18]. These results validated the construct validity.
Overall achievement score was low, with its median at 65.9% (Table 4). Lower scores were reported for dietary recommendations (median A vs. B scoring system 66.7% vs. 38,8%) and food groups (median A vs. B scoring system 63.8% vs. 36.1%). Higher scores were reported for healthy food choices (median A vs. B scoring system 76.9% vs. 53,8%), disease and weight associations with diet (median A vs. B scoring system 71.4% vs. 47,6%), and overall achievement scores of 65.9% and 40.9% in type A, respective type B scoring system. In a previous study performed in the 90s in England, Parmenter had found much lower scores in nutrition knowledge than the maximum that could be achieved, when using the tool in a general population [24]. Our results indicated low theoretical background in the Romanian adult population.
Previous studies [10,19,25,26] have reported that women tend to have higher knowledge scores as compared with men. In the sample analyzed in the fourth component (Table 4), women had more precision in nutrition knowledge and therefore had higher scores only in type B scoring system, as compared with men.
Significant differences were assessed in our sample between middle-aged adults and seniors, when compared with young adults. Higher knowledge in middle-aged and older adults could be a result of life experience (Table 4). Other recent studies performed in countries with better nutrition curricula in schools showed similarity in knowledge between young adults who benefited from a good nutrition curricula, and the older adults (due to their life experience) [18].
In our study, the higher education status was associated with better scores in nutrition knowledge, as already indicated previously [24,25]. Non-single status, the presence of children in families, and the presence of minors in families were items associated with higher knowledge, since the presence of a child often raises awareness about healthy lifestyles, including nutrition [24,27,28].
Although each question had the option "not sure" and responders were advised to choose that option over guessing, several classes of responders chose to force an answer and for some, the answer was incorrect. The two scoring systems assessed different aspects of the nutrition knowledge: if type A was a positive score, type B scoring was focused on wrong answers. Using their difference, we were able to identify and quantify those individuals who were more likely to give wrong answers. Demographic predictors of giving a wrong answer were male gender, high school or lower education, and no studies in nutrition. These data support the results of univariate analysis and previously published studies [11,18,24,29,30].
When comparing the present study with other similar studies (Tables 1 and 2), internal and external reliability coefficients for each section were, in general, similar. When compared with other studies, the present study had the highest correlation coefficients for external reliability sections 1-3 (Table 2), while having the lowest coefficients for internal reliability sections 2, 4, and overall (Table 1).
This study has some limitations. First of all, it is already known [31] that selection bias could exist for individuals willing to participate in a questionnaire targeting nutrition knowledge. These individuals were more likely to be educated, to have a genuine interest in healthy lifestyle and to have a better health status. The income was not assessed, and therefore no generalization can be made over different economical strata. Another limitation consists in the geographical and cultural representation of the selected participants (western Romania). The Romanian version of Nutrition Knowledge Questionnaire is the first validated tool designed to collect general nutrition knowledge in Romanian adults, to the best of our knowledge. The questionnaire overall, and its sections, had adequate construct and convergent validity, internal reliability, and external reliability, which make it a valuable tool in assessing nutritional knowledge.
Author contributions SP, CLS, and MDN designed the research; SP, SP, LCM, AMB, and DMS designed the online tool and collected the questionnaires; DMS performed data management; CLS analyzed the data; CLS and AMB drafted the manuscript; SP and MDN critically reviewed the manuscript. All authors agreed the final version of the manuscript.

Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of interest.
Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons. org/licenses/by/4.0/.