To evaluate validity and reliability of the food-frequency questionnaire (FFQ) developed for the Korean Genome Epidemiologic Study (KoGES).
FFQ was administered twice at 1-year interval (first FFQ (FFQ1) at the beginning and second FFQ (FFQ2) at the end of the study) and diet records (DRs) were collected for 3 days during each of the four seasons from December 2002 to May 2004 for those who attended the health examination center. At the end of the study period, we collected the 12-day DRs of 124 participants. The nutrient intakes from the DRs were compared with both FFQ1 and FFQ2.
The intakes of energy and some nutrients estimated from FFQ1 and FFQ2 were different from those assessed by the DRs. Especially, the consumption of carbohydrates was higher in FFQ1 and FFQ2 than in the DRs. The de-attenuated, age, sex and energy intake adjusted correlation coefficients between the FFQ2 and the 12-day DRs in Korean population ranged between 0.23 (Vitamin A) and 0.64 (carbohydrate). The median for all nutrients was 0.39. The correlations were similar when we compared nutrient densities of both methods. Joint classification of calorie-adjusted nutrient intakes assessed by FFQ2 and 12-day DRs by quartile ranged from 25.8% (vitamin A) to 39.5% (carbohydrate, iron) for exact concordance. Except vitamin A, the proportion of subjects classified into distant quartile was less than 7% in all nutrients. The median of correlations between the two FFQs 1 year apart were 0.45 for all nutrient intakes and 0.39 for nutrient densities.
We conclude that the FFQ we have developed appears to be an acceptable tool for assessing the nutrient intakes in this population. Further studies for calibration of the FFQ collected from multicenters participating in the KoGES are needed.
This study was supported by the budget of the National Genome Research Institute, Korea National Institute of Health (2002-347-6111-221).
Dietary factors are one of the most important environmental effectors on the development of chronic diseases. The food-frequency questionnaire (FFQ) is a tool for the estimation of food and nutrient consumption and has been widely used in investigating the association between diet and chronic diseases in large population-based studies (Willett et al., 1985; Pietinen et al., 1988). This method has an advantage in that a long-term, usual dietary intake of an individual can be easily obtained by making a single measurement. As there is, however, great variability in the foods consumed and dietary culture among diverse populations, an FFQ which is valid for one population may not be valid when applied to a second population (Willett and Lenart, 1998). For this reason, a validation study is required whenever an FFQ is developed and is applied to a new population. Although there is no perfect measure of dietary intake, multiple days of diet records (DRs) over four seasons may be superior to FFQ and have been frequently used as the reference method in many validation studies (Willett et al., 1985). DRs, which is not dependent on recall of food items like FFQ, appear to be a suitable method for validating the performance of FFQ in that errors of these two methods being compared are not correlated (Willett and Lenart, 1998).
Large population-based cohort studies, the Korean Genome Epidemiologic Study (KoGES), have been initiated since May 2001 in Korea. The main objectives of this study were to collect information on lifestyle factors including dietary habit and various kinds of biomarkers including DNA through repeated survey every 2 years and to see the effects of genes, preclinical markers, environment and their interaction on the occurrence of chronic diseases, such as cardiovascular diseases and diabetes mellitus, which are accounting for more than half of all cause mortality in Korea. KoGES recruits study participants living in the local community or visiting a clinic for health screening. As of December 2005, this on-going study has recruited 40 000 participants aged between 40 and 69 years old from 40 centers scattered nationwide.
To assess usual dietary intakes of Korean adults who are participating in this cohort, an FFQ was developed based on the large collection of data from a nationwide nutrition survey (Ahn et al., 2003). For validation of this FFQ, we collected information using 3-day DRs as a reference method during each of the four seasons over a period of 1 year, totaling 12 days for 124 participants. We report here the validity and 1-year reproducibility of the FFQ being used in the large cohort study in Korea.
Subjects and methods
Subjects for the validation study were recruited from participants in the Health Examination Center at Hallym University Sacred Heart Hospital (Anyang, Korea), which is one of the participating centers for KoGES, from December 2002 to May 2004. A total of 199 subjects, aged between 40 and 70 years old, consented to join the study and completed the first FFQ (FFQ1) and the first DR. Among them, 130 subjects finished the 3-day DRs over four seasons and the second FFQ (FFQ2). A total of 124 participants were included in the final analyses after excluding six people who changed their diet for weight reduction during the study period. The study design is shown in Figure 1.
We developed a 103-item FFQ to assess the usual dietary intakes of Korean adults. The procedure to design the FFQ is described in detail elsewhere (Ahn et al., 2003). Briefly, the dietary data for designing the FFQ were obtained from the Korea Health and Nutrition Examination Survey (KHANES) in 1998, which is a nationwide dietary survey for the representative sample of the whole population in Korea. A subset of data on informative food items was collected using the 24-h recall method with 2714 adults aged 40 or older living in a middle-sized city or in rural areas in Korea. The cumulative percent contribution of each food and the cumulative R2 of multiple regression of each nutrient were considered when choosing food items for the FFQ. Two hundred and forty-nine food items, which were selected based on their 0.9 cumulative percent contribution, and 254 items, which were selected based on their 0.9 cumulative R2, respectively, were grouped into 97 food groups according to their nutrient contents. Several popular Korean foods, which were missing from the list due to the seasonality of the survey, were included. Finally, 103 food items were included in the questionnaire as follows: rice and other cereals (7), noodles and breads (10), vegetables (23), potatoes (3), mushrooms (2), soybean, soybean products and other beans (4), common fish (7), other fish and shellfish (8), meats (9), seaweeds (2), eggs (1), milk and dairy products (5), fruits (12), beverages (5), snacks (3), nuts (1) and fats (1). The frequency of servings was classified into nine categories: never or seldom, once a month, 2–3 times a month, one to two times a week, three to four times a week, five to six times a week, once a day, twice a day or three times or more every day. For the food items with different seasonal availability, we requested the participants to mark one on how long they ate among four categories: 3, 6, 9 and 12 months. The portion size was determined depending on the median value of each food determined from the 24-h recall data obtained from the KHANES. The portion size of each food item was classified as follows: small, medium or large. For easy understanding of portion size, we provided pictures on serving size for food items on their own pages. We asked separate questions about portion size for all food item(s) listed in this FFQ. The same FFQ was administered twice, at the beginning and the end of the study, to assess the reproducibility of the FFQ by two trained dietitian interviewers.
As a reference method to assess the validity of this FFQ, we collected information on 12-day DRs for 1 year on each participant. The participants were asked to keep non-consecutive 3-day DRs and to include 1 weekend day or holiday during each of the four seasons to capture the seasonal and day of week variation of food intake. Eighty five percent of total participants adhered precisely to the recording days. To minimize the errors of portion size, the subjects were asked to record the intake amount with multiples of household tableware. Starting this study, a research dietitian supervisor (J Shim) had prepared a protocol for coding DRs. Based on the protocol, trained dietitian interviewers collected the DRs, reviewed unclear descriptions, errors, omissions or doubtful entries, and asked the participants to clarify them. All completed records were checked by a research dietitian supervisor for accuracy.
Nutrient intake of each food item was converted based on the weight derived from the consumption frequency and the portion size in each food item. Daily nutrient intakes of each individual were the sum of the nutrient intake of each food item, which were calculated using DS 24 (Human Nutrition Lab, Seoul National University & AI/DB Lab., Sookmyung Women's University, 1996). The food composition table used in the two calculations was the seventh edition Food Composition Table of Korea (The Korean Nutrition Society, 2000).
The nutrient intakes from the DRs were compared with both FFQ1 and FFQ2. Differences of each comparison were presented as a percentage of the consumption from the DR and were tested by use of paired t-tests. Correlations of nutrient intakes between the two methods, DRs and FFQ, were assessed by Pearson's correlation coefficients. The root or natural log was applied to improve normality of the distribution of the nutrient intake. To correct the within-person error in the measurement of the DRs, the observed correlation was multiplied by the de-attenuation factor (1+γ/n)½, where γ is the ratio of the within- and between-person variances and n is the number of repeats (here n=12). Within- and between-person variances were calculated using SAS Varcomp procedure (Ver. 8.2 SAS Institute Inc., Cary, NC, USA). These analyses were also carried out for nutrient densities (nutrient/energy).
To measure the degree of agreement, subjects were classified into quartiles based on the nutrient intakes from the two methods and the percentages of agreement and complete disagreement were presented. The reproducibility of two FFQs given 1 year apart was presented as Pearson's correlation coefficients.
The total number of subjects included in the study was 124 (male subjects 28%, female subjects 72%). The mean age of the subjects was 48.8 years old and the body mass index was 23.4. Current drinkers and current smokers were 41.1 and 12.9% of the total subjects, respectively (Table 1).
The mean nutrient intakes of the 12-day DRs, FFQ at baseline and FFQ at the end of the study, are presented in Table 2. The absolute values estimated by the two methods (DRs and FFQs) were different for some nutrients. Energy and carbohydrates in DRs were reported as lower than those in both FFQs. For other nutrients like protein, fat, vitamin A, vitamin B6, folate, retinol, carotene, fiber and cholesterol, estimated consumptions through FFQ2 were lower than those through the DRs. Between the two FFQs, protein, fat, carbohydrates, zinc, vitamin B6, folate, retinol, carotene, fiber and cholesterol, the estimated consumptions were different.
Crude Pearson's correlation coefficients of nutrients were 0.23 (Vitamin B6) to 0.40 (Vitamin C) between FFQ1 and the DRs, and 0.10 (Vitamin A) to 0.46 (retinol) between FFQ2 and the DRs. For FFQ1, the age, sex and energy intake adjusted correlation coefficients ranged between 0.17 (zinc) and 0.44 (phosphorus and folate) and also ranged between 0.25 (zinc) and 0.53 (phosphorus) in the de-attenuated correlation coefficients. For FFQ2, the age, sex and energy intake adjusted correlation coefficients ranged between 0.17 (zinc) and 0.54 (carbohydrate) and the de-attenuated correlation coefficients ranged from 0.23 (vitamin A) to 0.64 (carbohydrate). Medians for all nutrients were 0.41 and 0.39, respectively (FFQ1, FFQ2). De-attenuated correlation coefficients of some nutrients showed little improvement because the ratio of intra- to inter-individual variability was low (Table 3). Crude Pearson's correlation coefficients to nutrient density values were from 0.27 for cholesterol to 0.52 for phosphorus between the DRs and FFQ1 and from 0.16 for vitamin A to 0.53 for carbohydrate between the DRs and FFQ2. After adjusting for age, sex and energy, de-attenuated correlation coefficients were improved (Table 4).
Subjects were classified into quartiles by their nutrient intakes estimated from the 12-day dietary record and FFQ (Table 5). The proportion of classification into the same quartile ranged from 25.8% (vitamin A) to 39.5% (carbohydrate, iron). On the average, more than 70% of the subjects fell into the same or the adjacent categories. Except for vitamin A, the proportion of subjects classified into opposite quartiles was below 7% for all nutrients.
Pearson's correlation coefficients between the two FFQs administered at a 1-year interval are shown in Table 6. The correlation coefficients for nutrient intakes varied from 0.24 (carbohydrate) to 0.58 (cholesterol). The correlation coefficients for nutrient densities were lower than nutrient intakes. The averages of correlations were 0.45 for all nutrient intakes and 0.39 for nutrient densities.
In this study, we examined the validity and reproducibility of a 103-food item FFQ used for the KoGES. The de-attenuated, age, sex and energy intake adjusted correlation coefficients between the FFQ and the 12-day DRs in Korean population ranged between 0.23 and 0.64 (median for all nutrients 0.39). Cross-classification between these two methods was reasonably acceptable. The averages of correlations between the two FFQs 1 year apart were 0.45 for all nutrient intakes and 0.39 for nutrient densities.
Nutrients consumption estimated by the FFQ was lower than those from the DR for fat, vitamin A, β-carotene and cholesterol. The Korean diet is typically served as pre-seasoned dishes and various kinds of seasonings like soy sauce, soy paste, pepper paste, salt and seasoning oils were substantial sources of energy, sodium, iron and β-carotene (Paik, 1987; Kim and Kim, 1992; The Korean Nutrition Society, 2000). It was previously reported that the total nutrient intake, especially of fat, iron and β-carotene, could be underestimated when using only FFQ because seasonings and oils are not included in this method (Shim et al., 1997). Carbohydrate consumption assessed by the FFQ2 was higher than that of the DRs. It may be overestimated in the FFQ because rice is the staple food of Korea and grains and noodles were also included in the food items. An overestimation of carbohydrates was found in other reports of validation studies of Korean FFQs (Paik et al., 1995; Kim and Yang, 1998; Won and Kim, 2000; Lee et al., 2002; Shim et al., 2002).
The observed correlations of absolute nutrient intakes in this study appear to be lower than that reported in western countries. Willett noted that correlation coefficients of 0.5–0.7 were typically seen in recent validation studies (Willett and Lenart, 1998). In his initial validity study of a 61-item semiquantitative FFQ with 173 women participants, he reported that validities after energy-adjustment ranged from 0.28 (vitamin A without supplement) to 0.61 (cholesterol) between the 28 days DRs and FFQ1, and from 0.36 (vitamin A without supplement) to 0.75 (total vitamin C) between the DRs and FFQ2 (Willett et al., 1985). Several Latin American (Rodriguez et al., 2002; Bautista et al., 2005) and European (Johansson et al., 2002) studies also presented validities with a similar range of correlation of coefficient. In the Asian region, validation studies for Japanese cohort studies showed correlation coefficients of nutrient intakes ranged from 0.13 to 0.81, and their observed median was 0.3–0.4, which are similar to our results (Egami et al., 1999; Tsubono et al., 2001; Ogawa et al., 2003; Tsugane et al., 2003; Date et al., 2005). The Chinese study reported higher correlation coefficients (0.41–0.66) than the Japanese study in validity (Shu et al., 2004). In other Korean studies, they reported correlations ranging from 0.16 to 0.71, and on the average from 0.3 to 0.5 (Paik et al., 1995; Kim et al., 1996; Kim and Yang, 1998; Won and Kim, 2000; Choi et al., 2001; Lee et al., 2002; Shim et al., 2002).
The reasons for a lower correlation of the FFQ in Korea may be related to the unique way of serving and eating Korean foods. A Korean meal is typically comprised of a bowl of cooked rice with a pre-seasoned mixed soup and multiple side dishes. More often than not, meals are served family-style with multiple people sharing the side dishes. Therefore, Koreans may have low perception of portion size and have difficulties in answering the consumption frequency of a specific food item, especially which is not a prepared dish. Another reason for this lower correlation of our study compared with those of western studies may be due to lower inter-individual variations in nutrient intakes. Coefficients of variances (CVs) of between-individual variance in our data ranged 7.7–44.7% (mostly about 10–20%), and other Korean studies for college students reported also ranged 15–25% (Chung et al., 1992; Oh et al., 1996; Kwon et al., 2004). Japanese study reported a little bit higher coefficients of between-person variances (13.3–36.2%) than Korean's (Ogawa et al., 1999). However, between-person CVs of Beaton's report for the western diet ranged 17.2–74.9% (mostly 20–50%) for macro nutrients, which appears to be fairly higher than Korean's (Beaton et al., 1979).
Furthermore, this FFQ was developed based on individual food items, not on prepared dishes. The seasonings and cooking oils omitted in food-based FFQ might affect not only the difference of absolute intakes of some nutrients but also the correlation between a structured questionnaire (FFQ) and an open questionnaire for detailed information (DR). Although our food-based FFQ has relatively low validity, we could not choose a dish-based FFQ, because we had developed a data-based FFQ. The foods for this FFQ were selected on the basis of each food's contribution either to total dietary intake or to the between-person difference in dietary intakes from the nationwide data of the Korea Health and Nutrition Examination Survey (Ahn et al., 2003). Additionally, there was no evidence that a dish-based FFQ had more precision in assessing dietary intakes as we had no standard recipe or recipe data for the dishes. Food-based FFQs developed in Japan have similar problems. As in the case of the Korean diet, seasonings, such as shoyu, the Japanese soy sauce, are important dietary elements whose intake is difficult to measure in FFQs, although they only account for a small proportion of total dietary intake.
One would have expected better correlations between FFQ2 and DRs than between FFQ1 and DRs because of training effect (Kristal et al., 1997). Our study, however, did not show the expected results. Some previous studies have shown that a second FFQ produced lower nutrient estimates and the reason for this observation has not yet been elucidated (Subar et al., 2001). Compared with the correlation coefficients based on absolute nutrient value, the correlation coefficients of nutrient densities was higher than those of absolute intakes. Stram et al. (2000) stated that nutrient densities generally have better correlations than do absolute values of nutrients.
Though correlation coefficients between the two methods were relatively low, joint classification showed better results. In this study, the average exact agreement proportion of quartile classification between the two assessments was about 34% and more than 70% of the subjects were classified within the same or adjacent quartiles. Other validation studies in Korea with 24-h recalls and FFQs showed average exact agreement proportions of 28%, which was lower than our results (Paik et al., 1995). A Japanese study reported a relatively high exact agreement proportion (35–56%) and an adjacent agreement proportion was over 80% on the average (Ogawa et al., 2003).
Reproducibility of our questionnaire ranged from 0.24 (carbohydrate) to 0.58 (cholesterol). This result was comparable to other studies in macronutrients (Willett et al., 1985; Giovannucci et al., 1992; Kabagambe et al., 2001; Johansson et al., 2002; Ogawa et al., 2003; Sasaki et al., 2003; Shu et al., 2004). The Korean report showed reproducibility from 0.37 to 0.60 (Kim et al., 1996).
In this study, we presented the results of validation and reproducibility of FFQs for the KoGES. The FFQs have the advantage of being cost effective and of being quickly administered in a large-scale epidemiological study. The overall performance of our FFQs appears to make them reasonably acceptable to apply to a large cohort study in the Korean population. Further studies are needed to determine the calibration of the FFQs collected from multicenters participation in KoGES.
This study was supported by the budget of the National Genome Research Institute, Korea National Institute of Health (2002-347-6111-221). We thank Dr Willett for his invaluable comments on our paper and also thank Ms Min Haesook for statistical advice.