Accurate measurement of dietary intake is essential for understanding the long-term effects of adolescent diet on chronic disease risk. However, adolescents may have limited food knowledge and ability to quantify portion sizes and recall dietary intake. Therefore, food frequency questionnaires (FFQs) deemed appropriate for use among adults may not be suitable for adolescents.
To evaluate an FFQ in comparison with a 3-day food record (FR) in 14-year olds participating in a population-based cohort study in Western Australia.
Nutrient intakes estimated by a semi-quantitative FFQ were compared with those from a 3-day FR using Bland & Altman limits of agreement (LOA), tertile classifications and Pearson's correlation coefficients.
A total of 785 adolescents provided data from both dietary methods. Mean agreement between the FR and FFQ ranged from 73 (starch) to 161% (vitamin C). The LOA ranged from 27 (retinol) to 976% (carotene), with most nutrients being overestimated by the FFQ. For most nutrients, agreement between the two methods varied significantly with the magnitude of intake. Pearson's r ranged from 0.11 (polyunsaturated fats) to 0.52 (riboflavin). The FFQ classified 80 to 90% of subjects’ nutrient intakes into the same or adjacent tertile as their FR. Boys performed slightly better for all of these indices.
Agreement between individual FFQ and FR nutrient intakes was less than ideal. However, the FFQ was able to correctly rank a reasonable proportion of adolescents.
Behaviours contributing to disease risk in adulthood often originate in childhood and adolescence (WHO, 2003). Reliable estimation of nutrient intake at this time is important for investigations of diet-related conditions such as cardiovascular disease and diabetes. To assess nutrient intake, a food frequency questionnaire (FFQ) is often the tool of choice in epidemiological studies, as these are self-administered, economical and convenient.
However, assessing usual diet in adolescents can be difficult. Knowledge of food names, a good memory and adequate attention span are required, and tasks such as estimation of portion sizes can be a complex cognitive process (Livingstone and Robson, 2000). Specific factors associated with adolescence may also affect the overall quality of dietary reporting; unstructured eating patterns, increased eating outside the home and a heightened concern about body image (Livingstone et al., 2004). Although many studies have evaluated FFQs in adults, relatively few have tested FFQs within adolescent populations (Rockett et al., 1997; Lietz et al., 2002; Slater et al., 2003; Papadopoulou et al., 2008; Deschamps et al., 2009).
The Western Australian Pregnancy Cohort (Raine) Study has followed children from gestation to adolescence. The aim of this paper was to evaluate the FFQ administered to cohort members at 14 years of age. We investigated individual levels of agreement, ranking ability, and correlations between the FFQ and a 3-day food record (FR).
Subjects and methods
Full details of the Raine Study have been published (Newnham et al., 1993). Briefly, the study recruited 2900 women at 16 to 20 weeks gestation between 1989 and 1991 through the main public antenatal clinic and local private clinics in Perth, Western Australia. A total of 2804 women (97%) had 2868 live births, and these children have been followed up at regular intervals from birth to 14 years of age. Data collection for the Raine Study occurred in accordance with Australian national guidelines (NHMRC, 2007) and was approved by the ethics committees of King Edward Memorial Hospital for Women and Princess Margaret Hospital for Children, Perth, Western Australia.
At the 14-year follow-up, questionnaires were completed on a range of physical and mental health factors. In addition, all adolescents were requested to visit the Telethon Institute for Child Health Research, Perth, Western Australia for physical assessments. Informed consent was obtained from the adolescent and their primary caregiver.
Food frequency questionnaire
The 14-year follow-up provided the first opportunity to conduct a comprehensive assessment of dietary intake in this cohort. A semi-quantitative FFQ developed by the CSIRO (Commonwealth Scientific and Industrial Research Organisation) in Adelaide, Australia (Baghurst and Record, 1984) was used to assess the study adolescent's usual dietary intake over the previous year. To date, this FFQ had only been evaluated in adults (Baghurst and Record, 1983; Ambrosini et al., 2003). The FFQ collected information on the frequency of consumption of 212 foods, mixed dishes and beverages, and was modified to include beverages and snacks popular among adolescents, excluding alcohol. Consumption frequency options were: never, rarely, number of times per month, number of times per week and number of times per day. Owing to the age of respondents and their potential difficulty with completing the FFQ (Nelson and Bingham, 1997) or a lack of interest that might affect FFQ completion, we asked the primary caregiver to complete the FFQ and consult with the adolescent where necessary, for example, regarding foods eaten outside the home. Respondents were asked to record if the typical serving size differed in relation to a standard serving size given in household units, which was based on weighed FRs collected in previous work (Rohan and Potter, 1984). Detail was sought on cooking methods, types of spreads and oils, low-fat, fresh, frozen and canned foods. Respondents were also asked to record any additional items often eaten but not included in the FFQ. All FFQs were checked by a research nurse and queries were clarified with the adolescent. Data from the FFQ were doubly entered into a database and verified by the CSIRO. Estimated daily nutrient intakes were provided by CSIRO using Australian food composition data (FSANZ, 1995).
Three-day FRs recorded in household measures have been shown to be an appropriate method of assessing usual diet in children (Crawford et al., 1994) and were used as a reference method to assess the reliability of the FFQ. The term ‘reliability’ used throughout this paper refers to ‘intermethod reliability’. We have chosen not to use the term ‘validity’ because this requires that an error-free method be used for comparison (Armstrong et al., 1992). Although FR is thought to provide a more accurate estimate of dietary intake, they are not error free.
Adolescents attending their physical assessment received a standard 3-day FR booklet. The FR was designed to be completed by the adolescents with parental assistance if required. Instructions were given in the FR booklet on how to measure serves and subtract any leftovers. Metric measuring cups and spoons were provided to assist with estimating serving sizes. Details requested included cooking methods, condiments, fat on meat and individual ingredients used in mixed dishes. Adolescents were also asked to record if any of the 3 days recorded were not representative of their typical eating habits.
Completed FRs were checked by a dietician and clarifications were made either face-to-face at the physical assessment or by telephone (Di Candilo et al., 2007). All FR were entered into the FoodWorks diet analysis package (Xyris Software, 2007) by a dietician. Average daily nutrient intakes were calculated by linking the FRs with Australian Food Composition Tables (FSANZ, 1995).
We excluded FFQ and FR in which the total energy intake was <3000 or >20 000 kj per day, as these levels were considered implausible and have been used in another study of adolescents (Rockett et al., 1997). In addition, we excluded FR in which the subject recorded that the FR was not representative of their usual diet. Three statistical methods were used to assess different capabilities of the FFQ, separately for boys and girls.
The limits of agreement (LOA) method was used to determine agreement between absolute values from each method (Bland and Altman, 1986), which is important if individual nutrient intakes are to be compared with cut-offs such as recommended daily intakes (Ambrosini et al., 2003). The LOA approach provides an informative analysis of reliability, including information about the magnitude of errors between methods, the direction of bias between methods and whether or not bias is constant across levels of intake (Bland and Altman, 1986).
Mean agreement and the 95% LOA between absolute nutrient intakes estimated by the FFQ and FR were calculated (Bland and Altman, 1999). Mean agreement indicates how well the FFQ and FR agree on average, at the population or study sample level. Mean agreement for each nutrient was estimated as the mean of all individual differences between methods, that is, (∑ FFQ − FR)/n. The 95% LOA represents a range of values within which, 95% of all differences between methods are expected to fall. The LOA help to identify the extent of differences between methods. Using the SD of differences between methods (s.d.diff), the 95% LOA were calculated for each nutrient as mean agreement ±1.96 (s.d.diff).
All nutrient intakes were transformed to their natural logarithms before analyses because of the usual skewness in intake distributions. Therefore, the LOA findings are reported as ratios, that is, a multiple of the FFQ relative to the FR (Bland and Altman, 1999). For example, mean agreement of 100% for energy intake would suggest exact agreement, whereas mean agreement of 120% indicates that the FFQ overestimates energy intake by 20%, on average. Furthermore, 95% LOA of 55–184% for energy would suggest that 95% of all subjects’ FFQ estimates are between 55 and 184% of their FR energy estimate.
For each nutrient, we also examined whether agreement between the FR and FFQ was constant across the range of intakes. This was achieved by estimating the regression slope of differences (β) between the FR and FFQ, that is, regressing the average of the two methods on their differences (Bland and Altman, 1999). A slope (β) that is significantly different from zero indicates that agreement between the FR and FFQ varies significantly according to the magnitude of intake. This regression equation can also be used to calibrate individual FFQ estimates against the FR (Bland and Altman, 1999).
Correct ranking ability is a more commonly desired outcome from an FFQ and is thought to be adequate for the purpose of analysing diet-disease relationships (Willett, 1998). As an indicator of how well the FFQ could correctly rank individuals compared with the FR, we compared classifications of nutrient intakes into low, medium and high tertiles from each method.
Although we (Ambrosini et al., 2003) and others (Chinn, 1990; Hebert and Miller, 1991) have previously shown that the correlation coefficient can be a misleading indicator of agreement, we calculated Pearson's r to allow comparisons with other studies that have not used the LOA or tertile misclassification methods.
At 14 years of age, 2337 (81.5%) out of the total cohort of 2868 adolescents were eligible for the follow-up, that is, were alive and had not withdrawn from the study. Of these, 1857 (79.5%) responded to questionnaires and 1286 (55%) attended their physical assessment. Table 1 compares adolescents in the reliability study (n=785) with the remainder of the cohort (n=1552). These comparisons must be interpreted carefully, as the ‘remainder of the cohort’ includes subjects who did not respond or only partially responded to the 14-year follow-up, therefore a considerable proportion have missing data. However, on the basis of the available data (Table 1), subjects in the reliability study were slightly less likely to be overweight or at risk of being overweight and to be from very low-income families (⩽ $30 000 AUD per year), and were more likely to have highly educated mothers.
The FFQ was completed by 1631 respondents and of these, 18 (7 girls and 11 boys) were excluded due to implausible energy intakes, leaving 1613 FFQ for analysis. An FR was returned by 930 subjects, of which 884 were complete. Of the 884 complete 3-day FR, 99 adolescents (55 girls and 44 boys) reported at least 1 day that was ‘not representative of the usual diet’ and these were excluded from the analyses. All of the remaining FR had total energy intakes that were within our ‘plausible’ criteria. In total, 785 subjects had an FFQ and 3-day FR available for comparison. Nutrient intakes from both methods are shown in Table 2.
For both girls and boys, the majority of nutrients showed average agreement that was significantly different from 100% (Table 3). The FFQ overestimated most nutrients with the exception of starch, niacin and folate, which the FFQ underestimated. For nearly all nutrients, agreement between the FR and FFQ was significantly poorer at high levels of intake (positive slope in differences). Some micronutrients, for example, thiamin, riboflavin, vitamin C and carotene showed significantly poorer agreement at low intakes (negative slope in differences). Most LOA ranged from 50 to 250%, which is similar to studies in adults examining LOA between an FR and FFQ (Hodge et al., 2000; Ambrosini et al., 2001; Ambrosini et al., 2003). Polyunsaturated fats, vitamin C and carotene showed the poorest agreement in both sexes; all were significantly overestimated by the FFQ, had wide LOA and large slopes in differences. Retinol also showed very wide LOA for both sexes. Figure 1 and 2 illustrate the mean agreement, 95% LOA and regression slope of differences for macronutrients in girls and boys.
Girls were more likely to overestimate nutrient intakes in their FFQ. Although both boys and girls showed significant variation in agreement across intake for nearly all nutrients, the slopes of differences for girls were almost always greater, indicating larger discrepancies between their FR and FFQ with changes in intake (see macronutrients in Figures 1 and 2).
The ranking analysis indicated that between 80 and 90% of subjects were classified by the FFQ into either the same or adjacent tertile as the FR (Table 4). The FFQ incorrectly classified 10–19% of girls and 6–19% of boys into the opposite tertile to the FR. Ranking was poorest for polyunsaturated fats.
Pearson's correlation coefficients between the FR and FFQ ranged from 0.11 (polyunsaturated fats) to 0.52 (riboflavin) and all were statistically significant (Table 3).
This study reports on agreement between absolute nutrient intakes, their correlations and the ranking ability of a FFQ compared with an FR, in a cohort of adolescents. The LOA analysis showed that the FFQ tended to overestimate nutrient intakes in comparison with the FR. In addition, differences between the FR and FFQ were not consistent across levels of intake, but were dependent on the magnitude of intake for nearly all nutrients, even after log-transformation of data. Polyunsaturated fats, vitamin C and carotene showed the poorest agreement in both sexes. These findings correspond with those found in another study where we examined LOA between the same FFQ and a 28-day FR in adults (Ambrosini et al., 2003). Although agreement between absolute nutrient intakes was less than ideal, the FFQ showed a modest ability to rank subjects into the same or adjacent tertile as the FR.
Boys performed marginally better than girls for all indicators of reliability in this study. This may be due to greater variations in dietary intake among girls, which has been consistently reported by others (Livingstone et al., 2004). One explanation might be that girls are more likely to be concerned about body image (Livingstone et al., 2004) and as a result, they may be more prone to dieting and erratic food intake or recall bias.
We are aware of only one other study that has reported LOA between an FFQ and FR in an adolescent population. The EPIC (European Prospective Investigation of Cancer) FFQ was compared with a 7-day weighed FR in a small sample of Scottish adolescents (n=50) with a mean age of 12.3 years (Lietz et al., 2002). Their LOA analysis of energy, total fat, sugar, protein and calcium intake suggested poor agreement, with the FFQ overestimating all of these nutrients. However, the EPIC FFQ's ranking ability was thought to be acceptable; the proportion of subjects classified by their FFQ into the same tertile as their FR ranged from 37.8 to 48.6%. These figures are comparable with our FFQ.
Few other studies have examined the reliability of FFQ in similarly aged adolescents. Most have used a series of 24-h recalls as the reference method and relied on Pearson's correlation coefficients (Rockett et al., 1997; Slater et al., 2003; Papadopoulou et al., 2008; Deschamps et al., 2009). Their correlation coefficients ranged from 0.18 (retinol) to 0.86 (energy) (Rockett et al., 1997; Slater et al., 2003; Papadopoulou et al., 2008; Deschamps et al., 2009), with polyunsaturated fats having the lowest correlation in two studies (Rockett et al., 1997; Deschamps et al., 2009). Our results show that the Pearson's correlation coefficient is not able to indicate the degree or direction of bias between two methods. Even though the LOA agreement were not ideal for many nutrients in this study, all Pearson's correlation coefficients were statistically significant, and most were of magnitudes that are frequently deemed acceptable in other reliability studies.
This study's main strength is its large number of respondents, which exceeds most other reliability studies and allowed gender comparisons. The response fractions for both dietary methods were very good. We have utilized different statistical methods, including one that quantifies individual differences and the direction and degree of bias between dietary methods. This study is population-based, whereas most reliability studies reviewed for this paper were school-based. However, as is often the case with reliability studies (Nelson, 1997), subjects who participated in this reliability study seem slightly more likely to be of higher socioeconomic status compared with the rest of the cohort and less likely to be overweight. These differences must be considered if wanting to apply these findings to other populations. Another possible limitation is that the FFQ was completed by the adolescent's primary caregiver, and this may have introduced some error into FFQ estimates.
For this reliability analysis, we assumed that the 3-day FR represented usual food intake. However, some subjects may have altered their usual food intake as a result of keeping an FR, and under-reporting is thought to be common among adolescents (Livingstone et al., 2004). Furthermore, 3 days may not have been sufficient to capture usual intake in this adolescent cohort. However, requesting more than 3 days of FRs would have increased subject burden and most likely diminished the quality and number of completed FRs. For some nutrients, biomarkers may provide a better reference method than an FR, as their errors are unlikely to be correlated with those in an FFQ. However, out of the currently available dietary assessment methods, the FR is thought the least likely to have errors correlated with an FFQ (Willett and Lenart, 1998).
In conclusion, this FFQ seems to perform similarly in adolescents and adults when compared with an FR using Bland and Altman's methods (Ambrosini et al., 2003). The level of agreement for both sexes suggests that calibrations will be necessary when investigating the effects of absolute nutrient intakes at an individual level. Although the ranking ability of this FFQ may be adequate for the purpose of analysing diet-disease relationships, the potential errors in both dietary methods highlight the importance of maximizing subject numbers to obtain adequate statistical power to detect significant relationships between dietary and other variables (de Klerk et al., 1989).
Ambrosini G, de Klerk NH, Musk AW, Mackerras D (2001). Agreement between a brief food frequency questionnaire and diet records using two statistical methods. Public Health Nutr 4, 255–264.
Ambrosini G, Mackerras D, de Klerk N, Musk A (2003). Comparison of an Australian food frequency questionnaire to diet records: implications for nutrition surveillance. Public Health Nutr 6, 415–422.
Armstrong BK, White E, Saracci R (1992). Validity and reliability studies. In: Kelsey JL, Marmot MG, Stolley PD, Vessey MP (eds). Principles of Exposure Measurement in Epidemiology. Oxford University: New York Press. pp 79–136.
Baghurst KI, Record SJ (1983). Intake and sources in selected Australian subpopulations, of dietary constituents implicated in the etiology of chronic diseases. J Food Nutr 40, 1–15.
Baghurst KI, Record SJ (1984). A computerised dietary analysis system for use with diaries or food frequency questionnaires. Community Health Stud 8, 11–18.
Bland JM, Altman DG (1999). Measuring agreement in method comparison studies. Stat Methods Med Res 8, 135–160.
Bland MJ, Altman DG (1986). Statistical methods for assessing agreement between two methods of clinical assessment. Lancet 1, 307–310.
Chinn S (1990). The assessment of methods of measurement. Stat Med 9, 351–362.
Crawford P, Obarzanek E, Morrison J (1994). Comparative advantage of 3-day food records over 24-hour recall and 5-day food frequency validated by observation of 9- and 10-year-old girls. J Am Diet Assoc 94, 626–630.
de Klerk N, English D, Armstrong B (1989). A review of the effects of random measurement error on relative risk estimates in epidemiological studies. Int J Epidemiol 18, 705–712.
Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M (2009). Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 63, 282–291.
Di Candilo KG, Oddy WH, Miller M, Sloan N, Kendall GE, de Klerk NH (2007). Follow-up phone-calls increase nutrient intake estimated by three-day food diaries in 13 year old participants of the Raine study. Nutr Diet 64, 165–171.
FSANZ—Food Standards Australia New Zealand (1995). NUTTAB95 Australian food composition tables. Last updated on 8 October 2008, retrieved on 10 October 2008 fromhttp://www.foodstandards.gov.au/monitoringandsurveillance/foodcompositionprogram/.
Hebert J, Miller D (1991). The inappropriateness of conventional use of the correlation coefficient in assessing validity and reliability of dietary assessment methods. Eur J Epidemiol 7, 339–343.
Hodge A, Patterson AJ, Brown WJ, Ireland P, Giles GG (2000). The Anti Cancer Council of Victoria FFQ: relative validity of nutrient intakes compared with weighed food records in young to middle-aged women in a study of iron supplementation. ANZ J Public Health 24, 576–583.
Kuczmarski RJ, Ogden CL, Grummer-Strawn LM, Flegal KM, Guo S, Wie R et al. (2000). CDC growth charts: United States. Advance Data 314, Last updated on 4 December 2000; retrieved on 9 March 2009 fromhttp://www.cdc.gov/nchs/data/ad/ad314.pdf.
Lietz G, Barton KL, Longbottom P, Anderson AS (2002). Can the EPIC food-frequency questionnaire be used in adolescent populations? Public Health Nutr 5, 783–789.
Livingstone M, Robson P (2000). Measurement of dietary intake in children. Proc Nutr Soc 59, 279–293.
Livingstone MBE, Robson PJ, Wallace JMW (2004). Issues in dietary intake assessment of children and adolescents. Br J Nutr 92, S213–S222.
Nelson M (1997). The validation of dietary assessment. In: Margetts BM and Nelson M (eds.). Design Concepts in Nutritional Epidemiology. Oxford University Press: New York. pp 241–272.
Nelson M, Bingham SA (1997). Assessment of food consumption and nutrient intake. In: Margetts BM and Nelson M (eds). Design Concepts in Nutritional Epidemiology. Oxford: Oxford University Press. pp 123–169.
Newnham JP, Evans SF, Michael CA, Stanley JF, Landau LI (1993). Effects of frequent ultrasound during pregnancy—a randomised controlled trial. Lancet 342, 887–891.
NHMRC—National Health and Medical Research Council of Australia (2007). National statement on ethical conduct in research involving humans. Last updated on 11 October 2007; retrieved on 10 October 2008 fromhttp://www.nhmrc.gov.au/guidelines/ethics/human_research/index.htm.
Papadopoulou SK, Barboukis V, Dalkiranis A, Hassapidou M, Petridou A, Mougios V (2008). Validation of a questionnaire assessing food frequency and nutritional intake in Greek adolescents. Int J Food Sci Nutr 59, 148–154.
Rockett HRH, Breitenbach M, Frazier AL, Witschi J, Wolf AM, Field AE et al. (1997). Validation of a youth/adolescent food frequency questionnaire. Prev Med 26, 808–816.
Rohan TE, Potter JD (1984). Retrospective assessment of dietary intake. Am J Epidemiol 120, 876–887.
Slater B, Philippi ST, Fisberg RM, Latorre MRDO (2003). Validation of a semi-quantitative adolescent food frequency questionnaire applied at a public school in Sao Paolo, Brazil. Eur J Clin Nutr 57, 629–635.
Xyris Software (2007). FoodWorks Professional v.4.0. Brisbane. Xyris Software: Australia.
WHO—World Health Organisation (WHO) and the Food and Agriculture Organisation (FAO) of the United Nations (2003). Diet, nutrition and the prevention of chronic diseases. Report of a Joint WHO/FAO Expert Consultation. WHO Technical Report Series no. 916. Geneva, World Health Organisation.
Willett W, Lenart E (1998). Reproducibility and validity of food-frequency questionnaires. In: Kelsey JL, Marmot MG, Stolley PD, Vessey MP (eds). Nutritional Epidemiology. Oxford University Press: New York. pp 101–147.
Willett WC (1998). Food frequency methods. In: Kelsey JL, Marmot MG, Stolley PD, Vessey MP (eds). Nutritional Epidemiology. Oxford University Press: New York. pp 74–100.
We are extremely grateful to the families who took part in this study and the Raine Study team; data collectors, cohort managers, data managers, clerical staff, research scientists and volunteers. This research was supported by the Telethon Institute for Child Health Research, the Telstra Research Foundation of Australia, the Australian Rotary Health Research Fund, the Raine Medical Research Foundation, the Western Australian Health Promotion Research Foundation (Healthway) and an Australian National Health and Medical Research Council Program Grant.
About this article
Cite this article
Ambrosini, G., de Klerk, N., O'Sullivan, T. et al. The reliability of a food frequency questionnaire for use among adolescents. Eur J Clin Nutr 63, 1251–1259 (2009). https://doi.org/10.1038/ejcn.2009.44
- validation studies
- diet surveys
- diet records
- Raine study
British Journal of Nutrition (2020)
International Journal of Food Sciences and Nutrition (2020)
European Journal of Nutrition (2020)
Dietary fibre intake and its associations with depressive symptoms in a prospective adolescent cohort
British Journal of Nutrition (2020)
Differences in dietary fibre intake and associated familial factors in a longitudinal study at two time points across adolescence
Public Health Nutrition (2020)