302743 446..456 A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method Corby K. Martin 1 *, Hongmei Han 1 , Sandra M. Coulon 1 , H. Raymond Allen 1 , Catherine M. Champagne 1 and Stephen D. Anton 2 1 Pennington Biomedical Research Center, 6400 Perkins Road, Baton Rouge, LA 70808, USA 2 Department of Aging and Geriatric Research, University of Florida, PO Box 112610, Gainesville, FL, USA (Received 12 December 2007 – Revised 25 March 2008 – Accepted 28 April 2008 – First published online 11 July 2008) The aim of the present study was to report the first reliability and validity tests of the remote food photography method (RFPM), which consists of camera-enabled cell phones with data transfer capability. Participants take and transmit photographs of food selection and plate waste to research- ers/clinicians for analysis. Following two pilot studies, adult participants (n 52; BMI 20–35 kg/m 2 inclusive) were randomly assigned to the dine-in or take-out group. Energy intake (EI) was measured for 3 d. The dine-in group ate lunch and dinner in the laboratory. The take-out group ate lunch in the laboratory and dinner in free-living conditions (participants received a cooler with pre-weighed food that they returned the following morning). EI was measured with the RFPM and by directly weighing foods. The RFPM was tested in laboratory and free-living conditions. Reliability was tested over 3 d and validity was tested by comparing directly weighed EI to EI estimated with the RFPM using Bland–Altman analysis. The RFPM produced reliable EI estimates over 3 d in laboratory (r 0·62; P,0·0001) and free-living (r 0·68; P,0·0001) conditions. Weighed EI correlated highly with EI estimated with the RFPM in laboratory and free-living conditions (r . 0·93; P,0·0001). In two labora- tory-based validity tests, the RFPM underestimated EI by 24·7 % (P¼0·046) and 25·5 % (P¼0·076). In free-living conditions, the RFPM under- estimated EI by 26·6 % (P¼0·017). Bias did not differ by body weight or age. The RFPM is a promising new method for accurately measuring the EI of free-living individuals. Error associated with the method is small compared with self-report methods. Digital photography: Food intake: Energy intake: Measurement: Self-report The gold standard for measuring food or energy intake (EI) in free-living humans is the doubly labelled water (DLW) method. DLW provides an accurate measure of total daily energy expenditure and, during a period of energy balance, total daily energy expenditure is equal to EI (1,2) . When a large energy deficit is present during the DLW period, how- ever, it is difficult to obtain an accurate (valid) estimate of an individual’s short-term EI using DLW, even if changes in energy stores are considered (3) . This limitation is noteworthy, since researchers and clinicians frequently require an estimate of EI during diets or periods of energy restriction. Additional limitations of the DLW method include: (1) cost; (2) avail- ability; (3) its inability to provide important information about the type and micro- and macronutrient composition of foods ingested. Nevertheless, seemingly few valid and reliable alternatives for estimating EI are available. Self-report methods are frequently used to collect EI data, including 24 h food recall and pen-and-paper food records. When estimating EI with 24 h food recall, a trained individual interviews the participant about his/her food and beverage consumption over the previous 24 h. This method relies on the ability of the participant to accurately recall the types and amounts of foods consumed during the previous 24 h, and it assumes that these foods are representative of habitual EI. Consequently, this method is subject to error (4–6) . For example, EI estimated with 24 h recall in New Zealand’s National Nutrition Survey resulted in significant underestima- tion of EI, particularly among women and obese partici- pants (5) , and 24 h recall with multiple-pass methodology resulted in significant underestimation of EI in a sample of African-American women diagnosed with type 2 diabetes (6) . Efforts to improve the accuracy of 24 h recall have been disappointing. For example, financial incentives were not found to improve diet recall (7) . Food records are another self-report method frequently used to estimate food intake. When using food records, participants estimate the portion sizes of foods that they eat and record this portion size and a description of the food on the food record, which is typically pen and paper. The accuracy of this method has been questioned (8,9) . Tightly controlled studies that com- pared self-reported EI to EI measured with DLW indicate that individuals significantly under-report their EI when using food records (10,11) . Moreover, overweight and obese individuals tend to under-report EI to a greater degree than * Corresponding author: Dr Corby K. Martin, fax þ1 225 763 3045, email Corby.Martin@pbrc.edu Abbreviations: DLW, doubly labelled water; EI, energy intake; EMA, ecological momentary assessment; ICC, intraclass correlation coefficient; PBRC, Pennington Biomedical Research Center; PDA, personal digital assistant; RFPM, remote food photography method. British Journal of Nutrition (2009), 101, 446–456 doi:10.1017/S0007114508027438 q The Authors 2008 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 lean individuals (10) , and individuals tend to selectively under- report fat intake (12) . Different methods have been used to try to improve the accuracy of food records, but the validity of these methods remains questionable. For example, an indivi- dualised approach to food records that considered variability in EI and estimated energy requirements resulted in underesti- mates of EI of 27 % or more (13) , and a method that included visual aids to improve portion-size estimates underestimated EI by 14 % with marked variability (14) . Software applications housed on hand-held devices such as personal digital assistants (PDA) have been developed to reduce the burden of keeping a food record and to improve their accuracy. Nevertheless, food records kept on PDA appear no more accurate than 24 h recall (15) or traditional food records (16) and do not improve the accuracy of self- report EI (17) . Importantly, Beasley et al. (15) noted that the lar- gest source of error in estimated EI resulted from participants’ poor estimation of portion size. Individuals have difficulty estimating portion size even after training and these errors negatively affect EI estimates (18) . Our laboratory found that participants more accurately estimated portion size after extensive training, though a large degree of variability in portion-size estimates remained (19) . Together, these findings indicate that methods are needed that do not rely on the participant to estimate portion sizes. Methods to measure EI that do not rely on the participant to estimate portion size are limited. Direct observation of food selection and plate waste by trained observers results in accu- rate EI estimates (20,21) , but this method is not applicable to free-living conditions. The digital photography of foods method (22,23) , however, could be modified for free-living con- ditions. When using this method, the plate of foods selected by an individual is photographed and the individual’s plate waste is photographed following the meal. Reference or standard portions of known quantities of the foods are also photo- graphed. In the laboratory, trained registered dietitians use these photographs to estimate the portion size of food selec- tion and plate waste by comparing these photographs to the standard portion photographs. These estimates are entered into a computer application that calculates the weight (g), energy (kJ or kcal) and macro- and micronutrients of food selection, plate waste, and food/EI based on a US Department of Agriculture database (24) . The digital photography method has been found to be highly reliable and accurate when used to measure the EI of adults (22,23) and children (25) in cafeteria settings. Portion-size estimates from digital photography correlate highly with weighed portion sizes (r . 0·90; P,0·0001) and mean differences between directly weighing foods and digital photography are minimal (,6 g) (23) . Agree- ment is high among trained registered dietitians who estimate portion sizes using digital photography; for example, intra- class correlation coefficients (ICC) are consistently .0·90 (25) . To our knowledge, only a few studies have used digital photographs to estimate EI in free-living conditions. A research group in Japan used a PDA with a digital camera to collect EI data in free-living conditions. The method results in EI estimates comparable with weighed food records. Never- theless, these studies relied on samples of young lean college women who were majoring in food and nutrition, and the criteria (food records) are prone to error (26,27) . When used in a more diverse sample of males and females, the method resulted in EI estimates that were significantly lower than esti- mates from weighed records, and under-reporting in males was associated with obesity (28) . Nevertheless, these important studies demonstrate the likely utility of such technology. The purpose of the present paper is to describe the develop- ment of the remote food photography method (RFPM) and to report the first reliability and validity tests of this method. The RFPM relies on the validated digital photography method (22,23) to analyse food photographs and estimate EI, but the photographs are collected in free-living conditions with a camera-enabled cell phone and are sent to the researchers in near real time via a cellular network. Two pilot studies and the main study, which includes tests of reliability and validity, are described. It was hypothesised that initial reliability and validity data would suggest that the RFPM is a viable method for measuring EI in free-living conditions. The three studies reported here were conducted at the Pennington Biomedical Research Center (PBRC; Baton Rouge, LA, USA). All appli- cable institutional and government regulations concerning the ethical use of human volunteers were followed. All participants provided written informed consent and the research was approved by the Institutional Review Board of the PBRC. Pilot study 1 The purpose of pilot study 1 was to overcome the primary limitation of adapting the digital photography method for free-living humans, which is the need for a photograph of a weighed standard portion from the dining location. When the digital photography method is utilised in the cafeteria, a photograph is taken of a standard portion of weighed food. In the laboratory, trained registered dietitians estimate EI by comparing the photographs of food selection and plate waste to the standard portion photograph. When photographs are collected in free-living conditions, it is unfeasible for partici- pants to weigh and take a photograph of a standard portion of each food that they eat. Procedures To overcome this limitation, we created a large database of standard portion photographs from previous studies and we took new standard portion photographs of foods that are frequently consumed (for example, cereal) but were under- represented in the database. This database is called the ‘archive’ and it includes over 2100 photographs of different foods, and over 250 photos of different portion sizes of the same foods. Each food is linked to energy and nutrient infor- mation from the US Department of Agriculture database (24) . In rare cases when a food item is not represented in the US Department of Agriculture database, data from the manufac- turer is utilised (for example, food label, website). During pilot study 1, we simulated food selection and plate waste in the laboratory by taking photographs of foods with a PDA (Palm Pilot, Ziree 72; Palm Inc., Sunnyvale, CA, USA) equipped with a digital camera. We took photographs of thirty-one different foods that were grouped together to simu- late sixteen meals (for example, hamburger, ketchup, French fries, soda). Each simulated meal consisted of a mean of 3·88 foods; hence, each food was represented twice with two different portion sizes. For each simulated meal, the Remote food photography method 447 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 amount of food selected and plate waste was determined randomly. Food selection ranged from 35 to 800 % of the standard portion, and plate waste ranged from 0 to 90 % of the amount of food selected. The amount of food in each photograph was carefully weighed and recorded. Data analytic plan To test if use of a standard portion photograph from the archive facilitated reliable and valid/accurate estimates of EI, trained registered dietitians estimated food selection and plate waste using two methods in the following order: (1) without photo- graphs of a standard portion; (2) with photographs of a standard portion from the archive. For each of the two methods, we com- pared the registered dietitians’ estimates of food selection to the actual amount of food that was selected with dependent t tests. Similarly, plate waste and EI from each method were compared with weighed plate waste and EI. To test inter-rater reliability, two registered dietitians estimated EI for all of the foods (100 % over-sample) and ICC were calculated. We predicted that the registered dietitians’ EI estimates would not differ significantly from EI measured by directly weighing foods when a standard-portion photograph from the archive was used, but the estimates would differ significantly when no photograph from the archive was used. Results Inter-rater agreement between the two trained registered dieti- tians who estimated EI was high with and without the archive (Table 1); therefore, the registered dietitians’ estimates were averaged. Estimated food selection and EI differed signifi- cantly from weighed food selection and EI when the archive was not utilised, but this difference was not significant when the archive was utilised (Table 1). Estimated plate waste did not differ from weighed plate waste for either method. Estimated EI only differed from weighed EI by 28·2 % on average when the archive was utilised. Discussion The first pilot study demonstrated that trained registered dietitians can reliably and accurately estimate EI with stan- dard-portion photographs from the archive. Estimated food selection, plate waste and EI differed by 28·2 % on average from directly weighed food when the archive was used. Pilot study 2 Participants and procedures Pilot study 2 was a feasibility study to determine if free-living people could use the PDA to take pictures of their food selection and plate waste. Male and female adults (aged 18–54 years) with BMI 19–35 kg/m 2 (inclusive) were enrolled. Participants were asked to take pictures of their food selection, including energy-containing beverages, and plate waste for four consecu- tive days and to label the pictures with a brief description of the foods. When participants returned the PDA to the researchers, they were asked about their satisfaction with the method and factors that hindered their ability to take photographs of their T a b le 1 . C o m p a ri s o n o f fo o d s e le c ti o n , p la te w a s te a n d e n e rg y in ta k e (E I) e s ti m a te d w it h th e re m o te fo o d p h o to g ra p h y m e th o d w it h w e ig h e d E I w h e n a s ta n d a rd -p o rt io n p h o to g ra p h fr o m th e a rc h iv e w a s a n d w a s n o t u ti lis e d d u ri n g th e e s ti m a ti o n p ro c e d u re (p ilo t s tu d y 1 ) IC C † E s ti m a te d E I (k J ) W e ig h e d E I (k J ) E s ti m a te d E I (k c a l) W e ig h e d E I (k c a l) D if fe re n c e o f th e m e a n s (% ) C o rr e la ti o n 9 5 % C I M e a n S E M M e a n S E M M e a n S E M M e a n S E M t d f P N o s ta n d a rd -p o rt io n p h o to w a s u s e d F o o d s e le c ti o n 0 ·9 3 ** * 0 ·8 1 , 0 ·9 8 5 0 2 5 9 5 4 8 5 9 1 2 0 1 4 1 3 1 1 4 2 8 ·4 2 2 ·2 1 6 1 0 ·0 1 P la te w a s te 0 ·9 4 ** * 0 ·8 4 , 0 ·9 8 1 4 2 2 5 1 4 6 2 1 3 4 6 3 5 5 2 2 ·9 2 0 ·3 5 6 1 0 ·7 3 E n e rg y in ta k e 0 ·9 1 ** * 0 ·7 6 , 0 ·9 7 3 6 0 4 6 4 0 6 5 0 8 6 1 1 9 7 1 2 2 1 1 ·3 2 2 ·3 0 6 1 0 ·0 3 S ta n d a rd -p o rt io n p h o to w a s u s e d F o o d s e le c ti o n 0 ·9 5 ** * 0 ·8 6 , 0 ·9 8 5 2 3 5 9 5 4 8 5 9 1 2 5 1 4 1 3 1 1 4 2 4 ·6 2 1 ·1 2 6 1 0 ·2 7 P la te w a s te 0 ·9 2 ** * 0 ·8 1 , 0 ·9 7 1 5 1 2 5 1 4 6 2 1 3 6 6 3 5 5 þ 2 ·9 0 ·5 1 6 1 0 ·6 1 E n e rg y in ta k e 0 ·9 2 ** * 0 ·7 9 , 0 ·9 7 3 7 2 4 6 4 0 6 5 0 8 9 1 1 9 7 1 2 2 8 ·2 2 1 ·5 8 6 1 0 ·1 2 IC C , in tr a c la s s c o rr e la ti o n c o e ffi c ie n t. ** *P , 0 ·0 0 0 1 . † IC C w it h 9 5 % C I re p re s e n t a g re e m e n t b e tw e e n th e tw o re g is te re d d ie ti ti a n s w h o e s ti m a te d E I (1 0 0 % o v e r- s a m p le ). C. K. Martin et al.448 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 foods. Trained registered dietitians estimated EI using the digi- tal photography method. During this process, they recorded problems with the pictures that limited their ability to estimate EI from the photographs. No data analyses were planned for this pilot and feasibility study, which served the purpose of identi- fying barriers to collecting digital photography data in free- living conditions and analysing the data. Results Participant characteristics for pilot study 2 are depicted in Table 2. The sample was predominantly Caucasian (67 %) and female (79 %). Pilot study 2 demonstrated that free-living indi- viduals can use a PDA to take pictures of their food selection and plate waste and also identified factors that inhibit application of the method to free-living conditions. Moreover, the study identified ways to modify procedures to overcome these factors. First, participants occasionally forgot to take photographs of their food selection and plate waste. To address this problem, ecological momentary assessment (EMA) (29) technology was adopted and alarms were installed on the Palm Pilotse to remind participants to take photographs of their foods. Second, the researchers could not review the data until the par- ticipants returned the PDA at the end of the 4 d study. There- fore, systematic mistakes in taking the pictures could not be corrected, and these mistakes negatively affected data quality for the entire study. To address this problem, future research relied on cellular phones equipped with digital cameras and data transfer capabilities that allowed photographs to be sent to the researchers in near real time. Third, many photographs were difficult to analyse due to poor lighting; consequently, the new data collection devices (cell phones) included a flash and computer software is being developed to digitally enhance photographs. Fourth, participants occasionally pro- vided poor descriptions of foods in pictures due to the time required to type the description. To correct this problem, participants were instructed to quickly record voice (or text) messages with the cell phone to describe the foods. Finally, participants required experiential training with the data collec- tion devices to take good photographs; therefore, we created a 20–30 min training paradigm. Discussion The results of the second pilot study provided important infor- mation about factors that inhibit data collection in free-living conditions, and the study identified ways to overcome these limitations. The results of this study were instrumental in guiding development of the RFPM and addressing factors that affected data quality. Main study The aim of the main study was to finalise the development and procedures for the RFPM and to obtain reliability and validity data from laboratory and free-living conditions. Measures Remote food photography method. The procedures for the RFPM were finalised based on the results of the two pilot studies and consisted of the following: (1) Participants were trained to use the cell phones to take pictures of their food selection and plate waste, label the pictures, and send these pictures to the researchers over the cellular network; (2) Participants received and were asked to respond to four to six automated prompts that reminded them to take photo- graphs and to send the photographs to the researchers. These prompts utilised EMA (29) principles and consisted of emails and text messages; (3) Participants were provided with a telescoping pen to stan- dardise the distance of the camera from the food and were instructed to take photographs at a 458 angle; (4) Participants were instructed to record their EI using tra- ditional pen-and-paper methods or a voice recording on the phone in case of technology failure (we also inter- viewed participants at the end of the data collection period to name foods that were difficult to identify or were poorly described). Participants were trained on these procedures during a 20–30 min individual experiential training session during which they showed mastery of the procedures before being dismissed. The phones used during the research were Motorola i860 phones (Motorola, Inc., Schaumburg, IL, USA), which were equipped with 1·3-mega pixel cameras and 4 £ digital zoom. Data analysis/estimation of EI followed the validated digital photography of foods method (22,23) and is outlined in the Introduction of the present paper. Participants completed a form before data collection that asked them to rate their comfort level with using a cell phone (1 ¼ not at all comfortable; 5 ¼ extremely comfortable). After data collection, participants completed a form that asked Table 2. Baseline characteristics and sex and race distribution of study participants (Mean values with their standard errors or numbers and percentages) Sex Race Age (years) Height (cm) Weight (kg) BMI (kg/m2) Male Female Caucasian African American Other Mean SEM Mean SEM Mean SEM Mean SEM n % n % n % n % n % Pilot study 1 – – – – – – – – – – – – – Pilot study 2 (n 42) 33·9 1·6 166·8 1·7 70·7 1·9 25·5 0·7 9 21 33 79 28 67 12 28 2 5 Main study (n 50) 32·4 1·5 170·6 1·4 77·2 1·9 26·5 0·5 23 46 27 54 35 70 15 30 0 0 Remote food photography method 449 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 about their satisfaction with the RFPM, and their satisfaction with sending the photographs to the researchers (1 ¼ extremely dissatisfied; 6 ¼ extremely satisfied). Participants also rated their perceived ease of using the RFPM (1 ¼ very difficult; 6 ¼ extremely easy), and if they would rather use the RFPM or pen-and-paper records to record intake. Collection of these data began after we started the trial; therefore, data were only available on a subset of the study sample. Weighed energy intake. EI measured by directly weighing foods served as the gold standard criterion for EI and was used to test the validity of the RFPM over 3 d. EI was measured under two conditions. First, participants ate meals in the Inges- tive Behavior Laboratory of the PBRC. Second, a group of participants were provided with pre-weighed food in a cooler and were asked to eat this food in their natural environ- ment for dinner. The participants returned the cooler the fol- lowing morning and the remaining food was weighed to calculate EI. Participants used the RFPM in the laboratory and their natural environment. Participants Fifty-two participants were randomised and two participants failed to complete the study. Inclusion criteria were: healthy male or female aged 18–54 years with BMI 20–35 kg/m 2 (inclusive). Exclusion criteria were: use of medications that influenced eating behaviour, diagnosis of chronic illness, tobacco use, aversion to study foods, and, for females, irregu- lar menstrual cycle and pregnancy. The inclusion and exclu- sion criteria were lenient to recruit a diverse sample that is representative of people who participate in research or seek weight-loss treatment, increasing the generalisability of the study. Random assignment and procedures All participants completed training in the RFPM. Participants were randomly assigned (1:1 ratio) to the dine-in or take-out group. On each test day, all participants consumed a standard breakfast (Table 3). In the dine-in condition, participants com- pleted 3 d of food intake testing, during which lunch and dinner were eaten in the laboratory. In the take-out condition, participants completed 3 d of food intake testing, but they only ate lunch in the laboratory. Dinner was eaten out of the cooler in free-living conditions. Participants used the RFPM during all lunch and dinner meals; therefore, two EI measures were collected: (1) weighed EI; (2) EI estimated with the RFPM. In the dine-in and take-out conditions, the same types and amounts of foods were provided (Table 3) and test days were 2–7 d apart. Excess food was provided to ensure plate waste and provide a conservative test of the RFPM since the presence of plate waste creates more opportunity for error and is not always present in people’s natural environment, i.e. many people eat all of their portions. Data analytic plan The reliability of the RFPM was examined by calculating ICC for EI measured with the RFPM and directly weighing foods over the 3 d of testing in both free-living and laboratory conditions. Agreement among the three registered dietitians who scored the photographs was examined with ICC (27 % of meals were over-sampled). For the following ana- lyses, EI data were averaged over the 3 d of testing. Three series of analyses were conducted to test the validity of the RFPM. First, EI estimated with the RFPM was com- pared with weighed EI measured in the Ingestive Behavior Laboratory. These analyses relied on data from the dine-in group (their lunch and dinner data were combined). A second laboratory-based test of the RFPM relied on the take-out group’s lunch data. Third, the take-out group’s dinner data were analysed to test the validity of the RFPM in free-living conditions. The second and third series of ana- lyses allowed examination of the accuracy of the RFPM in laboratory v. free-living conditions with the same sample of participants. For each series of analyses, Pearson correlation coefficients were calculated between the RFPM and weigh- back method, and the Bland–Altman technique (30) was used to determine if: (1) the methods differ significantly from each other; (2) the accuracy of the RFPM varies with the amount of food eaten (EI). The validity of the RFPM was also examined as a function of body mass or weight (kg), age and sex. Error was calculated by determining the percentage difference between EI esti- mated with the RFPM and directly weighed foods. Error was regressed against body weight with and without sex in the model to test for significance of slopes. A similar analysis was conducted with age. These analyses were conducted for each of the three samples outlined in the previous paragraph, namely: (1) the combined lunch and dinner data from the dine-in group, (2) the take-out group’s lunch data, which were collected in the laboratory; (3) the take-out group’s dinner data that were collected in free-living conditions. Results Participant characteristics for the main study are depicted in Table 2. The sample consisted of 54 % females and 70 % Caucasians. Ecological momentary assessment and missing data. A total of 157 EMA prompts were sent near dinner time to par- ticipants in the take-out group and all were received, though thirty-seven (23·6 %) were delivered at the incorrect time (prompts were sent via an email server that delayed delivery of some messages). Participants responded to 118 (98·3 %) of the 120 prompts that were delivered on time. Each group, dine-in and take-out, consumed 150 meals that were measured with the RFPM and by directly weighing foods. For the dine-in group, all 150 meals were consumed in the Ingestive Behavior Laboratory. When using the weigh-back method, data from three meals were not analysed due to technical error or protocol violations, including eating only certain ingredients from the sandwiches, which made it impossible to obtain weighed intake. Four meals were not analysed due to poor picture quality or missing pictures. The take-out group consumed 150 meals (seventy-five meals in the laboratory and seventy-five meals in free-living conditions). In the laboratory, data from seven meals were not analysed: one meal had missing pictures; three meals had poor-quality pictures; three meals involved protocol violations. In free-living conditions, data from eleven meals were not analysed: five meals had poor-quality pictures; six C. K. Martin et al.450 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 Table 3. Description (energy and macronutrient content) of the foods served during breakfast, lunch and dinner during the main study Serving size (g) Energy (kJ) Energy (kcal) Fat (g) Protein (g) Carbohydrate (g) Breakfast Nutri-Grain w bar (one bar) 37·0 586 140 3 1 26 Skimmed milk (one carton) 230·8 377 90 0 8·1 11·4 Sun-Maidw raisins (one box) 43·3 540 129 0 1·3 34 Totals 311·1 1502 359 3 10·4 71·4 Lunch* Ham sandwiches (three hoagies†) 708·0 4858 1161 36·8 81·4 123·2 Roast beef sandwiches (three hoagies†) 708·0 5243 1253 36·1 105·5 120·4 Turkey sandwiches (three hoagies†) 708·0 4653 1112 28·3 89·2 120·4 Rufflesw reduced-fat potato chips 122·8 2544 608 26·0 8·7 82·4 Famous Amosw chocolate chip cookies 224·0 4845 1158 54·0 15·5 154·6 Totals 1054·8 12 042 to 12 631 2878 to 3019 108·3 to 116·8 105·6 to 129·7 357·4 to 360·2 Dinner Spaghetti noodles with Prego sauce 300 g noodles, 265 g sauce 3021 722 12·4 19·1 131·1 Spaghetti noodles with Alfredo sauce 300 g noodles, 265 g sauce 3858 922 45·5 19 97·7 Arezzow meatballs (ten meatballs) 480 2021 483 42 16·6 8·4 Pepperidge Farmw Texas Toast with cheese (four slices) 192 3012 720 44 16 68 Dolew tropical fruit salad (one container) 732 2008 480 0 0 126 Pepperidge Farm w iced brownie (four brownies) 296 5021 1200 48·4 14·4 189·2 Totals 2830 15 920 3805 179·9 66·1 522·7 * Participants selected and were served one of the three sandwich choices for lunch. † Elongated rolls. R e m o te fo o d p h o to g ra p h y m e th o d 4 5 1 British Journal of Nutrition Downloaded from https://www.cambridge.org/core. Carnegie Mellon University, on 06 Apr 2021 at 01:16:43, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 meals had protocol violations. Protocol violations included not eating the food provided by the researchers due to eating other foods. Reliability of the remote food photography method. Agreement was high among the three registered dietitians who used photographs from the RFPM to estimate EI. The ICC for food selection, plate waste and EI were 0·99 (95 % CI 0·99, 0·99), 0·91 (95 % CI 0·87, 0·94) and 0·88 (95 % CI 0·81, 0·91), respectively. The ICC for intake (kJ) of fat, carbohydrate and protein were 0·92 (95 % CI 0·88, 0·94), 0·85 (95 % CI 0·77, 0·89) and 0·85 (95 % CI 0·79, 0·90), respectively. These values are consistent with inter-rater agreement when the digital photography method is used in cafeteria settings (22,23,25) . EI estimated with the RFPM was reliable over the 3 d of testing. Based on the dine-in group’s combined lunch and dinner data, the RFPM produced reliable estimates of EI in laboratory conditions (ICC 0·62; P,0·0001). Moreover, data from the dinner meal of the take-out group indicated that the RFPM produced reliable estimates of EI in free living con- ditions (ICC 0·68; P,0·0001). These values are consistent with ICC calculated from directly weighing food intake in the laboratory over a number of days (31) . In the present study, the ICC for food intake measured in the laboratory were 0·68 and 0·70 for lunch and dinner, respectively (P,0·0001). Validity of the remote food photography method. Participants took and sent photographs of good quality to the researchers. Fig. 1 contains representative pictures from the study. The first series of analyses tested the validity of the RFPM in laboratory conditions and included the dine-in group’s com- bined lunch and dinner data. EI estimated with the RFPM corre- lated highly with weighed EI, though the RFPM significantly underestimated EI (2368 kJ (288 kcal); 24·7 %; Table 4). Bland–Altman analysis indicated that this bias was consistent over different levels of EI (Fig. 2). The second series of analyses tested the validity of the RFPM in laboratory conditions by analysing the take-out group’s lunch data. EI estimated with the RFPM correlated highly with weighed EI, and the RFPM non-significantly underestimated EI (2151 kJ (236 kcal); 25·5 %; Table 4). Bland and Altman analysis indicated that this bias was consis- tent over different levels of EI (Fig. 3). The third series of analyses tested the validity of the RFPM in free-living conditions by analysing the take-out group’s dinner data. EI estimated with the RFPM correlated highly with weighed EI, and the RFPM significantly underestimated EI (2406 kJ (297 kcal); 26·6 %; Table 4). Bland and Altman analysis indicated that this bias was consistent over different levels of EI (Fig. 4). Validity of the remote food photography method as a func- tion of body mass (kg), age and sex. Regression analysis indicated that there was no significant association between body weight (kg) and the RFPM error (percentage difference between EI estimated with the RFPM and weighed EI) with (adjusted R 2 20·07 to 0·08; P values .0·45) or without (adjusted R 2 20·04 to 0·01; P values .0·30) sex in the models. Age also was not associated with error (adjusted R 2 20·07 to 0·08; P values .0·65). Error did not differ signifi- cantly between men and women in the take-out group at lunch (t(21) ¼ 20·73; P¼0·47) or dinner (t(22) ¼ 20·75; P¼0·46). Error was significantly larger for women (29·2 %) compared with men (20·7 %) in the dine-in group (t(23) ¼ 22·08; P¼0·05). Satisfaction ratings. Thirty-five participants had data on their comfort level (1 ¼ not at all comfortable; 5 ¼ extremely comfortable) of using cell phones before data collection. Thirty (85·7 %) participants rated their comfort level as 4 or higher. After data collection, forty-seven participants had satisfaction data (1 ¼ extremely dissatisfied; 6 ¼ extremely satisfied) and thirty-seven (78·8 %) participants rated their satis- faction with the RFPM as 5 or higher and forty (85 %) parti- cipants rated their satisfaction with sending photographs to the researchers as 5 or higher. Last, forty-four (93·6 %) participants rated the ease of use of the RFPM as 5 or higher, and almost all participants (forty-four; 93·6 %) indicted they would rather use the RFPM rather than pen and paper to record food intake. Discussion The RFPM was found to produce reliable and valid estimates of EI in both laboratory and free-living conditions. The degree of error for the RFPM was very small (24·7 to 26·6 %) when compared with error of 37 % or more associated with self- report methods (10–12) , and the error was similar in both labora- tory and free-living conditions. Moreover, error associated Fig. 1. Representative pictures of food selection (a) and plate waste (b) taken by a participant and sent to the researchers via the wireless network. C. K. Martin et al.452 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 with the RFPM was not found to vary by the amount of energy/food eaten or body weight, which is a significant lia- bility of self-report methods (5) . Error also did not differ by age, and only one of three comparisons found that error dif- fered by sex (this comparison relied on data from the labora- tory). The majority of participants rated their satisfaction with the RFPM and with sending pictures to the researchers T a b le 4 . S u m m a ry o f a n a ly s e s to te s t th e v a lid it y o f e n e rg y in ta k e (E I) e s ti m a te d w it h th e re m o te fo o d p h o to g ra p h y m e th o d (R F P M ) c o m p a re d w it h d ir e c tl y w e ig h e d fo o d s in la b o ra to ry a n d fr e e -l iv in g c o n d it io n s (m a in s tu d y )† L o c a ti o n o f E I e s ti m a te d b y R F P M (k J ) E I m e a s u re d b y d ir e c tl y w e ig h e d fo o d (k J ) E I e s ti m a te d b y R F P M (k c a l) E I m e a s u re d b y d ir e c tl y w e ig h e d fo o d (k c a l) D if fe re n c e in E I (k J ) D if fe re n c e in E I (k c a l) P e a rs o n D if fe re n c e o f th e d a ta c o lle c ti o n M e a n S E M M e a n S E M M e a n S E M M e a n S E M M e a n S E M M e a n S E M c o rr e la ti o n t d f P m e a n s (% ) E I (l u n c h a n d d in n e r, d in e -i n g ro u p ) L a b o ra to ry 7 4 4 8 4 6 9 7 8 1 6 4 3 5 1 7 8 0 1 1 2 1 8 6 8 1 0 4 2 3 6 8 1 7 4 2 8 8 4 2 0 ·9 3 ** * 2 2 ·1 0 2 4 0 ·0 4 6 2 4 ·7 E I (l u n c h , ta k e -o u t g ro u p ) L a b o ra to ry 2 5 9 0 2 2 2 2 7 4 1 2 0 9 6 1 9 5 3 6 5 5 5 0 2 1 5 1 8 1 2 3 6 1 9 0 ·9 3 ** * 2 1 ·8 6 2 2 0 ·0 7 6 2 5 ·5 E I (d in n e r, ta k e -o u t g ro u p ) F re e -l iv in g 5 7 0 7 4 8 1 6 1 1 3 4 3 9 1 3 6 4 1 1 5 1 4 6 1 1 0 5 2 4 0 6 1 5 9 2 9 7 3 8 0 ·9 5 ** * 2 2 ·5 7 2 3 0 ·0 1 7 2 6 ·6 ** *P , 0 ·0 0 0 1 . † E I e s ti m a te d w it h th e R F P M w a s c o m p a re d w it h w e ig h e d E I w it h P e a rs o n c o rr e la ti o n s a n d d e p e n d e n t t te s ts . Fig. 2. Bland and Altman analysis comparing the remote food photography method (RFPM) with weighed energy intake (EI) in the laboratory. Lunch and dinner data from the dine-in group were included in these analyses. The RFPM bias in estimating EI was consistent over different levels of EI (R 2 0·03; adjusted R 2 20·01; P¼0·39). Fig. 3. Bland and Altman analysis comparing the remote food photography method (RFPM) with weighed energy intake (EI) in laboratory conditions by analysing lunch data from the take-out group. The RFPM bias in estimating EI was consistent over different levels of EI (R 2 0·03; adjusted R 2 20·02; P¼0·45). Remote food photography method 453 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 favourably and almost all (93·6 %) participants indicated that they would rather use the RFPM compared with pen-and- paper records to record food intake. Thus, the RFPM appears to be a promising new method for estimating group and individual EI. Implications of the findings. The RFPM has important implications for both research and clinical practice. Obtaining accurate estimates of food and macronutrient intake from free- living individuals is important for researchers who investigate the nutritional adequacy of the diet and interventions to modify food intake and body weight. Additionally, the RFPM allows individuals to receive virtually immediate feed- back about their energy and micro- and macronutrient intake from professionals in a remote location. Individuals can obtain clinical and dietary recommendations based on food intake data that are reviewed in near real time, and they do not need to go to a clinic to receive this feedback. The RFPM is a promising method to estimate EI with inno- vative technology. Nevertheless, our experiences during these studies remind us of the complexities of collecting EI data from free-living individuals and the barriers that negatively affect data quality, as described in the following paragraphs. Therefore, the RFPM continues to be modified to address these barriers, and these modifications include the following. First, the RFPM relies on the ability of participants to remember to take photographs of their food selection and plate waste. The use of EMA methodology provided auto- mated prompts reminding participants to take photographs, but missing data are unavoidable. To reduce missing data, it is clear that secondary methods are needed to capture EI data when the participant forgets to take photographs or when photographing food is not possible. In current and future studies, we include explicit instructions for participants to record food intake with pen and paper or to use the phone to make voice recordings describing the food and portion size. Additionally, we monitor incoming data and contact partici- pants immediately if their data are of poor quality or if photo- graphs are missing. Participants have responded positively to these procedures, since the procedures indicate that the researchers are invested in data collection and that participants are accountable. Second, participants’ food descriptions do not always convey important details about the food item, such as the method of preparation. Therefore, when participants return the cell phone to the researchers, they answer questions about foods that are difficult to identify and that have poor descriptions. During this meeting, the participant also provides the researchers with pen-and-paper records or voice recordings that were used to record EI in the case of technology failure. Third, a considerable amount of personnel time is required to schedule EMA prompts, monitor incoming data, etc. There- fore, the methodology is being further modified and computer applications are being developed to: (1) automatically deliver EMA prompts and modify the prompts based on the partici- pants’ adherence; (2) monitor incoming data and alert the researcher and participant when data appear to be missing, for example, when an odd number of photographs are received; (3) digitally enhance the quality and lighting of photographs; (4) automatically estimate the amount of food represented in pictures using computer imaging algorithms. These aims are part of ongoing research and development to facilitate data collection in free-living conditions where par- ticipants eat ad libitum and to minimise under-recording or participants failing to take and send photographs. Also, the computer imaging techniques being developed will determine the angle and distance of the camera phone from the food, which will facilitate accurate estimates of EI even when foods are eaten off plates that vary in size. To accomplish this aim, participants will include a standard-size object in the photograph, for example, a dollar bill or six-inch ruler. An additional aim of the ongoing research is to determine if participants under-eat when they use the RFPM. Under- eating is known to occur when food intake is monitored using other methods, for example, Goris et al. (12) . Strengths and limitations. The study has important strengths, including a diverse study sample and excellent retention (only two participants failed to complete the protocol). Additionally, participants considered under-repor- ters were not excluded from the data analysis, which is common in other studies. Last, the RFPM was tested in controlled laboratory and free-living conditions in the same sample (the take-out group), which allows direct comparisons of the accuracy between laboratory and free-living conditions. The study has important limitations that also must be con- sidered. First, participants in the take-out group of the main study collected data during free-living conditions, though they ate food from a cooler provided by the researchers. Consequently, the variety of food was limited and not necess- arily representative of their habitual daily intake. Although these conditions were contrived, they were necessary since other criterion measures of EI during free-living conditions, for example, DLW, were not feasible for this study. Second, EI was only measured for 3 d. Fig. 4. Bland and Altman analysis comparing the remote food photography method (RFPM) with weighed energy intake (EI) in free-living conditions by analysing the dinner data from the take-out group. The RFPM bias in estimating EI was consistent over different levels of EI (R 2 0·08; adjusted R 2 0·04; P¼0·18). C. K. Martin et al.454 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 Conclusions The studies reported here describe the development and initial support for the reliability and validity of the RFPM, as well as high user satisfaction ratings. Ongoing research is refining the methodology and testing the RFPM in free- living conditions against EI measured with DLW. Our find- ings suggest that the RFPM is a promising new method for estimating EI in free-living conditions. Acknowledgements This work was partially supported by the National Institutes of Health and the National Science Foundation (National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) grant 1 K23 DK068052-01A2; National Heart Lung and Blood Institute, National Institute on Aging, and National Science Foundation grant 1 R21 AG032231-01) and a Clinical Nutrition Research Unit Center grant (1P30 DK072476) entitled ‘Nutritional Programming: Environmen- tal and Molecular Interactions’ sponsored by the NIDDK. The content is solely the responsibility of the authors and does not necessarily represent the official view of the National Institutes of Health. There are no conflicts of inter- est associated with the study. C. K. M. was the study principal investigator and laboratory director; H. H. conducted the statistical analyses; S. M. C. was the laboratory manager who collected and managed data, H. R. A. developed the digital photography computer application and data collection software; C. C. supervised the dietary aspects of the protocol; S. D. A. was involved in study design, collection of data, and results interpretation. References 1. Livingstone MB & Black AE (2003) Markers of the validity of reported energy intake. J Nutr 133, Suppl. 3, 895S–920S. 2. Schoeller DA (1990) How accurate is self-reported dietary energy intake? Nutr Rev 48, 373–379. 3. de Jonge L, DeLany JP, Nguyen T, Howard J, Hadley EC, Redman LM & Ravussin E (2007) Validation study of energy expenditure and intake during calorie restriction using doubly labeled water and changes in body composition. Am J Clin Nutr 85, 73–79. 4. O’Neil PM (2001) Assessing dietary intake in the management of obesity. Obes Res 9, Suppl. 5, 361S–366S, 373S–374S. 5. Pikholz C, Swinburn B & Metcalf P (2004) Under-reporting of energy intake in the 1997 National Nutrition Survey. N Z Med J 117, U1079. 6. Samuel-Hodge CD, Fernandez LM, Henriquez-Roldan CF, Johnston LF & Keyserling TC (2004) A comparison of self-reported energy intake with total energy expenditure estimated by accelerometer and basal metabolic rate in African-American women with type 2 diabetes. Diabetes Care 27, 663–669. 7. Hendrickson S & Mattes R (2007) Financial incentive for diet recall accuracy does not affect reported energy intake or number of underreporters in a sample of overweight females. J Am Diet Assoc 107, 118–121. 8. Beaton GH, Burema J & Ritenbaugh C (1997) Errors in the interpretation of dietary assessments. Am J Clin Nutr 65, 1100S–1107S. 9. Tran KM, Johnson RK, Soultanakis RP & Matthews DE (2000) In-person vs telephone-administered multiple-pass 24-hour recalls in women: validation with doubly labeled water. J Am Diet Assoc 100, 777–783. 10. Schoeller DA, Bandini LG & Dietz WH (1990) Inaccuracies in self-reported intake identified by comparison with the doubly labeled water method. Can J Physiol Pharmacol 68, 941–949. 11. Bandini LG, Schoeller DA, Cyr HN & Dietz WH (1990) Validity of reported energy intake in obese and nonobese adolescents. Am J Clin Nutr 52, 421–425. 12. Goris AH, Westerterp-Plantenga MS & Westerterp KR (2000) Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake. Am J Clin Nutr 71, 130–134. 13. RennieKL,CowardA&JebbSA(2007)Estimatingunder-reporting of energy intake in dietary surveys using an individualised method. Br J Nutr 97, 1169–1176. 14. Koebnick C, Wagner K, Thielecke F, et al. (2005) An easy- to-use semiquantitative food record validated for energy intake by using doubly labelled water technique. Eur J Clin Nutr 59, 989–995. 15. Beasley J, Riley WT & Jean-Mary J (2005) Accuracy of a PDA-based dietary assessment program. Nutrition 21, 672–677. 16. Yon BA, Johnson RK, Harvey-Berino J, Gold BC & Howard AB (2007) Personal digital assistants are comparable to tra- ditional diaries for dietary self-monitoring during a weight loss program. J Behav Med 30, 165–175. 17. Yon BA, Johnson RK, Harvey-Berino J & Gold BC (2006) The use of a personal digital assistant for dietary self-monitoring does not improve the validity of self-reports of energy intake. J Am Diet Assoc 106, 1256–1259. 18. Blake AJ, Guthrie HA & Smiciklas-Wright H (1989) Accuracy of food portion estimation by overweight and normal-weight subjects. J Am Diet Assoc 89, 962–964. 19. Martin CK, Anton SD, York-Crowe E, Heilbronn LK, VanSkiver C, Redman LM, Greenway FL, Ravussin E, Williamson DA & for the Pennington CALERIE Team (2007) Empirical evaluation of the ability to learn a calorie counting system and estimate portion size and food intake. Br J Nutr 98, 439–444. 20. Comstock EM, St Pierre RG & Mackiernan YD (1981) Measur- ing individual plate waste in school lunches. Visual estimation and children’s ratings vs. actual weighing of plate waste. J Am Diet Assoc 79, 290–296. 21. Wolper C, Heshka S & Heymsfield SB (1995) Measuring food intake: an overview. In Handbook of Assessment of Methods for Eating Behaviors and Weight-related Problems, pp. 215–240 [D Allison, editor]. Thousand Oaks, CA: Sage Publications. 22. Williamson DA, Allen HR, Martin PD, Alfonso A, Gerald B & Hunt A (2004) Digital photography: a new method for estima- ting food intake in cafeteria settings. Eat Weight Disord 9, 24–28. 23. Williamson DA, Allen HR, Martin PD, Alfonso AJ, Gerald B & Hunt A (2003) Comparison of digital photography to weighed and visual estimation of portion sizes. J Am Diet Assoc 103, 1139–1145. 24. United States Department of Agriculture & Agricultural Research Service (2000) Continuing Survey of Food Intakes by Individuals, 1994 – 1996, 1998. Washington, DC: USDA & ARS. 25. Martin CK, Newton RL Jr, Anton SD, Allen HR, Alfonso A, Han H, Stewart T, Sothern M & Williamson DA (2007) Measurement of children’s food intake with digital photography and the effects of second servings upon food intake. Eat Behav 8, 148–156. 26. Wang DH, Kogashiwa M & Kira S (2006) Development of a new instrument for evaluating individuals’ dietary intakes. J Am Diet Assoc 106, 1588–1593. Remote food photography method 455 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438 27. Wang DH, Kogashiwa M, Ohta S & Kira S (2002) Validity and reliability of a dietary assessment method: the application of a digital camera with a mobile phone card attachment. J Nutr Sci Vitaminol (Tokyo) 48, 498–504. 28. Kikunaga S, Tin T, Ishibashi G, Wang DH & Kira S (2007) The application of a handheld personal digital assistant with camera and mobile phone card (Wellnavi) to the general popu- lation in a dietary survey. J Nutr Sci Vitaminol (Tokyo) 53, 109–116. 29. Stone AA & Shiffman S (1994) Ecological momentary assess- ment (EMA) in behavioral medicine. Ann Behav Med 16, 199–202. 30. Bland JM & Altman DG (1986) Statistical methods for asses- sing agreement between two methods of clinical measurement. Lancet i, 307–310. 31. Martin CK, Williamson DA, Geiselman PJ, Walden H, Smeets M, Morales S & Redmann S Jr (2005) Consistency of food intake over four eating sessions in the laboratory. Eat Behav 6, 365–372. C. K. Martin et al.456 B ri ti sh Jo u rn al o f N u tr it io n D o w n lo ad ed fro m h ttp s://w w w .cam b rid g e.o rg /co re . C arn eg ie M ello n U n iversity , o n 06 A p r 2021 at 01:16:43 , su b ject to th e C am b rid g e C o re term s o f u se, availab le at h ttp s://w w w .cam b rid g e.o rg /co re/term s . h ttp s://d o i.o rg /10.1017/S0007114508027438 https://www.cambridge.org/core https://www.cambridge.org/core/terms https://doi.org/10.1017/S0007114508027438