Establishing Inter-Rater Reliability for a Plate Waste Study used to Determining the Relationship Between Elementary Students’ Consumption of School Lunches and Lunch Table Time Haley Billings Catherine Millburg, BS Gail Gates, PhD, RD Undergraduate Student – Nutritional Sciences Graduate Student – Nutritional Sciences Department of Nutritional Sciences Oklahoma state University Oklahoma State University Oklahoma State University Nancy Betts, PhD, RD Deana Hildebrand, PhD, RD/LD, SNS Department of Nutritional Sciences Department of Nutritional Sciences Oklahoma State University Oklahoma State University

Background Nutrition Concern •  The National School Lunch Program is designed to provide students with one-third of the Dietary Reference Intakes (USDA, 2012). •  The Report of the Dietary Guidelines Advisory Committee on the Dietary Guidelines for Americans, 2010, acknowledged vitamin A, vitamin C, calcium, and fiber as shortfall nutrients in the American population, with calcium and fiber being identified as nutrients of concern (USDA, 2010). Table Time •  In an analysis of 3 studies, Conklin et al., (2002) concluded that lunch periods be scheduled for 30 minutes including travel time, waiting in line, eating and bussing the dishes. Further, students need approximately 20 minutes at the table to eat at a comfortable pace. •  Plate waste studies found that students’ consumption of the meal was higher in schools with a 30 minute lunch period versus a 20 minute lunch period (Bergman et al., 2004). •  The School Health Policies and Program Study of 2000 revealed that 19.7% of schools provided students less than 20 minutes to eat once they have been seated (CDC, 2000). Plate Waste •  Plate waste studies play a key role in measuring the impacts of food behavior interventions (Hanks, 2013). Inter-Rater Reliability •  The Cohen Kappa Method was developed to account for the possibility that raters actually guess on at least some variables due to uncertainty (McHugh, 2012). •  The kappa can range from -1 to +1 (McHugh, 2012). •  Values of Kappa from 0.40 to 0.59 are considered moderate, 0.60 to 0.79 substantial, and 0.80 outstanding (Landis & Koch, 1977).

Results Inter-Rater Reliability •  The Kappa value was 0.81, p < .001. Table Time •  Students who exited the lunch line first had significantly more table time than students who exited the line last (p < 0.001). •  Students with the most table time consumed more vitamin C, calcium, fiber, and calories.

Pre Photograph

Discussion

Purpose

The purpose of the study was to evaluate if 1) if the results from the plate waste study were reliable 2) if the Cohen's Kappa inter-rater reliability test is useful for further studies.

Pre Photograph

Methods Study Setting •  The study was conducted in an urban elementary school with 2nd, 3rd and 4th grade students. •  The school cafeteria had two identical food service lines. •  The study was approved by the OSU Institutional Review Board. Plate Waste Analysis •  The first 5 students and last 5 students of each grade group were asked if a photograph could be taken of their tray. If agreeing, a numbered card was placed on the tray and the student was allowed to eat as usual. •  As the students were dismissed, those with a numbered card were instructed to place their tray on the stage area where a second picture was taken and milk weighed with a digital scale. The weight of the milk was recorded on the numbered card. (see photos) Inter-Rater Reliability •  Consumption analysis consisted of determining the proportion of the each menu item consumed (none, ¼, ½, all). •  A random sample of paired pre and post photographs was analyzed by a rater who recorded the amount of each individual food item that was consumed. •  A second rater analyzed the same photographs and recorded the amount consumed from the random samples according to the same scale (none, ¼, ½, all). •  The two sets of data recorded from each rater was compared using the Cohen's Kappa interrater reliability test in SPSS to determine the rate of reliability.

Post Photograph

Inter-Rater Reliability •  The inter-rater reliability was outstanding meaning 1) the data from the first rater was reliable and 2) the two raters had very good compatibility. Table Time •  While none of the students, on average, were provided table time to meet the 20 minute recommendation, they were consuming adequate amounts of vitamin A. This may be because the entrée items on observation days contained tomato product (e.g., spaghetti, BBQ chicken sandwich) as an ingredient. •  Table time was positively associated with fiber, however none of the students met the 1/3 Average Intake provided by the school lunch for fiber. Fruits and vegetables are a primary source of fiber in the school meal. •  Table time did not seem to positively affect the amount of calories or calcium consumed by the students. Further, none of the students ate enough of the meal to meet the 1/3 RDA for calories or calcium provided by the meal.

Conclusion Inter-Rater Reliability •  Once outstanding reliability is established with the Cohen's Kappa inter-rater reliability test, results can be split amongst raters with compatibility. •  Cohen's kappa inter-rater reliability test can be used to insure the reliability of an individual or groups opinion of variables that are open to interpretation. Table Time •  Students that receive more table time during lunch consumer more calcium, vitamin C, fiber, and calories. Plate Waste •  Plate waste studies can be used to determine the nutrients that are being consumed by a certain population in settings such as schools, nursing homes, and hospitals. •  Plate waste results can be compared before and after changes are made to see if there has been in increase in nutrients consumed or a decrease in waste.

Implications for Practice and Future Research •  Increase research project reliability in fields where variables do not have certain intervals. •  Determine the amount consumed for future plate waste studies. Post Photograph