Research Article

Accuracy of Responses to Public Perception Questions of the Dairy Industry using ChatGPT

George R Gallagher1* and Tyler J Gallagher²

¹Department of Animal Science California Polytechnic State University, San Luis Obispo, CA, USA

²Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA

Submission: November 18, 2024;Published: November 26, 2024

*Corresponding author: George R Gallagher, Department of Animal Science California Polytechnic State University, San Luis Obispo, CA, USA.

How to cite this article: Gallagher, G.R. and T.J. Gallagher. Accuracy of Responses to Public Perception Questions of the Dairy Industry using ChatGPT. Arch Anim Poult Sci. 2024; 2(5): 555600.. DOI: 10.19080/AAPS.2024.02.555600

Abstract

Artificial intelligence (AI) and chat bots, such as Chatgpt provide an avenue for consolidating information from the internet. This study’s objective was to determine the quality of responses generated by Chatgpt to ten questions about the dairy industry. Accuracy and completeness of Chatgpt responses to each question were assessed by dairy industry professionals (n=25) on a Likert scale (1-5). Across all questions the accuracy of Chatgpt information averaged 46.0%, and completeness of each response 47.0%. This study suggests that current chat bot programming does not provide accurate or complete responses to dairy related questions.

Keywords: Accuracy; Chatgpt; Completeness; Dairy Industry; Education

Introduction

Potential applications of artificial intelligence (AI) for use in food production systems [1] including the dairy industry [2] continue to expand. While most examples involve production systems, use of AI for agricultural extension and educational purposes has been proposed [3]. At the current time, AI approaches are reported to be incapable of identification of disinformation and misinformation when generating a response to a given question [4]. The objective of this study was to determine the accuracy and completeness of responses generated by Chatgpt to10 questions containing somewhat negative public perceptions of the dairy industry, derived from common topic searches on the internet. Assessments of accuracy and completeness of Chatgpt responses were based on a survey given to dairy industry professionals.

Materials and Methods

This study was a cross-sectional survey-based investigation. Ten questions portraying somewhat negative public perceptions of the dairy industry were developed based upon google searches of common dairy principles and activities on the internet. Examples of internet search words included dairy cattle humane treatment, environmental impact, cheese and milk consumption, and management practices such as artificial insemination, separating the cow and calf, and typical use of male calves. Subsequently, each query was inputted into a new Chatgpt (version 3.5) chat box, December 7, 2023, and the first generated responses were recorded.

Through utilization of REDCap data management software, an anonymous survey was created to assess the accuracy and completeness of each of the ten responses. For each query and Chatgpt response, individuals were asked to rate the accuracy and completeness of the response on a Likert scale as follows: 1) Very low accuracy/completeness, 2) Low accuracy/completeness, 3) Moderately accurate/complete, 4) Mostly accurate/complete, and 5) Very accurate/complete. A link to the survey was sent via email to dairy industry professionals (N=100) across the United States, between March 5, 2024 to April 22, 2024. Respondents were also requested to provide current career titles and the region of the country based on US Census criteria. Surveys were distributed from March 5, 2024 to April 22, 2024. Descriptive statistical analysis of surveys was completed using STATA (StataCorp, 2024, version 18, College Station, TX). Mean rating of the cumulative responses and each individual question were calculated along with 95% confidence intervals.

Among the 25 surveys completed, 48% of respondents were involved with dairy extension, 32% commercial dairy producers or processors and 20% dairy related faculty at a university. The highest proportion of respondents were in the geographic West (52.0%) followed by the Midwest (32.0%), South (8.0%), and Northeast (8.0%).

Results

Results of the survey respondents’ ratings of accuracy and completeness to the 10 dairy related questions answered by Chatgpt are presented in (Figure 1). Across all questions, the average accuracy was 2.76 (95% CI: 2.43-3.09) and an average completeness rating of 2.82 (95% CI: 2.48-3.17), based on a Likert scale of 1-5. While no differences (P<0.05) were observed across all questions, trends in the results do provide potentially useful information. The question with the highest average rating for both accuracy, 3.48 (95% CI: 3.05-3.91) and completeness, 3.40 (95% CI: 2.94-3.86) was: Is cheese from cows bad for humans? The questions with the lowest ratings included, Is the dairy industry just another factory farm? and, Are baby male calves disposable by-products of the dairy industry? The overall average on the Likert scale would suggest the accuracy of Chatgpt was 55.2%, and completeness of each response approximately 56.4%. All responses to questions were below 70% accuracy or completion. The degree of anthropomorphic or questions stated that could enhance an emotional response did not seem to alter the quality of information presented by Chatgpt on a consistent basis.

Discussion

The incorporation of artificial intelligence systems into animal production systems is rapidly evolving, with significant benefits already achieved. The use of chat bots such as Chatgpt has the potential to be useful in expanding both general and extension educational opportunities for the agriculture industries. At this time, use of such AI systems for educational purposes seems to be limited in providing what industry professionals perceive as being accurate or complete responses to given questions. With the increased use of AI and chat bots such as Chatgpt, to derive, consolidate and present information from internet sources, into what appears to be accepted information, these systems may increase the level of misinformation obtained using this methodology. While programming AI systems to provide more accurate information may be feasible; It should also be noted that the same systems have been reported to have been designed to intentionally provide misinformation [4]. Programming AI systems to provide technical information useful to producers may be readily feasible. However, recognizing the potential benefits and extensive risks of utilizing AI for public agricultural education should be considered before implementation of these systems for that purpose.

Conclusions

The use of Artificial Intelligence as a livestock extension teaching tool may be of limited value related to public perceptions of the dairy industry. Chat bots, such as Chatgpt may enhance the propagation of inaccurate information at this time. The development of Artificial Intelligence chat bots design to meet specific industry programming may be required to achieve accurate and complete information for specific animal production systems.

References

1. Javaid M, A Haleem, IH Kahn and R Suman (2023) Understanding the potential applications of artificial intelligence in the agriculture sector. Advanced Agrochem 2(1): 15-30.
2. DeVries A, N Bliznyuk and P Pinedo (2023) Invited Review: Examples and opportunities for artificial intelligence (AI) in dairy farms. Applied Animal Science 39(1): 14-22.
3. Tzachor A, Devare M, Richards C, Pypers P et al. (2023) Large language models and agricultural extension services. Nature Food 4(11): 941-948.
4. Aimeur E, S Amri and G Brassard (2023) Fake news, disinformation and misinformation in social media: a Review. Soc Netw Anal Min 13(30): 1-36.