Implementation of a chatbot based on artificial intelligence language model to respond to frequently asked questions from university students

Authors

DOI:

https://doi.org/10.51252/rcsi.v3i2.570

Keywords:

conversational agent, virtual assistant, university care, GPT, student support

Abstract

The use of virtual assistants is common in the service processes of different universities around the world. Nowadays, several artificial intelligence tools such as natural language processors are offered for a more efficient development of this service. This article presents the implementation and usability evaluation of a chatbot based on the GPT-3.5-Turbo artificial intelligence language model for the attention of first semester students of Systems Engineering at the National University of Trujillo. The research design was pre-experimental and the chatbot usability questionnaire or CUQ was applied as a research instrument. The results obtained were that the chatbot presents an acceptable level of perceived usability and 93% of students agreed that the answers provided by the virtual assistant were useful. The conversational agent stood out in ease of use and navigation, and also exhibited problems in understanding the questions asked by the user and providing answers in a natural way.

References

Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776

Barus, S. P., & Surijati, E. (2022). Chatbot with Dialogflow for FAQ Services in Matana University Library. International Journal of Informatics and Computation, 3(2), 62. https://doi.org/10.35842/ijicom.v3i2.43

Busch, K., Rochlitzer, A., & Diana Sola, H. L. (2023). Just Tell Me: Prompt Engineering in Business Process Management. Computer Science, 1. https://arxiv.org/abs/2304.07183

Casazola Cruz, O. D., Alfaro Mariño, G., Burgos Tejada, J., & Ramos More, O. A. (2021). La usabilidad percibida de los chatbots sobre la atención al cliente en las organizaciones: una revisión de la literatura. Interfases, 014, 184–204. https://doi.org/10.26439/interfases2021.n014.5401

Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. http://arxiv.org/abs/2303.10130

Holmes, S., Moorhead, A., Bond, R., Zheng, H., Coates, V., & Mctear, M. (2019). Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces? Proceedings of the 31st European Conference on Cognitive Ergonomics, 207–214. https://doi.org/10.1145/3335082.3335094

Hsu, C.-L., & Lin, J. C.-C. (2023). Understanding the user satisfaction and loyalty of customer service chatbots. Journal of Retailing and Consumer Services, 71, 103211. https://doi.org/10.1016/j.jretconser.2022.103211

Jimenez Flores, V. J. (2019). Entidad conversacional de inteligencia artificial y calidad del servicio percibido por estudiantes de la Universidad José Carlos Mariátegui Filial Tacna, 2018 – II. [Universidad Nacional Jorge Basadre Grohmann]. http://repositorio.unjbg.edu.pe/handle/UNJBG/3848

Jordan, P. W., Thomas, B., McClelland, I. L., & Weerdmeester, B. (1996). Usability Evaluation In Industry (1st ed.). CRC Press.

Kumar, H., Musabirov, I., Shi, J., Lauzon, A., Choy, K. K., Gross, O., Kulzhabayeva, D., & Williams, J. J. (2022). Exploring The Design of Prompts For Applying GPT-3 based Chatbots: A Mental Wellbeing Case Study on Mechanical Turk. http://arxiv.org/abs/2209.11344

Kumar, M. (2019). Serverless Architectures Review, Future Trend and the Solutions to Open Problems. American Journal of Software Engineering, 6(1), 1–10. https://doi.org/10.12691/ajse-6-1-1

Lee, K., Jo, J., Kim, J., & Kang, Y. (2019). Can Chatbots Help Reduce the Workload of Administrative Officers? - Implementing and Deploying FAQ Chatbot Service in a University (pp. 348–354). https://doi.org/10.1007/978-3-030-23522-2_45

Lewis, J. R., & Sauro, J. (2018). Item Benchmarks for the System Usability Scale. Journal of Usability Studies, 13(3), 158–167. https://uxpajournal.org/wp-content/uploads/sites/7/pdf/JUS_Lewis_May2018.pdf

Lin, Z., Xu, P., Winata, G. I., Siddique, F. Bin, Liu, Z., Shin, J., & Fung, P. (2020). CAiRE: An End-to-End Empathetic Chatbot. Proceedings of the AAAI Conference on Artificial Intelligence, 34(09), 13622–13623. https://doi.org/10.1609/aaai.v34i09.7098

Madhan Kumar, C., Fardeen, M., Rohit, P., & Rakhes, M. (2023). AI based chatbot to answer faqs. Interantional Journal of Scientific Research in Engineering and Management, 7(3). https://doi.org/10.55041/IJSREM18072

Mariani, M. M., Hashemi, N., & Wirtz, J. (2023). Artificial intelligence empowered conversational agents: A systematic literature review and research agenda. Journal of Business Research, 161, 113838. https://doi.org/10.1016/j.jbusres.2023.113838

Mohammed, C. M., & Zeebaree, S. R. . (2021). Sufficient Comparison Among Cloud Computing Services: IaaS, PaaS, and SaaS: A Review. International Journal of Science and Business, 5(2), 17–30. https://ideas.repec.org/a/aif/journl/v5y2021i2p17-30.html

Nugraha, K. A., & Sebastian, D. (2021). Proceedings of the 7th International Conference on Education and Technology (ICET 2021). Proceedings of the 7th International Conference on Education and Technology, 601, 119–122. https://www.atlantis-press.com/proceedings/icet-21/125964506

Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003

Reddy K, G. K., Pal, A., V, S. K., J, R., & K, S. (2022). Cross Domain Answering FAQ Chatbot. 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA), 1–4. https://doi.org/10.1109/ICACTA54488.2022.9752986

Sánchez, W. O. (2011). Logotipo del repositorio Comunidades Iniciar sesión Inicio Revistas y libros Revista Ing-Novación No. 02, agosto 2011 La usabilidad en Ingeniería de software: definición y características La usabilidad en Ingeniería de software: definición y característic. Revista Ing-Novación, 2, 7–22. http://hdl.handle.net/11715/519

Santana, R., Ferreira, S., Rolim, V., Miranda, P., Nascimento, A., & Mello, R. F. (2021). A Chatbot to Support Basic Students Questions. CEUR Workshop Proceedings, 3059, 58–67. http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-3059/paper6.pdf

Saranya, A., & Subhashini, R. (2023). A systematic review of Explainable Artificial Intelligence models and applications: Recent developments and future trends. Decision Analytics Journal, 7, 100230. https://doi.org/10.1016/j.dajour.2023.100230

Shaji George, A., & Sagayarajan, S. (2023). Securing Cloud Application Infrastructure : Understanding the Penetration Testing Challenges of IaaS , PaaS , and SaaS Environments Partners Universal International Research Journal ( PUIRJ ). Partners Universal International Research Journal, March, 24–34. https://doi.org/10.5281/zenodo.7723187

Sotnik, S., Shakurova, T., & Lyashenko, V. (2023). Development Features Web-Applications. International Journal of Academic and Applied Research, 7(1), 79–85. https://openarchive.nure.ua/handle/document/21600

Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. http://arxiv.org/abs/2102.02503

Wang, X., Lin, X., & Shao, B. (2022). How does artificial intelligence create business agility? Evidence from chatbots. International Journal of Information Management, 66, 102535. https://doi.org/10.1016/j.ijinfomgt.2022.102535

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. http://arxiv.org/abs/2302.11382

Wolde, B. G., & Boltana, A. S. (2021). REST API Composition for Effective Testing the Cloud. Journal of Applied Research and Technology, 19(6), 676–693. https://doi.org/10.22201/icat.24486736e.2021.19.6.924

RCSI

Published

2023-07-10

How to Cite

Espinosa-Luna, B. H. ., Castillo-Oliva, J., Montañez-Díaz, B. A., & Mendoza-De-los-Santos, A. . (2023). Implementation of a chatbot based on artificial intelligence language model to respond to frequently asked questions from university students. Revista Científica De Sistemas E Informática, 3(2), e570. https://doi.org/10.51252/rcsi.v3i2.570

Most read articles by the same author(s)