Designing a Chatbot to Enhance Reading Comprehension Skills Simulating the PIRLS Test in Light of User-Centered Design (UCD) Theory
Abstract
This study aimed to develop a chatbot application based on the User-Centered Design theory (UCD) for training reading literacy skills targeted in the International Study of (PIRLS), and to examine the role of usability principles and user experience in improving the development and usage of the application. The study tools included questionnaires, interviews, system usability scale (SUS), and user experience questionnaire (UEQ). The study found the need to intensify innovative training programs to develop reading literacy skills in fourth-grade female students, and that designing the chatbot application according to the User-Centered Design theory enhanced the employment of usability principles and user experience based on feedback from users and their context of application usage. Based on the study's results, several recommendations were made, the most important of which are the careful use of artificial intelligence techniques based on cognitive, social, and cultural foundations to enhance reading literacy skills, the use of User-Centered Design theory in the development of educational chatbot applications, and the development of instructional designers' and developers' skills in designing digital educational products according to usability principles and user experience objectives.
References
2. Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications, 37(4), 445–456.
3. Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, 100006.
4. Addey, C., & Sellar, S. (2020). The rise of international large-scale assessments and rationales for participation. In The Legacy of Jullien’s Work for Comparative Education (pp. 132–150). Routledge.
5. Ali, A. M., Khamaj, A., Kang, Z., Moosa, M., & Alam, M. M. (2023). User-centered design (UCD) of time-critical weather alert application. International Journal of Advanced Computer Science and Applications, 14(1).
6. Andre, Andre & Dinata, Hendra. (2018). Interaction Design to Enhance UX of University Timetable Plotting System on Mobile Version. IOP Conference Series: Materials Science and Engineering. 407. 012174. 10.1088/1757-899X/407/1/012174. Retrieved from http://repository.ubaya.ac.id/33598/
7. Applying a user-centered design methodology to develop usable interfaces for an Automated Teller Machine | Proceedings of the XVIII International Conference on Human Computer Interaction. (n.d.). Retrieved June 17, 2023, from https://dl.acm.org/doi/abs/10.1145/3123818.3123833
8. Bahja, M., Hammad, R., & Butt, G. (2020). A user-centric framework for educational chatbots design and development. HCI International 2020-Late Breaking Papers: Multimodality and Intelligence: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22, 32–43.
9. Belda-Medina, J., & Calvo-Ferrer, J. R. (2022). Using Chatbots as AI Conversational Partners in Language Learning. Applied Sciences, 12(17), Article 17. https://doi.org/10.3390/app12178427
10. Bodea, C.-N., Dascalu, M.-I., & Hang, A. (2021). Chatbot-Based Training for Project Management: Another Way of Corporate Training or a Must-Have Tool for Sustainable Education? Research on Project, Programme and Portfolio Management: Integrating Sustainability into Project Management, 249–259.
11. Chen, L., Chen, P., & Lin, Z. (2020). Artificial Intelligence in Education: A Review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
12. Chocarro Eguaras, R., Cortiñas Ugalde, M., & Marcos Matas, G. (2021). Teachers’ attitudes towards chatbots in education: A technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. https://doi.org/10.1080/03055698.2020.1850426
13. Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539–548.
14. Crompton, H., Burke, D., & Lin, Y. C. (2019). Mobile learning and student cognition: A systematic review of PK‐12 research using Bloom’s Taxonomy. British Journal of Educational Technology, 50(2), 684-701.
15. Cunningham-Nelson, S., Boles, W., Trouton, L., & Margerison, E. (2019). A review of chatbots in education: Practical steps forward. 30th Annual Conference for the Australasian Association for Engineering Education (AAEE 2019): Educators Becoming Agents of Change: Innovate, Integrate, Motivate, 299–306.
16. Dore, R. A., Amendum, S. J., Golinkoff, R. M., & Hirsh-Pasek, K. (2018). Theory of Mind: A Hidden Factor in Reading Comprehension? Educational Psychology Review, 30(3), 1067–1089. https://doi.org/10.1007/s10648-018-9443-9
17. Elia Fraoua, K., Leblanc, J.-M., & David, A. (2020). Use of an Emotional Chatbot for the Analysis of a Discussion Forum for the Improvement of an E-Learning Platform. In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. Human and Technology Ecosystems (pp. 25–35). Springer International Publishing. https://doi.org/10.1007/978-3-030-50506-6_3
18. Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022a). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6
19. Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022b). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6
20. Galkienė, A., & Monkevičienė, O. (2021). Preconditions of Transforming the Educational Process by Applying Inclusive Education Strategies: Theoretical Background. In A. Galkienė & O. Monkevičienė (Eds.), Improving Inclusive Education through Universal Design for Learning (pp. 1–21). Springer International Publishing. https://doi.org/10.1007/978-3-030-80658-3_1
21. Giuliani, M., Szczęśniak-Stańczyk, D., Mirnig, N., Stollnberger, G., Szyszko, M., Stańczyk, B., & Tscheligi, M. (2020). User-centred design and evaluation of a tele-operated echocardiography robot. Health and Technology, 10(3), 649–665. https://doi.org/10.1007/s12553-019-00399-0
22. Guney, Z. (2019). Considerations for Human-Computer Interaction: User Interface Design Variables and Visual Learning in IDT. Cypriot Journal of Educational Sciences, 14(4), 731–741.
23. Helander, M. G. (2014). Handbook of human-computer interaction. Elsevier.
24. Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257.
25. Human Computer Interaction | IEEE Conference Publication | IEEE Xplore. (n.d.). Retrieved June 17, 2023, from https://ieeexplore.ieee.org/document/5698279
26. Hwang, G.-J., & Chang, C.-Y. (2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 1–14.
27. Lin, M. P., & Chang, D. (2020). Enhancing Post-secondary Writers’ Writing Skills with a Chatbot: A Mixed-Method Classroom Study. J. Educ. Technol. Soc. https://www.semanticscholar.org/paper/Enhancing-Post-secondary-Writers'-Writing-Skills-a-Lin-Chang/ef7e34011f020c1175a6b1d87d2b0f3cf31be773
28. Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576.
29. Liu, C.-C., Liao, M.-G., Chang, C.-H., & Lin, H.-M. (2022). An analysis of children’interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576.
30. Mullis, I. V., & Martin, M. O. (2019). PIRLS 2021 Assessment Frameworks. ERIC.
31. Navaitienė, J., & Stasiūnaitienė, E. (2021). The goal of the universal design for learning: Development of all to expert learners. In Improving inclusive education through universal design for learning (pp. 23–57). Springer International Publishing Cham.
32. Neo, M., & Mahendru, N. (2021). SCAFFOLDING LEARNING WITH MERLIN: MALAYSIAN STUDENTS’PERCEPTIONS OF AN AI CHATBOT. EDULEARN21 Proceedings, 4260–4265.
33. Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033.
34. Ondáš, S., Pleva, M., & Hládek, D. (2019). How chatbots can be involved in the education process. 2019 17th International Conference on Emerging Elearning Technologies and Applications (ICETA), 575–580.
35. Palasundram, K., Sharef, N. M., Nasharuddin, N., Kasmiran, K., & Azman, A. (2019). Sequence to sequence model performance for education chatbot. International Journal of Emerging Technologies in Learning (IJET), 14(24), 56–68.
36. Pin-Chuan Lin, M., & Chang, D. (2020). Enhancing Post-secondary Writers’ Writing Skills with a Chatbot: A Mixed-Method Classroom Study. Journal of Educational Technology & Society, 23(1).
37. Preece, J., Sharp, H., & Rogers, Y. (2015). Interaction design: Beyond human-computer interaction. John Wiley & Sons.
38. Rogers, Y. (2012). HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centered Informatics, 5(2), 1–129. https://doi.org/10.2200/S00418ED1V01Y201205HCI014
39. Sequence to Sequence Model Performance for Education Chatbot—Learning & Technology Library (LearnTechLib). (n.d.). Retrieved June 15, 2023, from https://www.learntechlib.org/p/217029/
40. Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
41. Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for Bezverhny, E., Dadteev, K., Barykin, L., Nemeshaev, S., & Klimov, V. (2020). Use of chat bots in the Facebook Messenger. Computers & Education, 151, 103862.
42. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1016/0364-0213(88)90023-7
43. Tamayo, P. A., Herrero, A., Martín, J., Navarro, C., & Tránchez, J. M. (2020). Design of a chatbot as a distance learning assistant. Open Praxis, 12(1), 145–153.
44. Twomlow, A., Grainger, S., Cieslik, K., Paul, J. D., & Buytaert, W. (2022). A user-centred design framework for disaster risk visualisation. International Journal of Disaster Risk Reduction, 77, 103067. https://doi.org/10.1016/j.ijdrr.2022.103067
45. Vanichvasin, P. (2021). Chatbot Development as a Digital Learning Tool to Increase Students’ Research Knowledge. International Education Studies, 14(2), 44–53.
46. Wu, E. H.-K., Lin, C.-H., Ou, Y.-Y., Liu, C.-Z., Wang, W.-K., & Chao, C.-Y. (2020). Advantages and Constraints of a Hybrid Model K-12 E-Learning Assistant Chatbot. IEEE Access, 8, 77788–77801. https://doi.org/10.1109/ACCESS.2020.2988252
47. Wu, E., Lin, C.-H., Ou, Y. Y., Liu, C.-Z., Wang, W.-K., & Chao, C.-Y. (2020). Advantages and Constraints of a Hybrid Model K-12 E-Learning Assistant Chatbot. IEEE Access, PP, 1–1. https://doi.org/10.1109/ACCESS.2020.2988252
48. Yang, G., Badri, M., Al Rashedi, A., & Almazroui, K. (2018). The role of reading motivation, self-efficacy, and home influence in students’ literacy achievement: A preliminary examination of fourth graders in Abu Dhabi. Large-Scale Assessments in Education: An IEA-ETS Research Institute Journal, 6(1). https://doi.org/10.1186/s40536-018-0063-0
Copyright (c) 2025 د. فدوى ياسين فلمبان , د. نجلاء محمد بن عايض العمري , إيمان ع بده حسن عوض , أفنان عبدالله أحمد الغامدي , دينا محمد محمود الأزوري

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.