Understanding artificial intelligence chatbot quality and experience: A higher education student perspective - Ines Dužević; Tomislav Baković; Vivien Surman (Entrepreneurial Business and Economics Review (EBER), 2025)
Objective: The article aims to identify the key aspects that define service quality for artificial intelligence chatbots (AICB) in higher education, based on insights from students. The second objective is to put AICB quality into the broader context of other key variables associated with student experience of AICB, such as AICB adoption, AICB usability, AICB engagement, and AICB mistrust. Research Design & Methods: Based on extant service quality research and established scale development techniques, the study constructs, refines, and validates a multidimensional AICB service quality scale through a series of pilot and validation studies. The article includes both qualitative and quantitative technics, as we developed a questionnaire based on a literature review and 48 mini focus group interviews. In total, 308 participants filled out the questionnaire. For the analyses, we applied both exploratory and confirmatory factor analysis together with scale validation and correlation analysis. Findings: We began the AICB service quality scale with 27 items across five dimensions: AICB quality, AICB mistrust, AICB usability, AICB adoption, and AICB engagement. The final scale consisted of 15 items across four dimensions with only AI engagement left out. Data analysis emphasised the critical role of AI quality in AI usability and AI adoption. The research also confirmed AI mistrust is an important aspect with a negative connection to AI quality. Implications & Recommendations: The study results have several theoretical and practical implications. From the theoretical standpoint, we confirmed that the quality of artificial intelligence (AI) plays a central role in forming student experience. Quality of AICB received the highest score in this analysis (5.03) while AICB mistrust scored lowest (3.58). On the other hand, when it comes to individual correlations between student experience elements and AICB quality, mistrust in AICB shows a negative correlation with the highest score (-0.48). Use and adoption are both connected to AICB quality in a positive way. Results show us there is room for improvement in both AICB quality and student experience since average scores were in the range of 4.5-5.0. The results also emphasised the importance of reducing AICB mistrust for improving AICB quality and overall experience. Contribution & Value Added: The AICB quality scale facilitates theory development by providing a reliable scale to improve the current understanding of student perceptions regarding different aspects of AICB quality. Higher education institutions (HEI) can use the study results to understand the impact of new technologies such as AICB on student experiences.
Keywords: C38; I23; 330; AICB experience; AICB quality; artificial intelligence (AI); artificial intelligence chatbots (AICB); chatbots; higher education