Implementation of an AI Support Chatbot Based on Microsoft Azure OpenAI with Special Consideration of Quality

10 pages•Published: November 6, 2025

Bernd Decker, Sarah Grzemski and Ingo Hengstebeck

Abstract

This article describes the development and evaluation of an AI powered chatbot that was developed specifically to improve IT support at the IT-ServiceDesk (IT-SD) within the IT Center of RWTH Aachen University. The implementation is carried out using Microsoft Azure OpenAI and a Retrieval-Augmented Generation (RAG) approach. Given the varying complexity of support requests among different customer groups—namely, students, employees, and IT administrators—the AI chatbot will be a valuable supplement to established support channels such as email, telephone, chat, ticket portals, and personal interaction.
The article describes the key requirements for the chatbot and explains how the quality of the response is ensured by a structured feedback system. It further addresses the challenges that arise from the fact that there are two different scenarios with different requirements. On one hand, standardized and publicly available RWTH support content is provided for RWTH members as well as interested parties of the IT services and, on the other, specific internal support content for the supporters at the IT-SD. This necessitates that specific measurable quality requirements and criteria must be identified implemented and adapted.

Keyphrases: chatbot, generative ai, it support, quality management, servicedesk

In: Laurence Desnos, Raimund Vogl, Lazaros Merakos, Carmen Diaz, Janina Mincer-Daszkiewicz and Stuart Mclellan (editors). Proceedings of EUNIS 2025 annual congress in Belfast, vol 107, pages 237-246.

Links:	https://easychair.org/publications/paper/KjnC
	https://doi.org/10.29007/l6ss

BibTeX entry

@inproceedings{EUNIS2025:Implementation_AI_Support_Chatbot,
  author    = {Bernd Decker and Sarah Grzemski and Ingo Hengstebeck},
  title     = {Implementation of an AI Support Chatbot Based on Microsoft Azure OpenAI with Special Consideration of Quality},
  booktitle = {Proceedings of EUNIS 2025 annual congress in Belfast},
  editor    = {Laurence Desnos and Raimund Vogl and Lazaros Merakos and Carmen Diaz and Janina Mincer-Daszkiewicz and Stuart Mclellan},
  series    = {EPiC Series in Computing},
  volume    = {107},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2398-7340},
  url       = {/publications/paper/KjnC},
  doi       = {10.29007/l6ss},
  pages     = {237-246},
  year      = {2025}}

Download PDF Open PDF in browser