Download PDFOpen PDF in browserBuilding UniGPT: A Customizable On-Premise LLM-Solution for Universities9 pages•Published: January 13, 2025AbstractLarge Language Models (LLMs) have become the hot topic in Artificial Intelligence (AI) in the last few years, especially with the advent of the Generative Pretrained Transformer (GPT) models and the release of ChatGPT in November 2022. Demand from faculty members for access to such models quickly arose and proved to be hard to address in an orderly manner by central IT providers due to technical, privacy and payment constraints by the major suppliers. Additionally, specific research questions might require more control over the model and deployment. Thus, access to non-public on-premises models became desirable and became possible with open-source solutions (e.g. Llama 2 by Meta and Mixtral 8x7B by Mistral AI). Due to the large model sizes, on-premises deployments are demanding in terms of hardware and system engineering. In this paper, we present our deployment of a large language model for the University of Mu ̈nster, a service we call UniGPT. We focus on the high-level architecture, consisting of the frontend, backend, and models, and also discuss the experiences with our deployed service.Keyphrases: artificial intelligence ai, large language model llm, on premises service deployment In: Raimund Vogl, Laurence Desnos, Jean-François Desnos, Spiros Bolis, Lazaros Merakos, Gill Ferrell, Effie Tsili and Manos Roumeliotis (editors). Proceedings of EUNIS 2024 annual congress in Athens, vol 105, pages 108-116.
|
