Abstract:

When working with cloud-based large language models (LLMs) on inference tasks involving sensitive information, data sovereignty is compromised and the leakage of confidential content is possible when local information is transmitted to the cloud for processing. In this article, we outline the problems and solutions for keeping data private, even while utilizing state-of-the-art LLMs. Based on this foundation, we lay out and describe our proactive knowledge discovery and assistance system, Doku-Assist, which targets first-level service desk agents. Integrating a DokuWiki-derived knowledge base with the ticket system Znuny, Doku-Assist proactively finds and offers documentation to firstlevel support agents, thereby assisting with the resolution of issues without replacing the human agent. By employing local LLMs, the system does not compromise the privacy and confidentiality of ticket content and wiki documentation, keeping all sensitive data on-premise and safe.