Optimizing Workflow in Production Control Issues through the Implementation of Genie AI/BI: A Conversational Agent for Natural Language Database Interaction

In today's competitive and fast-paced manufacturing environment, optimizing workflows in production managers and controls can be defined as an essential strategy for companies attempting to maintain their competitive advantage. This drives businesses to adopt generative artificial intelligence (GenAI) technologies in search of quicker and more precise outcomes. The current production planning environment at Krones is marked by inefficiencies and usability issues. Employees rely on derived lists such in Waylist, which aggregates data from different systems for daily operational purposes. Despite the availability of numerous Power BI reports for data analysis, users struggle with their inflexibility and lack of an integrated view. Finding relevant information takes considerable time, especially for large orders and often requires expert support. Users frequently must consult multiple static reports to answer even basic questions.

Additionally, relational databases play a crucial role as the core of data management systems. Accessing data in these databases typically requires writing SQL queries, a task that demands specialized knowledge. Thus, many users, especially those who are not in data science fields, encounter difficulties when working with SQL to retrieve specific information or manipulate data. It is worth noting that these issues increase their dependency on a small group of experts and lead to further delays.

These challenges underscore the necessity for a more adaptable, transparent, and user-friendly system that minimizes IT dependency and enhances efficient decision-making. With this in mind, we tried to develop an efficient and fast working method for the support teams by implementing a chatbot (Genie BI) that translates natural language into SQL queries. Such a system can answer the most frequently asked questions of users and enable effective analysis as well as interaction with planning data. To ensure the effectiveness of the proposed system, it is vital to assess how well it functions in practice with zero-short prompting, including measuring the frequency and types of errors it produces during the Text-to-SQL conversion process. At the end of the work, a prototype application will be deployed and evaluated in terms of usability and trustworthiness.

• Literature research
• Requirements analysis through interviews with key users
• Data preparation
• Iterative development of the chatbot:
• Adaptation of the instruction prompt
  • Improving the underlying data structure (new columns, renaming, new filter columns, format changes,…)
  • Inserting SQL query examples in Genie
  • Comparison of the improved model with the standard model on the standard question set
  • Benchmark and iterative improvement of performance
• User studies: Testing the implementation with test data.
• Analyzing the collected data
• Documenting the work and writing the bachelor thesis.

• Basic knowledge of SQL and database management
• Understanding of business intelligence tools, especially Genie BI
• Experience with Large Language Models (LLMs) and prompting
• Knowledge of chatbot development and natural language processing
• Usability-Evaluation (qualitative/quantitative)

• Prego, Borja & Vilares, David & Lousa, Bruno. (2024). Large Language Model Based Chatbot for Database Interaction through Natural Language. 335-342. 10.17979/spudc.9788497498913.47.
• Steve Jeffrey, Tueno Fotso. (2024). Natural Language Query Engine for Relational Databases using Generative AI. 10.48550/arXiv.2410.07144.
• Nan, Linyong & Zhao, Yilun & Zou, Weijin & Ri, Narutatsu & Tae, Jaesung & Zhang, Ellen & Cohan, Arman & Radev, Dragomir. (2023). Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies. 14935-14956. 10.18653/v1/2023.findings-emnlp.996.
• SME, D. S. (2024, August 29). Best practices for AI/Bi Genie Spaces on Databricks. Medium. https://medium.com/dbsql-sme-engineering/best-practices-for-ai-bi-genie-spaces-on-databricks-6f101612c792
• Jannuzzi, M., Perezhohin, Y., Peres, F., Castelli, M., & Popovič, A. (2024). Zero-Shot Prompting Strategies for Table Question Answering with a Low-Resource Language. Emerging Science Journal, 8(5), 2003–2022. https://doi.org/10.28991/ESJ-2024-08-05-020