The Performance and AI Optimization Issues for Task-Oriented Chatbots

Date

2024

Authors

Gunnam, Ganesh Reddy

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

In recent years, there has been a significant surge in the popularity of human-machine digital assistants, particularly advanced chatbots, in consumer-supporting domains. Many of these chatbots are now deployed in the cloud due to their flexibility, scalability, reliability, and cost-effectiveness advantages. However, assessing the performance of these cloud-based chatbot systems presents challenges. This paper investigates various methodologies for performance evaluation, including real-time testing with real users and automated simulations, and identifies key performance metrics like response time and throughput. A case study using an automated protocol chatbot development framework provides valuable insights for practitioners. Understanding and responding appropriately to human utterances are critical, leading to integrating Natural Language Understanding (NLU) engines from major cloud service providers. The paper examines NLU platforms and presents a case study using Google DialogFlow's NLU service, focusing on intent recognition performance. The article explores the cloud deployment of chatbot systems, emphasizing performance assessment, response times, and natural language processing capabilities. It details a case study platform supporting deep-logic chatbots and discusses the need for extensive training data and advanced NLU models. Additionally, the paper delves into keyword detection's role in chatbots, especially in closed-domain models with limited vocabulary. It introduces a keyword reduction methodology and demonstrates substantial performance improvements. Finally, the paper emphasizes the relevance of closed-domain dialogue systems for specific applications, their reliance on keywords, and the importance of contextual keyword recognition. It also explores the integration of Large Language Models (LLMs) with intent prediction (IP) and their potential in addressing niche applications with protocol-driven chatbots, including challenges posed by certain questions.

Description

The full text of this item is not available at this time because the author has placed this item under an embargo until May 16, 2025.

Keywords

AI Optimization, Chatbots, Performance evaluation, Task-oriented dialog systems

Citation

Department

Electrical and Computer Engineering