TelBench: A Benchmark for Evaluating Telco-Specific Large Language Models
Sunwoo Lee, Dhammiko Arya, Seung-Mo Cho, and 9 more authors
In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, Nov 2024
The telecommunications industry, characterized by its vast customer base and complex service offerings, necessitates a high level of domain expertise and proficiency in customer service center operations. Consequently, there is a growing demand for Large Language Models (LLMs) to augment the capabilities of customer service representatives. This paper introduces a methodology for developing a specialized Telecommunications LLM (Telco LLM) designed to enhance the efficiency of customer service agents and promote consistency in service quality across representatives. We present the construction process of TelBench, a novel dataset created for performance evaluation of customer service expertise in the telecommunications domain. We also evaluate various LLMs and demonstrate the ability to benchmark both proprietary and open-source LLMs on predefined telecommunications-related tasks, thereby establishing metrics that define telcommunications performance.
A.X K1 Technical Report2026
TelAgentBench: A Multi-faceted Benchmark for Evaluating LLM-based Agents in TelecommunicationsIn Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, Nov 2025
TelBench: A Benchmark for Evaluating Telco-Specific Large Language ModelsIn Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, Nov 2024
Deep-plane sweep generative adversarial network for consistent multi-view depth estimationMachine Vision and Applications, 2022