ServiceNow is a cloud-based platform that leverages AI to automate and optimize workflows across enterprises in IT, operations, customer service and more. An area of interest for the ServiceNow research team is finding explainable patterns in customer service data, such as incorrect responses from virtual agents, LLM hallucinations, or misassigned cases. To accomplish this, the team has developed a semi-automated pipeline for identifying and categorizing the causes of such outcomes, enhancing understanding of failure cases and areas of improvement for better customer service experience.