Oct 2024
Collinear helps Kore.ai deliver safe, intelligent consumer experiences across 150+ F2000 customers
Kore.ai has been a leader in the contact center industry, offering a low-code conversational AI platform servicing customers worldwide. With recent AI advancements, Kore.ai has continued to innovate by creating the next generation of AI-optimized customer and employee experiences.
Underpinning Kore.ai’s XO platform has been a fine tuned model that has demonstrated significant performance gains to enable customer center automation outcomes. However, these gains could also lead to potential safety and reliability tradeoffs. Therefore, the Kore.ai team was looking to:
- Create a standard framework for evaluating model safety across categories such as unethical activities, self-harm and false assumptions.
- Refine fine tuning approaches to improve model performance across selected safety categories.
- Maintain customer helpfulness without being overly-empathetic and verbose, or supporting user-generated harmful statements.
Solution
Collinear partnered with Kore.ai to fine tune Kore.ai’s cutting edge models to align with organizational values for safety and customer satisfaction. The solution involved the following three stages:
- Custom Judge Creation: Customize the Collinear Judge for Kore.ai and benchmark the XO GPT for safety
- Intelligent Data Enrichment: Utilized Collinear's Weaver technology to generate high-quality synthetic training data for preference finetuning
- Model Improvement via Finetuning: Implemented RLAIF (Reinforcement Learning with AI Feedback) techniques to finetune XO GPT on high-quality curated data using AI judge.
Collinear post-training led to a step change in Kore.ai’s XO offerings with the following main results:
1. 91% of bot responses maintained or improved performance.
2. 44% of bot responses improved by a significant margin (1 pt or more)
3. 18% of bot responses improved by a massive margin (2 pts or more)
"As we scale our AI capabilities across hundreds of global enterprises, ensuring both safety and performance is mission-critical for our customer experience platform. Collinear's AI safety framework has enabled us to confidently deploy our conversational AI solutions while maintaining strict guardrails against harmful content, resulting in a 44% improvement in model safety and significantly faster deployment times for our enterprise customers.”
Ravi, KoreAI
.svg)
