700 agents' worth of AI support — until the quality gap showed up
Klarna · 2024–2025
What they did
In February 2024 Klarna put an OpenAI-powered assistant in front of customer service chat, live in 23 markets and 35 languages, and let it handle full conversations end to end rather than just suggesting replies to a human.
What happened
First month: 2.3 million conversations, doing the work of 700 full-time agents, resolution time down from 11 minutes to under 2, repeat-contact rate down 25%, an estimated $40M profit improvement for 2024. By May 2025, CEO Sebastian Siemiatkowski told Bloomberg the company was reversing course: the AI matched humans on simple queries but quality dropped noticeably on disputes, fraud claims, and hardship cases, and it sometimes gave confident, wrong answers about fees and policy. Klarna began rehiring human agents into a hybrid model; by Q3 2025 it reported the AI at ~853 agents' worth of work and ~$60M in annualized savings, response times 82% faster, alongside a rebuilt human layer for the cases that need one.
The so-what
The failure mode wasn't the AI being bad — it was routing 100% of volume through it and only later discovering which slice of tickets needed a human's judgment, not just a human's tone.