Ticket Deflection with an AI Support Assistant
I led the UX for a RAG-backed support assistant and federated semantic search, inside the support portal of a B2B analytics platform: trust, scannable answers, and a handoff to humans.
- RoleSenior UX Designer, Growth
- ClientGoodData
- IndustrySupport · AI
- Year2026
People were filing tickets the documentation could already answer.
This was a findability cost because the answer usually existed.
Reaching it meant guessing the right keywords, opening several results, reading technical material, then working out which part applied to your own setup.
At some point asking a human is simply cheaper than continuing to look.
My contribution
My role was end-to-end UX: ticket analysis, scope definition, information architecture, interaction design, answer formatting, escalation design, testing, rollout.
This is a public feature. Everything here is from live, publicly accessible screens and does not interfere with NDA requirements from the company. Please note some decisions have changed since I left the company.
Diagrams and support materials are recreated to protect company's confidentiality.
Team
UX Designer (me)- Sr Manager
- Visual Designer
- Front-end Developer
- Back-end Developer
The team was composed of a Sr Manager, a UX Designer, a Visual Designer, a Front-end Developer and a Back-end Developer.
The approval of the project came from the Director of Customer Support. Support provided the Management leadership; the Director of Product Engineering, AI and UX approved my capacity and Marketing approved the capacity of the Visual Designer.
Process
One entry point. Two retrieval behaviours. One knowledge base.
Search: semantic retrieval over the indexed support content, returning matching sections.
Ask AI: a generated answer grounded in those same sources, with citations back to them.
Built on kapa.ai: I designed what the assistant covers, how retrieval and answers are presented, how uncertainty reads, and what happens when it cannot help.
Shipped: one entry, two modes. Search stays familiar. A direct answer is one click away. Neither intent is penalised.
Rejected entry models
Rejected: separate search and chat destinations.
Makes you choose a tool before you know which one you need. Only in documentation is fine, but in the support portal, the mental model was different.
Rejected: chat-first, search secondary.
Routes people who want the source document through a conversation to reach it, and puts the least predictable part of the system in front of every question.
What shipped
Search: Ask AI sits above the results rather than instead of them, so a question phrased as a question still starts where people start. Results resolve to sections inside a document, not just to the page.
Ask AI: ordered steps, bolded specifics, copyable code, and citations inline against the claim they support. The row above the answer says how long it thought and how many sources it used.
Decisions
How was it tested?
n = 6.
Six tasks: find a source document, ask a direct technical question, verify an answer through its citations, copy a code example, move from search to Ask AI, escalate when it cannot resolve.
Watched for comprehension, source discoverability, scannability, keyboard and mobile use, and whether the escalation cue read as a cue.
Support engineers reviewed answer quality separately, looking for misleading confidence and out-of-scope answers.
How success was measured
I tracked with the PM alongside deflection: escalation rate, source interaction, answer feedback, repeat questioning, and CSAT on assistant-handled cases.
Escalation Rate
Catches the assistant holding on too long.
Source Interaction
Catches answers nobody trusts enough to check.
Answers Feedback
Catches confidently wrong answers.
Repeat Questioning
Catches the ticket that comes back later.
CSAT, assistant handled
Catches a faster answer that is a worse answer.
Thinking, tools and context
Catches where the information is sourced and how the answer was generated.
Results
Specific figures are under NDA. Direction and method only.
Ticket deflection increased after launch. CSAT on assistant-handled cases came in above the human-support baseline.
That second result was the one I least expected and the one that mattered most: people were not trading a worse answer for a faster one.
What I learned
I went in thinking the job was getting the answers right. What the assistant does when it can't answer matters just as much. A dead end there costs you trust in the good answers too.
Narrowing the scope was the right call. Fewer questions answered well beat more questions answered unevenly.
