AI Customer Service Copilot for Wealth Operations
Designing a secure, context-aware support assistant that helps banking agents resolve complex financial queries faster, with verifiable, policy-backed answers instead of unaccountable guesses.
↑ Every suggested answer arrives with the exact policy clause it came from.
Fragmented knowledge, frustrated customers
When I joined the wealth operations team, customer support was struggling under the weight of complex financial queries. Agents navigated a fragmented landscape of legacy knowledge bases, PDF policy documents, and disjointed CRM tools just to answer questions about tax wrappers, portfolio rebalancing, or fund fees.
That friction produced long hold times and inconsistent answers. On the self-serve side, the existing chatbot was rigid, looping customers through generic responses before dumping them into an agent queue with zero context. We needed a system that empowered our human experts, not one that replaced them with a flawed bot.
45% of agent time went to searching policies. First contact resolution sat at 52%. 80% of handoffs to a human agent arrived with the conversation's context already lost.
Designing a trustworthy assistant, not a replacement
My objective was to design an AI copilot that acted as a knowledgeable partner to human agents, while handling safe, bounded self-serve queries directly. The core challenge was trust: every AI-generated answer had to be grounded in a verifiable policy document, with a clear signal for when human escalation was required.
Agent Research
Shadowed support agents across high volume shifts to map the moments where conversations stalled.
Boundary Design
Worked with legal and compliance to define exactly what the AI could answer autonomously.
Conversational UX
Designed the suggested-reply, confidence-scoring, and citation interface agents actually trusted.
Engineering Collaboration
Defined API contracts for RAG responses, confidence scores, and citations without latency cost.
Three compounding failures in the support workflow
No safeguards against wrong answers
Nothing stopped an agent from confidently giving outdated or incorrect financial guidance, since the knowledge base was disconnected from the live chat interface entirely.
Manual search inside every conversation
Agents copy-pasted customer details into separate search tools mid-conversation just to find a relevant policy, breaking their attention away from the customer in front of them.
Context lost at every handoff
When the rigid chatbot escalated to a human, the agent started from zero. The customer had to repeat their issue, and 80% of handoffs arrived with no usable context at all.
How might we give agents instant, verifiable access to policy knowledge inside the conversation itself, without ever letting an unverified AI answer reach a customer directly?
If we ground every AI suggestion in a cited policy document, scale agent autonomy to the model's confidence level, and preserve full context across every escalation, then handling time will drop, accuracy will rise, and agents will trust the tool enough to actually use it.
Mapping where trust broke down
I mapped the entire agent journey, from the moment a query arrived to final resolution, and shadowed agents directly rather than relying on their retrospective account of a stressful shift.
Agent Shadowing · 15 sessions
Observed high-volume shifts directly, logging the exact moments a conversation stalled or an agent switched tools.
Compliance Reviews · 4 rounds
Worked with legal to define strict, safe response boundaries for any answer the AI gave without a human in the loop.
Key research insight
"I don't trust the AI to talk directly to my clients about their wealth. But if it can find the exact policy clause I need while I'm on the chat, it would save me hours every day."
Senior Support Agent, Wealth Operations
What shadowing agents made obvious
Agents wanted augmentation, explicitly not automation
The distrust wasn't of AI in general. It was specific: agents didn't want the AI speaking to clients directly about their wealth, full stop.
The copilot needed to listen, not wait to be asked
Agents were maintaining conversational context and running searches simultaneously. A reactive tool that waited for a typed question added a step instead of removing one.
A citation was worth more than a confident tone
Agents didn't want the AI to sound sure of itself. They wanted to see exactly which document it was reading from, so they could judge the answer themselves.
The rules we designed by, and why
1 · Ground every suggestion in a verifiable source
Derived from agents refusing to trust unverified financial guidance.
→ RAG architecture citing the exact internal policy clause behind every suggestion.
2 · Match agent autonomy to model confidence
Derived from the need to move fast on easy questions without risking hard ones.
→ High-confidence suggestions send in one click; low confidence prompts a mandatory review.
3 · Preserve context across every handoff
Derived from 80% of handoffs arriving with the conversation's context already lost.
→ Escalation carries full history, intent, and the AI's attempted resolution into the agent's workspace.
4 · Keep humans as the final decision-maker
Derived directly from agents stating they didn't want the AI speaking to clients unsupervised.
→ Suggested actions the agent approves, never autonomous client-facing replies.
Copilot anatomy: how assistance stays grounded
Rather than one long chain of problems and solutions, the interaction model settled into four beats agents actually felt during a live shift.
01 Beat 01 Listen
Live transcript and CRM context fuse together so the copilot already knows the customer's product mix and risk tier before an agent asks anything.
02 Beat 02 Retrieve
Policy graph search returns the exact clause, not a whole PDF, cutting the hunt time that used to consume 45% of an agent's shift.
03 Beat 03 Compose
Drafts mirror the brand's voice, so agents edit a reasonable answer rather than writing one from a blank line under time pressure.
04 Beat 04 Prove
Every suggestion carries an audit card bundling its sources, model version, and the agent's approval, ready for compliance review without extra paperwork.
The choices that shaped the outcome
Decision 01 · RAG-grounded answers vs. a general-purpose LLM
Tension: A general-purpose model is faster to stand up than one grounded in an internal policy corpus.
Choice & trade-off: We built the RAG architecture anyway. Ungrounded wealth advice was a compliance and trust problem the business genuinely could not accept, regardless of speed to launch.
Decision 02 · Confidence-gated autonomy vs. mandatory review of everything
Tension: Reviewing every suggestion is safer, but erodes the speed gains the whole project was meant to deliver.
Choice & trade-off: High-confidence answers send in one click; low-confidence ones require review. This kept speed on easy questions without loosening the safety net on hard ones.
Decision 03 · Suggested actions vs. fully automated replies
Tension: Full automation is the more ambitious, more impressive-sounding product vision.
Choice & trade-off: Wizard-of-Oz testing showed agents explicitly did not want the AI talking to clients unsupervised. We built for that reality rather than the more ambitious pitch.
A context-aware support workspace
A unified conversation workspace brings the customer's portfolio data, the live chat, and the AI copilot into one view, so agents stop switching tabs mid-conversation.
Suggested Reply & Confidence Panel
A confidence indicator next to each draft: high confidence sends in one click, low confidence prompts a review.
Verifiable Source Citations
Every drafted response links to the internal policy document it came from, with a hover preview of the source text.
Product Explainer Cards
Bite-sized visual summaries of complex topics like ISA allowances, ready to drop straight into the customer chat.
Seamless Escalation Logic
Conversation history, intent, and the AI's attempted resolution pass directly into the human agent's workspace.
Design doesn't happen in isolation
Legal and compliance: defining the safe boundary
Four full compliance review rounds shaped exactly which questions the AI could answer autonomously and which always needed a human.
Engineering and data science: keeping latency invisible
Clear API contracts for RAG responses, confidence scores, and citations meant the interface never felt like it was waiting on the model.
Senior agents: validating what "trustworthy" actually looks like
Concept testing with senior agents specifically shaped how confidence scores and citations were displayed, so the interface read as honest rather than falsely reassuring.
How we arrived at the solution
Contextual Inquiry
Sat with support agents for 3 days, observing how they toggled between 4 different internal tools to resolve a single ticket.
Role-Playing Scenarios (Wizard of Oz)
Simulated AI responses using human experts to test how agents would react to the copilot before a line of code was written.
Illustration Placeholder
Prompt: A stylized illustration of designers running a Wizard-of-Oz customer-support test, a human expert simulating AI responses behind a screen, in friendly cyan and blue tones. Rendered as a clean illustration on a fully transparent background (no backdrop, scene, or color fill), so it displays cleanly on both light and dark page themes.
Empowered agents, satisfied customers
"I no longer dread complex policy questions. The copilot finds the exact clause I need instantly, letting me focus on actually helping the customer."
Senior Support Agent, Wealth Operations
Honest reflections from the process
What Worked
Designing for trust as a first-class requirement
Showing confidence scores and source citations, not hiding the model's uncertainty, was what overcame agents' initial skepticism.
Testing the concept with Wizard of Oz before writing any code
Simulating the AI with human experts surfaced the "suggested actions, not automation" preference early, before it became an expensive engineering decision to reverse.
What I'd Do Differently
Bring compliance into the Wizard of Oz sessions directly
Compliance reviewed the boundary definitions separately from the role-play testing. Having them observe the sessions directly would have caught edge cases sooner.
Instrument which citations agents actually open
We know citations mattered for trust, but not how often agents actually clicked through to verify one. That data would sharpen which sources deserve the most editorial care.