AI Fraud Detection & Transaction Monitoring
Designing a risk operations workbench that cuts through alert fatigue, prioritises high-risk anomalies, and accelerates fraud investigations for a high-volume payments platform.
↑ From five disconnected tools to one prioritised, explainable alert queue.
The alert fatigue crisis
When I joined the risk operations team, our payments platform was processing millions of transactions daily. The legacy rules-based monitoring system generated thousands of alerts every shift, and 85% of them were false positives.
To investigate a single suspicious transaction, analysts jumped between five different legacy tools to piece together a login history, device IDs, and transaction patterns. That fragmented workflow created real investigation bottlenecks and a genuine risk of missing an actual account takeover or laundering pattern buried under the noise.
85% false positive rate. 70% of analyst time spent gathering data across five tools, versus 30% actually spent analysing risk. The machine learning model was already finding real fraud rings. Analysts just couldn't interpret its raw scores fast enough to act.
Turning a black box into a clear narrative
I designed a centralised workbench that translated black-box AI scores into transparent, actionable insight, so analysts could make fast, accurate decisions with full context instead of hunting for it.
Analyst Research
15 shadowing sessions during live shifts, mapping the real investigation workflow tab by tab.
AI-to-UI Translation
Mapped machine-learning risk factors directly onto interface components analysts could scan in seconds.
Workflow Design
Redesigned the alert queue, entity graph, and case disposition flow around a single unified view.
Engineering Collaboration
Defined API contracts with data science so model weights reached the UI without performance trade-offs.
Three compounding failures behind the noise
Five tools, one investigation
Analysts pieced together login history, device IDs, and transaction patterns by hand across five disconnected systems for every single case they opened.
Unexplainable risk scores
The model flagged genuine fraud rings, but its raw score gave analysts no way to understand why a transaction was risky, so trust in the score itself was low.
First-in, first-out ordering
Alerts queued by arrival time, not risk, meant a critical account takeover could sit behind a dozen low-risk false positives simply because it arrived later.
How might we help an analyst go from alert to informed decision in minutes rather than tab-switching for most of their shift, without weakening the model's judgement?
If we replace the chronological alert feed with a dynamically risk-ranked queue, surface the model's reasoning alongside each score, and consolidate the five investigation tools into one workbench, then triage time will fall sharply and analysts will act on high-risk alerts with genuine confidence rather than hesitation.
Shadowing the shift, not just interviewing after it
To find the root cause of the triage delays, I embedded with the risk operations team, shadowing senior fraud analysts during live shifts rather than relying on retrospective interviews alone.
Where analyst time actually went
Analyst Shadowing · 15 sessions
Live-shift observation mapping every tab switch, copy-paste, and manual cross-reference an investigation required.
Data Science Workshops · 2 sessions
Worked with data science to understand how the anomaly-detection model reasoned, so its logic could be surfaced honestly in the UI.
Key research insight
"Analysts are not struggling to make decisions. They are struggling to find the data required to make them. If we surface the AI reasoning alongside the user's historical timeline, we can cut triage time in half."
Senior Fraud Analyst
What shadowing real shifts made clear
Data synthesis mattered more than data presentation
Analysts didn't need prettier charts. They needed the relevant facts already assembled in one place, since assembly itself was where most of the shift disappeared.
A score without a reason gets ignored under pressure
Analysts hesitated to act on a high-risk flag when they couldn't see what drove it, treating an unexplained score as noise rather than signal.
Fraud rings hide in relationships, not single transactions
The most damaging patterns only appeared when accounts sharing devices, IPs, or funding sources were viewed together, something no single-transaction alert could ever surface on its own.
The rules we designed by, and why
1 · Rank by risk, not arrival time
Derived from critical alerts sitting behind low-risk noise purely because of queue order.
→ Dynamic prioritisation reorders the queue in real time by risk score.
2 · Show the model's reasoning, not just its score
Derived from analysts hesitating on unexplained high-risk flags.
→ Explainable tags name the specific signals behind every score.
3 · Collapse five tools into one
Derived from 70% of analyst time spent gathering data across disconnected systems.
→ A single workbench view combining alert, entity graph, and history.
4 · Automate the paperwork, not the judgement
Derived from documentation consuming time that should go to actual analysis.
→ Auto-generated draft case notes, always reviewed and confirmed by the analyst before submission.
Three discovery phases, each shaping the next
01 Phase 01 Analyst Workflow Mapping
I mapped the exact investigation workflow step by step, noting every tab switch, every copied IP address, every manual cross-reference against a blocklist. This produced a literal inventory of friction, not an impression of it.
02 Phase 02 Model Transparency Workshops
Working closely with data science, I learned how the anomaly-detection model actually reasoned, so the explainable risk signals I later designed reflected what the model was really weighting, not a simplified guess at it.
03 Phase 03 Prototyping and Validation
I evaluated low-fidelity layouts directly with analysts to confirm the AI risk scoring stayed transparent and mapped cleanly to real interface components before any engineering investment began.
The choices that shaped the outcome
Decision 01 · Dynamic risk ranking vs. a stable chronological queue
Tension: A dynamically reordering queue can feel unpredictable to analysts used to a fixed order.
Choice & trade-off: We chose dynamic ranking. A predictable but wrong order let genuinely dangerous cases sit for hours, a worse outcome than a small adjustment period for analysts.
Decision 02 · Auto-drafted case notes vs. fully manual documentation
Tension: Auto-drafted notes risk analysts trusting a flawed AI summary without independent verification.
Choice & trade-off: We kept auto-drafting but required explicit analyst review and edit before any note could be submitted, saving typing time without outsourcing judgement.
Decision 03 · Full entity graph vs. a simple linked-account list
Tension: A graph view is a meaningfully bigger design and engineering investment than a plain list.
Choice & trade-off: We built the graph. Fraud rings are inherently relational, and a flat list would have hidden exactly the pattern analysts needed to see.
One workbench, full context, faster decisions
Live queue
Prioritised alert stack
↑ The Smart Alert Queue: ranked by live risk score, each row tagged with the specific signal driving it.
Signal Weighting
Visual indicators map directly to model signal weights, showing exactly which variable drove a score.
Behavioural Baselines
Comparative charts highlight deviations from a user's own history, making anomalies instantly recognisable.
Auto-Generated Notes
AI findings translate into a draft case note the analyst reviews and confirms, not a note that ships unread.
Design doesn't happen in isolation
Data science: keeping explanations honest
The model transparency workshops ensured the "explainable tags" in the UI reflected what the model actually weighted, not a simplified story that happened to look convincing.
Engineering: defining the API contract early
Agreeing on exactly how model weights would pass to frontend components before high-fidelity design began meant zero implementation delays at handover.
Risk operations leadership: aligning on risk tolerance
The Risk Matrix Ideation workshop forced an explicit conversation about false-positive versus false-negative tolerance, a trade-off the design had to reflect, not quietly assume.
How we arrived at the solution
Cognitive Walkthroughs
Walked through the system as a fraud analyst under time pressure, identifying where the UI buried critical risk signals under dense tables.
Risk Matrix Ideation
Facilitated a session with Risk Ops and Engineering to map false-positive versus false-negative scenarios against real business impact.
Illustration Placeholder
Prompt: A stylized illustration of a risk-ops research session, analysts and a designer reviewing a threat model on a large screen, in indigo and dark-blue accent tones. Rendered as a clean illustration on a fully transparent background (no backdrop, scene, or color fill), so it displays cleanly on both light and dark page themes.
Empowered analysts, protected revenue
Honest reflections from the process
What Worked
Trust requires transparency
Analysts would not act on a high-risk score they didn't understand. Exposing the underlying factors was the single change that unlocked real adoption of the new queue.
Workflow trumps algorithms
Even the best machine learning model fails if the surrounding human workflow is broken. Consolidating five tools into one delivered as much value as the model itself.
What I'd Do Differently
Bring engineering into the risk matrix workshop earlier
Engineering joined the false positive versus false negative conversation partway through. Their technical constraints on scoring latency would have sharpened that discussion had they been in the room from day one.
Measure analyst confidence directly, not just speed
Triage time and SLA compliance are strong metrics, but I'd add a direct, repeated confidence survey to catch any erosion in trust before it shows up as a slowdown in the numbers.