Design ML Fraud Detection - System Design | WinJob

ML Fraud Detection Problem Framing

Payments fraud ML is not batch analytics — it is a synchronous gate in the authorization path, where a false block angers a paying customer and a false allow becomes a chargeback. At 180k auth/s peak (≈900M auths/day) the model has a hard p99 < 85 ms budget, because every millisecond of scoring is added straight to checkout.

The framing that wins: fraud detection is hybrid rules + ML at authorize time, not a model in isolation. Deterministic rules catch known patterns and give analysts a lever; the ML score handles the long tail. How you combine them drives shard keys (by card/device), cache TTLs (velocity windows), and human-review staffing — well before which algorithm you pick.

Quantified controls

Control	Target	Rationale
Score p99	< 85 ms	Scoring sits inline in the auth path; it is added to checkout latency
Chargeback precision	precision-first	A false block costs a sale; tune the threshold for precision
Review queue depth	< 4h SLA	Bounds analyst backlog so flagged auths are worked in time
Decision idempotency	exactly-once	Payment-switch retries must not re-score or double-act

Mechanism

Every authorization gets a decision_id that references the model artifact hash, ruleset version, feature-view timestamps, and the calibrated score — so any decision is reproducible and auditable for a chargeback dispute. Switch retries hit an idempotency store keyed by decision_id rather than re-scoring with newer, inconsistent features.

Failure modes

Feature staleness after a broker pause, so the score uses outdated velocity counters.
A partial ruleset deploy, leaving some workers on the old pack and some on the new.
GPU preemption during a surge, dropping score p99 below SLO at the worst moment.
Label leakage when chargeback timestamps are joined without point-in-time guards.
Review-queue starvation when the precision threshold tightens without added analyst capacity.

Design pressures

Document a rollback pointer and an SLI dashboard before promotion.
Run a shadow scorer on live traffic before any model takes real decisions.
Cap review-queue ingress with backpressure so analysts are never buried.
Prove the PCI boundary on feature logs (no raw PAN).
Chaos-test Flink lag and score p99 together, not in isolation.
Tie the 0.35% fraud-dollar rate to an explicit dollar-loss model.

Implementation slice

javaOne Dark Pro

1 public record FraudDecision(
2     String decisionId,
3     String txnId,
4     double calibratedScore,
5     String action,
6     String modelVersion,
7     String rulesetVersion
8 ) {}

pythonOne Dark Pro

1 from dataclasses import dataclass
2  
3 @dataclass(frozen=True)
4 class FraudDecision:
5     decision_id: str
6     txn_id: str
7     calibrated_score: float
8     action: str  # allow|review|block
9     model_version: str
10     ruleset_version: str

typescriptOne Dark Pro

1 export interface FraudDecision {
2   decisionId: string;
3   txnId: string;
4   calibratedScore: number;
5   action: "allow" | "review" | "block";
6   modelVersion: string;
7   rulesetVersion: string;
8 }

Interview signal

Name the one SLI you would dashboard in week one — score p99, the inline-latency gate — and show how it forces design changes (smaller models, tighter batching, GPU headroom) the moment it burns. Framing fraud as a hybrid rules+ML gate with a dollar-loss model, not an algorithm beauty contest, is the senior tell.

Key Highlights

•Fraud scoring is a synchronous gate in the auth path: p99 < 85 ms at 180k auth/s peak (~900M/day).
•Hybrid rules + ML: deterministic rules for known patterns and analyst control, ML for the long tail.
•Every decision carries a decision_id (model hash, ruleset version, feature timestamps) for audit and idempotency.
•Combination strategy drives shard keys, cache TTLs, and review staffing — decided before the algorithm.

1	public record FraudDecision(
2	String decisionId,
3	String txnId,
4	double calibratedScore,
5	String action,
6	String modelVersion,
7	String rulesetVersion
8	) {}

1	from dataclasses import dataclass
2
3	@dataclass(frozen=True)
4	class FraudDecision:
5	decision_id: str
6	txn_id: str
7	calibrated_score: float
8	action: str # allow\|review\|block
9	model_version: str
10	ruleset_version: str

1	export interface FraudDecision {
2	decisionId: string;
3	txnId: string;
4	calibratedScore: number;
5	action: "allow" \| "review" \| "block";
6	modelVersion: string;
7	rulesetVersion: string;
8	}