AI-Powered Fraud Detection in E-commerce Using AWS Fraud Detector

awaze · E-commerce · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: E-commerce fraud detection

Awaze and similar e-commerce businesses need to detect rare, costly fraud events while minimizing false positives, review workload, and regulatory risk.

Constraints:

• Fraud is typically less than half a percent of orders
• GDPR and customer-data use require human review before decisions
• AWS Fraud Detector predictions are batch-oriented rather than fully real time

Architecture

The Awaze fraud-detection case uses AWS Fraud Detector as a managed risk-scoring service, trained from labeled historical transactions and deployed with threshold-based human review. The production design couples batch scoring, feature enrichment, reviewer explanations, SageMaker or Lambda deployment paths, feedback from reviewers and outcomes, and retraining cadence tuned to fraud-trend velocity.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  historical-labeled-orders["historical-labeled-orders"]
  aws-fraud-detector-model["aws-fraud-detector-model"]
  feature-enrichment-layer["feature-enrichment-layer"]
  threshold-router["threshold-router"]
  human-review-workflow["human-review-workflow"]
  reviewer-explanation-view["reviewer-explanation-view"]
  sagemaker-pipeline["sagemaker-pipeline"]
  lambda-batch-workflow["lambda-batch-workflow"]
  feedback-and-retraining-loop["feedback-and-retraining-loop"]
  historical-labeled-orders --> aws-fraud-detector-model
  aws-fraud-detector-model --> threshold-router
  aws-fraud-detector-model --> feature-enrichment-layer
  feature-enrichment-layer --> reviewer-explanation-view
  threshold-router --> human-review-workflow
  reviewer-explanation-view --> human-review-workflow
  human-review-workflow --> feedback-and-retraining-loop
  sagemaker-pipeline --> aws-fraud-detector-model
  sagemaker-pipeline --> feedback-and-retraining-loop
  lambda-batch-workflow --> aws-fraud-detector-model
  lambda-batch-workflow --> feedback-and-retraining-loop
  feedback-and-retraining-loop --> historical-labeled-orders
  feedback-and-retraining-loop --> aws-fraud-detector-model

Components

Data flow

Historical labeled order data trains AWS Fraud Detector. New or batched orders receive a 0-1000 fraud score; thresholds route risky orders to human reviewers for GDPR-compliant decisions. Feature-importance outputs and enrichment signals help reviewers interpret scores. Review outcomes and confirmed fraud labels feed back into retraining, with SageMaker pipelines used for high-volume cases and Lambda batch workflows used for lower-volume cases.

Why this architecture

The architecture is built around the operational nature of fraud: labels are delayed, fraud rates are highly imbalanced, reviewer capacity constrains thresholds, and GDPR keeps final decisions human-reviewed. Managed scoring, batch deployment choices, feature enrichment, and feedback loops let the team adapt thresholds and retraining cadence to each fraud pattern rather than treating fraud detection as a one-time model launch.

Models used: AWS Fraud Detector models

Scale

• AWS Fraud Detector expects 18 months of historical labeled data for this setup.
  “It takes 18 months of historical data that has known um fraudulent or legitimate labels and it builds a model.”transcript, stated
• Orders receive a fraud score from 0 to 1000.
  “your orders will receive a fraud score between zero and a thousand. So this is a likelihood of how fraudulent something might be”transcript, stated
• The speaker models one case with 2% expected fraud and 98% legitimate orders.
  “2% of all orders are expected to be fraud. And we're going to say that each item costs about £500.”transcript, stated
• Prediction cost was described as three cents per prediction, which becomes thousands of dollars at 100,000 monthly orders.
  “It is three cents per prediction, which doesn't sound like a lot, but if you've got a h 100,000 orders coming in monthly”transcript, implied
• The initial offline sampling progression uses 100 fraud examples, then a 50/50 split, then a realistic 2% fraud / 98% legitimate split.
  “Then I do a 5050 split of legitimate to fraudulent orders. And then finally, a sample of my 2% fraud against 98% legit”transcript, stated

Alternatives considered (roads not taken)

One universal fraud threshold

Rejected because: Fraud mix and reviewer capacity differ, so thresholds may be static, seasonal, or moving depending on the business case.

Always deploy with SageMaker pipelines

Rejected because: The speaker uses SageMaker for high volume, but suggests Lambda workflows for lower-volume orders.

Evaluate only on balanced fraud samples

Rejected because: The evaluation has to end with realistic fraud prevalence to prove the signal survives class imbalance.

Let model scores replace reviewer judgment immediately

Rejected because: GDPR and reviewer-bias concerns require human review and staged live trials before relying on score-driven decisions.

Patterns

confidence-thresholding (fraud-score-routing)

Orders receive fraud likelihood scores, and scores above the configured threshold go to human review.

human-in-the-loop-gate (regulated-fraud-decisioning)

Human reviewers examine scored cases before final fraud decisions are made for GDPR-related reasons.

eval-driven-development (class-imbalance-evaluation)

The evaluation focuses on true positives and false positives rather than headline accuracy because fraud is rare.

batch-offline-processing (fraud-prediction-windowing)

Because Fraud Detector processes one batch at a time, businesses may run hourly batches instead of true real-time prediction.

data-quality-loop (fraud-pattern-refresh)

Retraining frequency is tied to how quickly fraud patterns change and whether features begin to drift.

Tradeoffs

How often to retrain fraud models.

Chose cadence based on fraud trend velocity and feature drift over a fixed calendar schedule for all businesses.

Rationale: Fast-moving fraud categories may need monthly or faster refreshes, while seasonal fraud can refresh around calendar events.

How to choose fraud review thresholds.

Chose thresholds calibrated to business context and reviewer capacity over one universal score cutoff.

Rationale: Different goods, services, fraud distributions, and costs require different review thresholds.

Gotchas (what bit them)

Accuracy is misleading for rare fraud events.

Root cause: A model that marks every order legitimate can still appear highly accurate when fraud is below half a percent.

Resolution: Evaluate true positives and false positives against business loss and customer-friction tradeoffs.

Human reviewers can be under-informed by a raw fraud score alone.

Root cause: A numerical score does not explain which features drove the risk.

Resolution: Give reviewers the most important feature values and explanations so they can contextualize the score.

Stopping suspected fraud can starve the retraining loop of definitive labels.

Root cause: If the event is interrupted, the business may never know whether it would have become confirmed fraud.

Resolution: Risk tolerance and label-generation strategy must be considered explicitly before retraining plans are set.

Evaluation

The fraud workflow evaluates held-out data under class-imbalance-aware samples and calibrates thresholds based on true positives, false positives, fraud loss, review cost, and batch timing.

Metrics: true positives · false positives · fraud score · review cost · prediction cost · batch prediction timeliness · feature drift

Outcomes

• The system scores orders by fraud likelihood and routes high-risk cases to review.
• The presentation provides an operational pattern for thresholding, retraining, feature support, and batch deployment rather than a quantified production outcome.
• The summary reports UK e-commerce fraud context of roughly £1 billion stolen in 2024 and £1.5 billion prevented.

This is a traditional ML/MLOps case rather than an LLM or agent system, but its review, threshold, and retraining patterns are relevant to high-stakes AI operations.