LLM Evaluation Framework for Financial Crime Report Generation

sumup · Finance · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial crime compliance report generation

SumUp needed to evaluate LLM-generated AML Reason for Suspicion narratives where free-text quality depends on topic coverage, supporting facts, structure, and hallucination avoidance rather than n-gram overlap.

Constraints:

• Financial crime reports are high-impact regulatory artifacts and require prior human confirmation and investigation.
• Traditional NLP metrics such as ROUGE showed minimal differences between accurate and inaccurate narratives.
• Human review was reliable but too slow and expensive for fast iteration.
• LLM evaluator scores were application-specific and subject to position, verbosity, and self-enhancement biases.

Architecture

SumUp built an LLM-driven evaluator for generated Reason for Suspicion narratives. The evaluator compares generated text with reference narratives using prompt-embedded benchmark checks for topic coverage, customer profile data, supporting facts, invented facts, text structure, and conclusion quality. It returns 0-5 scores and qualitative explanations, then its assessments are compared with compliance-agent comments and scores to validate alignment.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Compliance_investigation["Compliance investigation"] --> LLM_report_generator["LLM report generator"]
  Compliance_investigation --> Reference_narrative["Reference narrative"]
  LLM_report_generator --> LLM_evaluator["LLM evaluator"]
  Reference_narrative --> LLM_evaluator
  Benchmark_check_prompt["Benchmark-check prompt"] --> LLM_evaluator
  LLM_evaluator --> Evaluator_score_output["Evaluator score output"]
  Evaluator_score_output --> Human_agent_review["Human agent review"]
  Evaluator_score_output --> Data_scientist_iteration["Data scientist iteration"]
  Human_agent_review --> Data_scientist_iteration

Components

Data flow

Compliance agents first investigate suspicious activity and confirm the case context. The report-generation application produces a narrative, and the evaluator receives generated and reference text plus custom benchmark checks. The LLM returns per-check scores, explanations, and a general score. Data scientists compare automated results with agent reviews and use the feedback to iterate without continuously burdening agents.

Why this architecture

The evaluator mirrors human review more closely than ROUGE by checking application-specific narrative requirements while still producing numeric scores that data scientists can compare during iteration.

Models used: LLM report generator · LLM-driven evaluator · LLM-as-judge · ROUGE · few-shot prompting

Scale

• Evaluator scores use a 0-5 range.
  “we instructed an LLM to create a score between 0–5, where 5 represents excellent coverage, and 0 signifies very poor coverage.”article, stated
• An accurately generated text scored 4.67 on the evaluator's general score.
  “When using an LLM in conjunction with the previous evaluation method, an accurately generated text produced the following results. We can observe an average score of 4,67”article, stated
• An inaccurately generated text scored 2.5 on the evaluator's general score.
  “if we evaluate an inaccurately generated text, we get an average general_score of 2.5”article, stated
• The evaluator explicitly accounts for three known LLM-as-judge bias areas.
  “LLMs introduce biases in three key areas: position bias, verbose bias, and self-enhancement bias.”article, stated

Alternatives considered (roads not taken)

ROUGE and other traditional NLP metrics

Rejected because: They produced minimal differences for opposing examples and could not check missing facts, structure, or invented information.

Human review for every iteration

Rejected because: Human reviews are reliable but impractical for rapid business iteration.

Generic cross-project judge score

Rejected because: The article warns that evaluator metrics are application-specific and not standardized across projects.

Patterns

llm-as-judge (evaluation)

SumUp used an LLM-driven evaluator to compare generated financial-crime narratives with reference text and produce scores plus explanations.

eval-driven-development (evaluation)

The evaluator lets data scientists test report-generation improvements without repeatedly adding compliance-agent review workload.

human-in-the-loop-gate (governance)

The report-generation workflow remains grounded in agent investigation and human review because false positives in AML escalation are high impact.

Tradeoffs

Use an LLM evaluator for narrative reports.

Chose Application-specific LLM-as-judge over Traditional ROUGE-style metrics.

Rationale: The LLM evaluator captures qualitative criteria and supporting arguments that overlap metrics miss.

Keep humans as the compliance authority.

Chose Agent investigation plus automated evaluator feedback over Machine-only suspicious activity escalation.

Rationale: Automated false positives in AML reporting have extremely high consequences.

Gotchas (what bit them)

ROUGE-like metrics missed application-critical quality differences.

Root cause: N-gram overlap does not check evidence support, structure, topic coverage, or invented facts.

Resolution: Use application-specific benchmark checks embedded in an LLM evaluator prompt.

LLM judge scores are not portable across projects.

Root cause: The meaning of a numeric score depends on the application-specific rubric.

Resolution: Calibrate the score range with few-shot examples and treat metrics as local to the application.

LLM evaluators introduce known bias modes.

Root cause: Judges may prefer first-position outputs, verbose responses, or LLM-generated answers.

Resolution: Use position swapping and few-shot prompting to counteract position and verbosity bias.

Evaluation

SumUp evaluated generated narratives with custom LLM judge checks, 0-5 scores, qualitative explanations, comparisons against accurate and inaccurate examples, and validation against agent comments and numeric scores.

Metrics: topic coverage · customer profile data · supporting facts · invented facts · conclusion · 0-5 score · general score 4.67 vs 2.5

Outcomes

• The LLM evaluator separated an accurate narrative scoring 4.67 from an inaccurate one scoring 2.5.
• Automated evaluator comments and scores were often consistent with human-agent feedback.
• Data scientists could iterate on report generation without adding extra workload to compliance agents.

The source discusses rapid assessment of large data points but does not provide exact production volume.