AI-Powered Market Surveillance System for Financial Compliance

london_stock_exchange_group · Finance · 2025 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial market surveillance and regulatory compliance

LSEG needed to help market-surveillance analysts triage suspicious trading alerts by determining whether price movements were associated with price-sensitive news, while keeping false negatives low, preserving auditability, and escalating ambiguous cases for manual review.

Constraints:

• market surveillance must cover all MiFID asset classes, markets, and jurisdictions
• analysts already face increasing alert volumes and false positives
• insider-dealing triage requires correlating trading activity with price-sensitive news
• regulated financial workflows require explainable decisions and audit trails
• uncertain news should not be automatically labeled non-sensitive
• reported perfect metrics come from a small representative evaluation sample

Architecture

LSEG built Surveillance Guide as a Bedrock-backed classification and triage prototype. Raw Regulatory News Service HTML articles are ingested, stripped of extraneous HTML, and transformed into clean news text. A prompt template is filled with the article text and sent to Anthropic Claude Sonnet 3.5 through Amazon Bedrock. The model returns a summary, a price-sensitivity classification, and a justification. The system uses a two-stage classification design: one pass identifies potentially price-sensitive news, and another identifies potentially non-price-sensitive news. Merge rules resolve agreement and route ambiguous or contradictory results to manual review. A Streamlit interface presents predictions and justifications to analysts, while SageMaker/Jupyter experimentation and expert-labeled data drive evaluation.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  rns-html-corpus["rns-html-corpus"] --> news-preprocessing-pipeline["news-preprocessing-pipeline"]
  news-preprocessing-pipeline --> eda-sampling-workflow["eda-sampling-workflow"]
  eda-sampling-workflow --> expert-labeled-dataset["expert-labeled-dataset"]
  expert-labeled-dataset --> evaluation-metrics["evaluation-metrics"]
  news-preprocessing-pipeline --> classification-prompt-template["classification-prompt-template"]
  classification-prompt-template --> bedrock-claude-inference["bedrock-claude-inference"]
  bedrock-claude-inference --> two-stage-classifier["two-stage-classifier"]
  two-stage-classifier --> classification-merge-rules["classification-merge-rules"]
  classification-merge-rules --> streamlit-inference-ui["streamlit-inference-ui"]
  classification-merge-rules --> manual-review-queue["manual-review-queue"]
  streamlit-inference-ui --> surveillance-analyst["surveillance-analyst"]
  manual-review-queue --> surveillance-analyst
  bedrock-claude-inference --> bedrock-monitoring-governance["bedrock-monitoring-governance"]

Components

Data flow

RNS HTML news documents are collected and preprocessed into clean text. The clean article text fills a classification prompt template. Claude Sonnet 3.5 is invoked through Amazon Bedrock to produce predictions and justifications. Step 1 classifies an article as potentially price sensitive or other; Step 2 classifies it as potentially not sensitive or other. Merge rules produce Sensitive, Non-sensitive, or Ambiguous/manual-review labels. The Streamlit inference application shows the results to surveillance analysts, who use the explanation to triage whether highlighted trading activity can be mitigated or should move toward investigation.

"Ingest raw RNS news documents in HTML format

2. Preprocess and extract clean news text
3. Fill the classification prompt template with text from the news documents"

— article, stated "Preprocess and extract clean news text

3. Fill the classification prompt template with text from the news documents
4. Prompt Anthropic’s Claude Sonnet 3.5 through Amazon Bedrock"

5. Receive and process model predictions and justifications
6. Present results through the visualization interface developed using Streamlit"

— article, stated "Receive and process model predictions and justifications

6. Present results through the visualization interface developed using Streamlit"

Why this architecture

LSEG decomposed the problem into preprocessing, LLM classification, merge rules, analyst-facing explanations, and expert-labeled evaluation because financial surveillance is high-stakes and ambiguous. The two-stage classifier lets the system optimize different risk profiles: high precision for non-sensitive news to reduce false positives and high recall for price-sensitive news to avoid missing risky cases. Ambiguous merge outcomes go to manual review. Bedrock provides managed Claude access, scaling, monitoring, governance, and audit support, while Streamlit gives analysts a practical interface for triage.

"The architecture consists of three main components:

• A data ingestion and preprocessing pipeline for RNS articles
• Amazon Bedrock integration for news analysis using Claude Sonnet 3.5"

— article, stated "The analytical approach was a two-step classification process, as shown in the following figure:

• Step 1: Classify news articles as potentially price sensitive or other"

— article, stated "* Precision for the NOT SENSITIVE class

• Recall for the PRICE SENSITIVE class"

— article, implied "* Precision for the NOT SENSITIVE class

• Recall for the PRICE SENSITIVE class"

— article, implied "Ambiguous – requires manual review i.e., Hard to Determine

Sensitive

Non-sensitive

Ambiguous – requires manual review i.e., Hard to Determine"

Models used: Anthropic Claude Sonnet 3.5 · Amazon Bedrock · Instructor library · traditional supervised learning methods · fine-tuning scenarios

Scale

• London Stock Exchange facilitates trading and reporting of more than £1 trillion of securities annually.
  “London Stock Exchange alone facilitates the trading and reporting of over £1 trillion of securities by 400 members annually.”article, stated
• The exchange has 400 members referenced in the surveillance context.
  “over £1 trillion of securities by 400 members annually.”article, stated
• The RNS dataset contained approximately 250,000 articles.
  “approximately 250,000 RNS articles spanning 6 consecutive months of trading activity in 2023.”article, stated
• The dataset spanned 6 consecutive months of 2023 trading activity.
  “spanning 6 consecutive months of trading activity in 2023.”article, stated
• The expert-labeled evaluation set contained 110 articles.
  “110 articles were selected to cover major news categories”article, stated
• Analysts labeled articles on a nine-point price-sensitivity scale.
  “evaluated each article’s price sensitivity on a nine-point scale”article, stated
• The reported result includes 100% precision on 6 analyst-confirmed non-sensitive articles.
  “100% precision in identifying non-sensitive news, allocating 6 articles to this category”article, stated
• The reported result includes 100% recall across 36 hard-to-determine and 28 price-sensitive articles.

  "* 100% precision in identifying non-sensitive news, allocating 6 articles to this category that analysts confirmed were non price sensitive

• 100% recall in detecting price-sensitive content, allocating 36 hard to determine and 28 price sensitive articles labelled by analysts into one of t"

— article, stated

• The prototype evaluation ran over a 6-week period.
  “Over a 6-week period, Surveillance Guide demonstrated remarkable accuracy when evaluated on a representative sample dataset.”article, stated

Alternatives considered (roads not taken)

Manual news-sensitivity triage for every suspicious alert

Rejected because: Manual collation of regulation, news, sentiment, and trading evidence is time-consuming and resource-intensive under high alert volume.

Rules-only surveillance systems

Rejected because: Evolving markets, tactics, alert frequency, and false positives required revisiting outdated automated rules-based systems.

Single-stage price-sensitivity classification

Rejected because: LSEG used separate passes for sensitivity and non-sensitivity, then merged results to handle ambiguous cases and optimize different error costs.

"* Step 1: Classify news articles as potentially price sensitive or other

• Step 2: Classify news articles as potentially price not sensitive or other"

— article, stated "* Step 1: Classify news articles as potentially price sensitive or other

• Step 2: Classify news articles as potentially price not sensitive or other"

— article, stated

Automatically mark doubtful news as non-sensitive

Rejected because: The prompt explicitly routes reasonable doubt to OTHER rather than NOT_PRICE_SENSITIVE, preserving manual review for risky cases.

Self-host and directly manage model infrastructure

Rejected because: Amazon Bedrock provided unified model access, serverless scaling, monitoring, and governance features.

Patterns

structured-output-contract (contracts)

The prompt elicits a summary, classification, and justification for each news article, producing structured analyst-facing outputs.

confidence-thresholding (evidence)

Ambiguous or doubtful classifications are routed to OTHER/manual review instead of being treated as non-sensitive.

human-in-the-loop-gate (control)

The system supports analysts and sends ambiguous cases to manual review rather than resolving all surveillance outcomes automatically.

eval-driven-development (evaluation)

The team built an analyst-labeled dataset, defined precision/recall/F1, and optimized prompts against the expert baseline.

guardrails-io-filtering (safety)

The prompt uses a conservative rule to prevent doubtful news from being marked not sensitive.

observability-tracing (operations)

The Bedrock deployment is described as using monitoring, governance, and audit trails for regulatory compliance.

model-routing (inference)

Bedrock provides unified access to foundation models, and the prototype selected Claude Sonnet 3.5 as the model for price-sensitivity analysis.

"* Instructor library: Provided integration capabilities with Anthropic’s Claude Sonnet 3.5 model in Amazon Bedrock

• Amazon Bedrock: Served as the API infrastructure for model access
• Custom data processing pipelines (Python): For data ingestion and preprocessing"

Tradeoffs

Use two-stage classification

Chose separate sensitive and non-sensitive classification passes with merge rules over one direct three-class classifier.

Rationale: The split lets LSEG optimize different error types and explicitly route contradictions to manual review.

"* Step 1: Classify news articles as potentially price sensitive or other

• Step 2: Classify news articles as potentially price not sensitive or other"

— article, stated "* Step 1: Classify news articles as potentially price sensitive or other

• Step 2: Classify news articles as potentially price not sensitive or other"

Favor recall for sensitive news

Chose 100% recall target for PRICE_SENSITIVE/HARD_TO_DETERMINE classes over maximizing automatic non-sensitive clearance.

Rationale: The system should not miss potentially price-sensitive content even if that means more analyst review.

"* 100% precision in identifying non-sensitive news, allocating 6 articles to this category that analysts confirmed were non price sensitive

• 100% recall in detecting price-sensitive content, allocating 36 hard to determine and 28 price sensitive articles labelled by analysts into one of t"

— article, stated "price sensitive_ articles labelled by analysts into one of these two categories (never misclassifying price sensitive content)

• Automated analysis of complex financial news
• Detailed justifications for classification decisions
• Effective triaging of results by sensitivity level"

— article, stated

Use expert analyst labeling

Chose market surveillance analysts labeling a curated article set over unsupervised or purely synthetic evaluation.

Rationale: Price sensitivity is a domain judgment, so the benchmark needed expert surveillance labels.

Use Bedrock managed inference

Chose Claude Sonnet 3.5 via Amazon Bedrock over direct model hosting and management.

Rationale: Bedrock reduced model-management complexity while supporting scaling, monitoring, governance, and audit needs.

Gotchas (what bit them)

Perfect reported metrics come from a small evaluation sample.

Root cause: The 100% precision and recall figures are based on a curated 110-article evaluation set, with only 6 non-sensitive articles in the precision claim.

Resolution: Treat the result as promising prototype evidence and expand the evaluation dataset before relying on generalization.

False negatives are more dangerous than extra manual review.

Root cause: Missing price-sensitive news can undermine market-abuse detection, while false positives mainly add analyst workload.

Resolution: Optimize recall for price-sensitive content and use conservative OTHER/manual-review routing for doubt.

"* Precision for the NOT SENSITIVE class

• Recall for the PRICE SENSITIVE class"

— article, stated "price sensitive_ articles labelled by analysts into one of these two categories (never misclassifying price sensitive content)

• Automated analysis of complex financial news
• Detailed justifications for classification decisions
• Effective triaging of results by sensitivity level"

Rules-based surveillance systems can age poorly as market behavior changes.

Root cause: Markets, asset classes, venues, and bad-actor tactics evolve, increasing alert frequency and false positives.

Resolution: LSEG used a generative AI prototype to support news-sensitivity assessment and plans to add data sources and typologies over time.

Explainability is required for analyst trust and regulatory compliance.

Root cause: Market surveillance decisions may be reviewed by compliance teams and regulators, so opaque classifications are insufficient.

Resolution: The model produces justifications and the deployment maintains audit trails.

Evaluation

LSEG built a representative expert-labeled evaluation set from 250,000 RNS articles, using EDA across categories, instruments, and article length. 110 articles were labeled by market-surveillance analysts on a nine-point scale and collapsed into PRICE_NOT_SENSITIVE, HARD_TO_DETERMINE, and PRICE_SENSITIVE. The team evaluated precision, recall, and F1, optimizing precision for non-sensitive news and recall for price-sensitive news.

Metrics: precision for NOT SENSITIVE · recall for PRICE SENSITIVE · F1 · nine-point analyst price-sensitivity label · manual-review ambiguous count · data pipeline completion

"1. Data pipeline implementation: Successful ingestion and preprocessing of RNS data

2. Metric definition: Clear articulation of precision, recall, and F1 metrics
3. Workflow completion: Execution of comprehensive exploratory data analysis (EDA) and experimental workflows"

— article, implied "* 1–3: PRICE_NOT_SENSITIVE – Low probability of price sensitivity

• 4–6: HARD_TO_DETERMINE – Uncertain price sensitivity
• 7–9: PRICE_SENSITIVE – High probability of price sensitivity"

— article, implied "* 1–3: PRICE_NOT_SENSITIVE – Low probability of price sensitivity

• 4–6: HARD_TO_DETERMINE – Uncertain price sensitivity
• 7–9: PRICE_SENSITIVE – High probability of price sensitivity"

— article, implied "* 1–3: PRICE_NOT_SENSITIVE – Low probability of price sensitivity

• 4–6: HARD_TO_DETERMINE – Uncertain price sensitivity
• 7–9: PRICE_SENSITIVE – High probability of price sensitivity"

— article, implied "* Precision for the NOT SENSITIVE class

• Recall for the PRICE SENSITIVE class"

— article, implied "* Precision for the NOT SENSITIVE class

• Recall for the PRICE SENSITIVE class"

— article, implied

Outcomes

• Surveillance Guide classified RNS news for price sensitivity and produced analyst-facing justifications.
• The representative evaluation reported 100% precision for six non-sensitive articles.
• The representative evaluation reported 100% recall across 36 hard-to-determine and 28 price-sensitive articles.
• The system reduces manual review time directionally by helping analysts focus on critical cases, though the source does not quantify hours saved.
• LSEG plans to validate in live environments alongside manual processes and expand data sources, prompting methods, evaluation scale, and abuse typologies.

The source reports more than £1 trillion of securities traded and reported by 400 members annually, roughly 250,000 RNS articles across 6 months, 110 expert-labeled evaluation articles, a nine-point sensitivity scale, 6 non-sensitive articles in the precision result, 36 hard-to-determine and 28 price-sensitive articles in the recall result, and a 6-week evaluation period.

• 100% recall in detecting price-sensitive content, allocating 36 hard to determine and 28 price sensitive articles labelled by analysts into one of t"