Agentic News Analysis Platform for Digital Asset Market Making

fsi · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Digital asset market making

Market makers needed to analyze streaming news and social posts within seconds to manage adverse selection and inventory risk.

Constraints:

• Sub-10-second latency
• Streaming data
• Deduplication
• Impact/sentiment reasoning

Architecture

The AWS-based news-analysis platform ingests streaming news into S3, triggers EventBridge and Lambda classification, calls a DeepSeek reasoning model to tag asset, urgency, and sentiment metadata, and writes results into Aurora Postgres and OpenSearch. A separate deduplication pipeline uses BGE-m3 embeddings and OpenSearch similarity checks to drop duplicate crypto-news reports before expensive LLM analysis. Unique events flow through impact assessment and spread-widening recommendation, then a report is posted to a Trader Deck Slack channel where humans decide whether to act.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  news-streaming-api[News streaming API] --> s3-news-bucket[S3 news bucket]
  s3-news-bucket --> eventbridge[EventBridge]
  eventbridge --> classification-lambda[Classification Lambda]
  classification-lambda --> deepseek-reasoning-model[DeepSeek reasoning model]
  classification-lambda --> deduplication-pipeline[Deduplication pipeline]
  deduplication-pipeline --> bge-m3-embedding-model[BGE-m3 embedding model]
  deduplication-pipeline --> opensearch[OpenSearch]
  classification-lambda --> aurora-postgres[Aurora Postgres]
  classification-lambda --> opensearch
  deepseek-reasoning-model --> trader-deck-slack[Trader Deck Slack]
  q-cli[Q CLI] --> opensearch

Components

Data flow

Streaming news arrives through a news API into S3, which triggers EventBridge and a Lambda classifier. The Lambda calls DeepSeek to classify asset, urgency, and sentiment metadata and stores tags in Aurora Postgres and OpenSearch. In parallel, a deduplication pipeline embeds incoming items with BGE-m3, compares similarity against OpenSearch duplicate and unique collections, and sends only truly unique news to analysis. The platform produces impact and spread-widening recommendations, generates a prediction report, and posts it to Trader Deck Slack for a human trading decision.

Why this architecture

The market-making use case is latency-bound and duplicate-heavy. LLM reasoning handles contextual, non-numeric events that dictionary matching and traditional supervised models struggle with, while inference serving and deduplication are optimized so only unique, impactful events consume expensive model time. Human oversight remains at the final trading decision point.

Models used: DeepSeek · BGE-m3 · Claude · Naive Bayes · FinBERT

Scale

• The platform must support market reactions within seconds rather than minutes.
  “we are talking about maybe in just a few seconds, we have to finish all the analysis determining how the market is going to be impact and action in the market immediately.”transcript, stated
• Inference optimization moved from 80 output tokens/s on SageMaker JumpStart to 140 with vLLM and 180 with SGLang.
  “we started with 80 output tokens per second. Not good enough. Later on in April, 2025, we replaced it with vLLM.”transcript, stated

  “In August, we replaced vLLM with SGLang, and this with the help of speculative decoding. Finally, we achieved 180 output tokens per second.”transcript, stated
• The optimized system targets under-10-second end-to-end latency and doubles news ingestion processing in the same time window.
  “Going from 80 to 180, which means that we can double our news ingestion processing in the same time window. These optimizations enable the end-to-end latency under 10 seconds.”transcript, stated

  “These optimizations enable the end-to-end latency under 10 seconds. That avoids the adverse selections avoidance possible.”transcript, stated
• Deduplication is needed because the same digital-asset news may be reported about 50 times across channels.
  “In crypto, in digital asset, the same news gets reported 50 times. For example, like, "Ethereum upgrade delayed" appears on Twitter, Reddit, Discord, the Telegrams, all over it and all within minutes.”transcript, stated

Alternatives considered (roads not taken)

Dictionary matching with sentiment lexicons

Rejected because: Limited vocabulary cannot handle contextual market meanings, such as a phrase that looks negative but is bullish.

Statistical machine learning such as Naive Bayes or FinBERT

Rejected because: It improves generalization but requires massive labeled data, spending, and time to market.

Processing every repeated news item through the LLM

Rejected because: Duplicate reports burn money and kill latency, so a deduplication pipeline screens them first.

Fine-tuning the LLM for every market reasoning task

Rejected because: The team keeps the reasoning model prompt-engineered and fine-tunes the smaller embedding layer for cost-effective domain behavior.

Patterns

fine-tuning-adaptation (embeddings)

The team fine-tuned embeddings for news deduplication and similarity.

latency-optimization (serving)

Inference infrastructure was optimized to meet near-real-time trading requirements.

Tradeoffs

How to handle duplicate crypto news before expensive model reasoning.

Chose embedding-based deduplication with fine-tuned BGE-m3 and OpenSearch collections over sending every duplicate report through the LLM.

Rationale: Repeated reports would burn cost and compound latency in a trading workflow where milliseconds matter.

Where to spend fine-tuning effort.

Chose fine-tune the small embedding layer and prompt-engineer the reasoning LLM over fine-tuning the LLM itself for the whole workflow.

Rationale: The team argues general reasoning models can handle novel events, while domain-specific embeddings improve duplicate separation with lower cost and faster time to market.

Whether the model should execute trading actions automatically.

Chose human decisioning through Trader Deck Slack over fully autonomous trading action from the prediction pipeline.

Rationale: The platform automates analysis and recommendation, but final action remains under trader oversight.

Gotchas (what bit them)

Generic embeddings were not good enough for the domain.

Root cause: Financial news similarity and deduplication require domain-specific behavior.

Resolution: The team fine-tuned BGE-m3 embeddings.

Evaluation

The team measured output tokens per second, end-to-end latency, and processing capacity.

Metrics: tokens per second · latency · capacity

Outcomes

• The platform reached under-10-second end-to-end latency and doubled news capacity, according to the summary.

Human decisioning remains in the final trading action path.