Semantic Search for Aviation Safety Reports Using Embeddings and Hybrid Search

beams · Other · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Aviation safety report search and risk analysis

Beams needed safety analysts to search and analyze large volumes of dense, multilingual, technical free-text aviation reports without manually reading every report or enumerating every synonym.

Constraints:

• Reports are written by pilots, cabin crew, and ground operations in free text under time pressure.
• Aviation language contains jargon, abbreviations, negation, and airline-specific terminology.
• Analysts need both semantic similarity and structured filters over extracted or customer-supplied attributes.
• The production stack had to fit AWS infrastructure, compliance requirements, and acceptable user latency.

Architecture

Beams built a hybrid aviation safety search system on AWS: reports arrive through S3 or web flows, PII removal and translation run through Lambda/SQS orchestration, Azure OpenAI text-embedding-3-large produces embeddings, PGVector on PostgreSQL/RDS stores vectors, retrieval and reranking prioritize semantically similar reports, and a filter service combines semantic search with structured DNF filters over 35+ aviation attributes.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Aviation safety reports"] --> B["Ingestion pipeline"]
  B --> C["SQS orchestration"]
  B --> D["Azure OpenAI embeddings"]
  D --> E["RDS PGVector store"]
  E --> F["Retriever"]
  F --> G["Reranker"]
  E --> H["Filter service"]
  I["Natural-language filter translator"] --> H
  G --> J["Search results"]
  H --> J

Components

Data flow

Customer reports enter through S3 buckets or realtime web submissions, pass through PII removal and translation, are embedded with Azure OpenAI, land in RDS with PGVector, and are searched through a retrieval-rerank path plus structured filters generated from a UI or natural-language filter translation.

Why this architecture

Beams chose a simple AWS/RDS/PGVector foundation because the team wanted to solve the aviation safety workflow with minimal complexity first, while hybrid search was necessary because semantic similarity alone did not cover structured safety filters.

Models used: Azure OpenAI text-embedding-3-large · rerank model · LLM for filter translation · small PII models

Scale

• Airlines can accumulate hundreds of thousands of safety reports for analysis.
  “because you want to run an analysis across like the hundreds of thousands of reports that you've been getting”transcript, stated
• The filter service handles more than 35 attributes with different operators.
  “each of those like 35 plus attributes, the different operators like in, not in, they all require different handling”transcript, stated
• The embedding configuration used text-embedding-3-large at around 2,000 dimensions as the precision-recall sweet spot.
  “using the text embedding 3 large and via empirical measurements for our purposes, we basically found that with a dimensionality of 2,00”transcript, stated
• Bulk uploads can create processing spikes, so queues process reports bit by bit.
  “they just send us data in bulk a few times per day. So there you have these clear spikes”transcript, stated

  “by basically storing these things in cues and like processing them uh bit by bit because some of the processing can take a fair amount of time.”transcript, stated

Alternatives considered (roads not taken)

Specialized vector databases first

Rejected because: The team favored starting simple on existing AWS/RDS infrastructure rather than adopting more complex vector database infrastructure immediately.

Semantic search without structured filtering

Rejected because: Aviation users already had structured flight and report metadata, and filtering became core to project success.

Best available reranker regardless of latency

Rejected because: User experience would suffer if users waited 10-15 seconds, so Beams chose a lightweight model with one-to-two-second speed.

Patterns

hybrid-search (retrieval)

Beams combines semantic vector search with structured filters over aviation attributes rather than treating search as only embeddings.

retrieval-ranking (retrieval)

The system retrieves semantically similar reports, then reranks the candidate set for better precision.

privacy-preserving-processing (privacy)

The ingestion pipeline removes PII before embedding/search processing, sometimes with customer-side small models.

async-message-orchestration (infrastructure)

SQS queues orchestrate Lambda-based report processing, including spike handling for bulk customer uploads.

Tradeoffs

Use PGVector on RDS.

Chose PostgreSQL/RDS with PGVector over specialized vector database first.

Rationale: PGVector let the team stay on AWS/RDS, use SQL, and keep complexity low for initial user testing.

Choose hybrid search.

Chose semantic search plus structured filtering over semantic similarity alone.

Rationale: Safety analysts need to combine incident meaning with flight data, classifications, and attribute constraints.

Queue report processing.

Chose SQS-backed asynchronous processing over instant synchronous processing for every ingest.

Rationale: Bulk customer uploads create spikes and some processing takes meaningful time.

Gotchas (what bit them)

Aviation text is hard for generic search and LLMs.

Root cause: Reports contain dense jargon, inconsistent abbreviations, multilingual text, and negation.

Resolution: Use embeddings, translation, domain classifications, and hybrid filters rather than only keyword matching.

Filter logic became complex even with tests.

Root cause: Dozens of attributes, operators, joins, and subselects interacted in SQL generation.

Resolution: Experiment with natural-language filter generation and consider more mature search infrastructure.

The best reranker can be too slow for users.

Root cause: Reranking precision competes with acceptable response time.

Resolution: Choose a lightweight reranker with good precision and one-to-two-second response expectations.

Evaluation

Beams evaluated choices empirically through precision-recall measurements for embeddings, reranker speed/precision tradeoffs, test-driven filter development, and user reception of the search feature.

Metrics: precision · recall · reranker latency · filter correctness · user acceptance · retrieval speed

Outcomes

• Users could search semantically across aviation safety reports without manually enumerating every phrasing variation.
• The feature was welcomed by users, but filter complexity pushed the team toward more mature search infrastructure.
• PGVector/RDS provided a simple starting point while hybrid search became a core requirement.

The source describes hundreds of thousands of reports, 35+ filter attributes, queued processing for bulk spikes, and retrieval expected in the microseconds area after ingestion.