Dark Vessel Detection System Using SAR Imagery and ML

defense_innovation_unit · Government · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Maritime surveillance and illegal fishing detection

The Defense Innovation Unit needed to detect illegal, unreported, and unregulated fishing vessels that turn off AIS transponders, across ocean areas too large for patrol vessels or manual image review to cover.

Constraints:

Illegal vessels can disable AIS and become invisible to transponder-based monitoring.
The ocean and exclusive economic zones are too large for continuous patrol coverage.
SAR imagery has artifacts, polarizations, shore effects, and large 20,000-by-20,000-pixel scenes.
Useful models require large annotated datasets and task-specific metrics.
Operational value depends on deploying detections into maritime-domain systems used by many countries.

Architecture

DIU and partners built a SAR-based dark-vessel detection pipeline: collect large open-source SAR scenes and co-registered metadata, annotate vessels with automated and manual quality checks, release a public challenge around detection/classification/shoreline-weighted metrics, train models, wrap the best model with preprocessing, and deploy detections into SeaVision for maritime-domain awareness users.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  sar-satellite-imagery["SAR satellite imagery"] --> annotation-dataset["Annotation dataset"]
  annotation-dataset --> public-challenge["Public challenge"]
  public-challenge --> dark-vessel-model["Dark vessel model"]
  sar-satellite-imagery --> dark-vessel-model
  dark-vessel-model --> preprocessing-pipeline["Preprocessing pipeline"]
  dark-vessel-model --> ais-correlation["AIS correlation"]
  preprocessing-pipeline --> seavision["SeaVision"]
  ais-correlation --> seavision

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`SAR satellite imagery`	Radar imagery source that can observe vessels through cloud cover, night, and visual concealment.	`Annotation dataset`, `Dark vessel model`
`Annotation dataset`	Quality-checked SAR scenes with automated and manual vessel annotations plus co-registered channels.	`SAR satellite imagery`, `Public challenge`
`Public challenge`	Prize-backed model-development process for vessel detection, sizing, classification, and shoreline performance.	`Annotation dataset`, `Dark vessel model`
`Dark vessel model`	ML detector that identifies candidate vessels in SAR imagery.	`SAR satellite imagery`, `Public challenge`, `Preprocessing pipeline`, `AIS correlation`
`Preprocessing pipeline`	Operational wrapper around model inputs and inference.	`Dark vessel model`, `SeaVision`
`AIS correlation`	Compares SAR detections against known AIS broadcasts to identify uncorrelated vessels.	`Dark vessel model`, `SeaVision`
`SeaVision`	Maritime domain awareness platform where users view AIS tracks and dark-vessel detections.	`Preprocessing pipeline`, `AIS correlation`

Data flow

SAR satellites capture radar scenes. The project assembles multimodal SAR data, bathymetry, wind, land/ice masks, and annotations. Challenge participants train vessel-detection models against task-specific metrics. DIU wraps the chosen model with preprocessing, correlates SAR detections against AIS broadcasts, and surfaces uncorrelated detections as dark vessels in SeaVision.

“we actually collected a large amount of open-source SAR scenes uh and it's a multimodal data that goes along with that”transcript, stated

“we actually were able to take it wrap it up wrap up the pre-processing pipeline um and I spend a lot of time with hands on keyboard”transcript, stated

Why this architecture

SAR was chosen because vessels cannot evade it with clouds, nighttime, or visual covers, and because AIS-only monitoring misses vessels that turn off transponders. The public challenge created model improvements on a specialized dataset, and SeaVision delivered the detections into an existing operational platform.

“space Bas radar is effectively impervious to cloud cover to nighttime to covering yourself with a blue tarp”transcript, stated

“those are SAR detections that are not correlated with a um that are not correlated with an AIS signal that's known and so these are all what you would call dark vessels”transcript, stated

Models used: Faster R-CNN baseline model · Challenge-submitted SAR vessel detection models

Scale

The dataset included about 243,000 annotations and 82 million square kilometers of imagery.
“we ended up with something like 243,000 annotations and 82 million square kilometers of imagery”transcript, stated
The SAR scenes were very large, about 20,000 by 20,000 pixels each.
“that data set it was 991 scenes now these are Big these are each 20,000 pixels by 20,000 pixels”transcript, stated
The deployed SeaVision system is used by over 100 countries.
“we deployed this thing um to a system called C Vision that is used by hundreds of countries around the world by over 100 countries around the world”transcript, stated
The speaker said one in five fish caught worldwide originates from illegal, unreported, and unregulated fishing.
“one in five fish caught around the world originate from illegal unreported and unregulated fishing”transcript, stated

Alternatives considered (roads not taken)

Rely on AIS broadcasts

Rejected because: Illegal vessels can turn AIS off, leaving no transponder signal to monitor.

“if you're fishing illegally uh or not where you're supposed to be you tend to turn those off because you don't tend to want to be broadcasting that you're somewhere you shouldn't be”transcript, stated

Rely on patrol vessels and manual image review

Rejected because: The ocean is too large, resourcing is uneven, and manual image analysis is slow and unreliable.

“finding ships that are doing that is traditionally very difficult because the ocean is enormous and you never have enough vessels to patrol everywhere you want to patrol”transcript, stated

Use standard optical imagery only

Rejected because: SAR provides all-weather/night capability and detects metal vessels in conditions optical imagery cannot.

“metal lights up like a Christmas tree um but what this doesn't show you is that sard your space Bas radar is effectively impervious to cloud cover to nighttime”transcript, stated

Patterns

`data-quality-loop` (data)

The project combined automated and manual annotations with quality checks before model training and challenge release.

“that we then created an enormous number of annotations using a combination of automated means and manual means and quality check these things”transcript, stated

`eval-driven-development` (evaluation)

The challenge task and metric were designed around operational needs such as object detection, size, vessel class, and near-shore performance.

“we have a metric that's that's pretty interesting it's a weighted average of of a bunch of different things”transcript, stated

`batch-offline-processing` (operations)

The system processes satellite imagery and queues inference to provide dark-vessel detections efficiently at large geographic scale.

“there's a fun kind of way that we do the queuing and the inference to to minimize costs um kind of on the you know from from a deployment perspective”transcript, stated

Tradeoffs

Use SAR imagery for dark-vessel detection

Chose space-based radar detections over AIS-only or optical-only monitoring.

Rationale: SAR can reveal vessels that disable AIS and is robust to cloud, night, and visual concealment.

“space Bas radar is effectively impervious to cloud cover to nighttime to covering yourself with a blue tarp”transcript, stated

Use public challenges for model development

Chose released dataset, task, and prize-backed competition over only internal government model development.

Rationale: The challenge attracted broad participation and improved performance over time.

“over time we got we ended up with a lot of participation in these sorts of challenges we had thousands of submissions from up around the world”transcript, stated

Gotchas (what bit them)

SAR images are not ordinary computer-vision inputs.

Root cause: Radar reflectivity, polarizations, shore artifacts, wind farms, and oil slicks create different visual artifacts from optical imagery.

Resolution: Design SAR-specific data augmentation, task metrics, and multimodal co-registered channels.

“there's a bunch of different weird types of artifacts that occur in this imagery that you won't find in electrooptical or kind of standard imagery”transcript, stated

Large SAR scenes make local chipping lose important context.

Root cause: Mother-ship patterns and vessel arrays may depend on long-range spatial relationships.

Resolution: Treat long-range context as a systems requirement rather than a standard object-detection detail.

“if I chip this image like I would often do to say fit it on a on a standard GPU um I'm going to lose sometimes the long range context”transcript, stated

Operational maritime AI still needs ongoing model maintenance.

Root cause: Sensors, environments, and bureaucratic deployment processes change over time.

Resolution: Continue surveying model performance, updating models, and managing operational refresh processes.

“we still have to constantly figure out a way to survey to to make sure that our models are working right to make sure that we update them at some reasonable frequency”transcript, stated

Evaluation

The task evaluated vessel detection, object size, vessel-vs-non-vessel classification, fishing-vessel classification, and near-shore performance with a weighted metric; deployment also requires ongoing model performance surveillance.

Metrics: Object detection · Object size · Vessel classification · Fishing-vessel classification · Near-shore weighted performance · Operational model health

“we care about can you detect the objects sure we care about can you tell me how big it is we care about can you classify whether it's a an object as a vessel or not”transcript, stated

Outcomes

The project produced a large annotated SAR dataset for dark-vessel detection.
Public challenges drew thousands of submissions and improved performance over time.
The model and preprocessing pipeline were deployed into SeaVision.
SeaVision users can see SAR detections not correlated with AIS and identify likely dark vessels.
The approach supports lower-cost monitoring for countries with limited patrol resources.

The operational platform is used by over 100 countries and the dataset covered 82 million square kilometers of imagery.