Multimodal Art Collection Search Using Vector Databases and LLMs

actum_digital · Media & Entertainment · year n/a · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Art collection multimodal search

Actum Digital needed search over a very large art collection where users could combine text descriptions and visual similarity rather than relying only on metadata.

Constraints:

• The collection contains approximately 40 million assets.
• Search must handle visual similarity and textual descriptions.
• Indexing and re-indexing costs matter at regional scale.
• The case is demo-oriented and does not publish production quality metrics.

Architecture

The team used AWS Bedrock embedding models, OpenSearch vector indexing, and a hybrid multimodal search workflow so users could search by text, similar images, or combined modalities.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  art-asset-collection["Art Asset Collection"]
  bedrock-embedding-layer["Bedrock Embedding Layer"]
  opensearch-vector-index["OpenSearch Vector Index"]
  hybrid-search-interface["Hybrid Search Interface"]
  art-asset-collection --> bedrock-embedding-layer
  bedrock-embedding-layer --> opensearch-vector-index
  opensearch-vector-index --> hybrid-search-interface

Components

Data flow

Art assets are embedded with AWS Bedrock models, indexed in OpenSearch, and queried through a hybrid search layer that can use text prompts, image similarity, or combined filters.

Why this architecture

A hybrid text-and-image vector search architecture matched the collection's visual nature while keeping the stack on AWS-managed services.

Models used: AWS Bedrock embedding models · OpenSearch vector search

Scale

• The target collection has approximately 40 million assets.
  “an art collection containing approximately 40 million assets.”summary, stated
• Infrastructure cost is about $1,000 per month per region.
  “The infrastructure runs at approximately $1,000 per month per region”summary, stated

Alternatives considered (roads not taken)

Text-only metadata search

Rejected because: It cannot retrieve assets based on visual similarity and material/style cues.

Frequent full re-indexing

Rejected because: Re-indexing at collection scale is a cost optimization concern.

Patterns

rag-retrieval (retrieval)

The collection is embedded and indexed so users can retrieve visually or textually similar assets.

cost-aware-routing (operations)

The design treats indexing and model choices as cost levers for regional deployment.

Tradeoffs

Use managed AWS services for embeddings and search

Chose AWS Bedrock and OpenSearch over Custom embedding serving and vector infrastructure.

Rationale: The managed stack supports a demoable multimodal search workflow while leaving cost optimization as the main operational concern.

Hybrid retrieval

Chose Text plus image similarity search over Only keyword or only image-nearest-neighbor search.

Rationale: Users need to refine results across visual and textual signals.

Gotchas (what bit them)

Vector search at tens of millions of assets makes re-indexing expensive.

Root cause: Every embedding/indexing change can affect a very large collection.

Resolution: Reduce re-indexing operations and evaluate fine-tuned models as a cost optimization path.

Single-modality search misses user intent in art discovery.

Root cause: Art queries often combine visual similarity, materials, style, and text descriptions.

Resolution: Expose hybrid text and image search paths.

Evaluation

The case demonstrates search capabilities but does not provide measured retrieval quality, latency, or user-success metrics.

Metrics: Demo result relevance · Monthly regional infrastructure cost

Outcomes

• Users can search art assets by text descriptions.
• Users can find visually similar images.
• The team identified re-indexing reduction and fine-tuned models as cost optimization paths.

The collection scale is approximately 40 million assets, with regional infrastructure around $1,000 per month.