Scaling AI Systems for Unstructured Data Processing: Logical Data Models and Embedding Optimization

coactive_ai · Tech · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Scalable AI systems for unstructured data processing

CoActive AI needed to make unstructured text, image, audio, and video data usable in production AI systems while avoiding logical-data-model bottlenecks and repeated foundation-model compute costs.

Constraints:

• Unstructured data is physically stored in formats that do not match how AI models consume task-specific inputs.
• Different modalities and tasks create many logical data models from the same underlying physical data.
• Neither data engineering nor AI teams naturally own the transformation layer.
• Running multiple foundation models over the same data can make costs grow out of control.
• Visual and video data dramatically increases the storage and compute burden relative to tabular or text data.

Architecture

CoActive frames production AI for unstructured data as a data-system problem. The architecture separates physical storage from task-specific logical data models, assigns ownership through hybrid data+AI teams, and treats embeddings as reusable cached feature-extractor outputs that can feed many downstream tasks rather than rerunning full foundation-model pipelines for every application.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Unstructured data store"] --> B["Logical data model layer"]
  C["Hybrid data AI team"] --> B
  B --> D["Foundation model feature extractor"]
  D --> E["Embedding cache"]
  E --> F["Task-specific AI applications"]
  F --> G["Downstream product consumers"]
  C --> G

Components

Data flow

Unstructured assets enter storage, are transformed into logical models based on modality and task, pass through foundation-model feature extraction, and produce reusable embeddings or metadata. Hybrid data+AI teams own the transformation layer so downstream product and BI consumers receive structured outputs without forcing every task to rerun the same expensive compute.

Why this architecture

The architecture avoids treating AI as a monolithic metadata generator. It makes logical data models and reusable embeddings first-class infrastructure so cost and bottlenecks do not grow with every new downstream task.

Models used: foundation models · embedding models · visual/video AI models

Scale

• The speaker frames the lessons as coming from more than three years of user research and two years of building.
  “lessons that we learned in designing in such a system for three plus years of user research and two procedures are building”transcript, stated
• The talk compares 10 million float rows at about 40 MB with 10 million text documents at about 40 GB.
  “10 million rows if you say flow 32s are going to be about 40 megabytes in size”transcript, stated
• Moving from text to video pushes data volume from gigabytes toward terabytes.
  “once you go from text to uh video this is this you know you go from uh from 40 gigabytes to something the order now terabytes”transcript, stated

Alternatives considered (roads not taken)

Treat AI as a monolith that generates metadata

Rejected because: This hides the mismatch between physical storage and model-specific logical inputs.

Let every downstream task rerun the same foundation-model compute

Rejected because: Repeated feature extraction is expensive and creates cost bottlenecks.

Leave transformation ownership split between data and AI teams

Rejected because: No team owns the impedance mismatch, creating a pipeline bottleneck.

Patterns

semantic-caching (cost-optimization)

Embeddings are treated as reusable cached foundation-model outputs that can serve multiple downstream tasks.

cost-aware-routing (optimization)

The system design reduces repeated expensive foundation-model compute before downstream tasks run.

data-quality-loop (data)

Hybrid data+AI teams own logical data transformations so downstream AI tasks receive the right input representation.

Tradeoffs

Data model ownership

Chose hybrid data+AI teams over separate data-engineering storage and ad hoc AI transformations.

Rationale: The bottleneck sits between storage and AI consumption, so ownership must span both sides.

Foundation-model compute strategy

Chose break up the monolith and reuse embeddings over rerunning the full model for every task.

Rationale: Reusable intermediate outputs reduce cost and latency as tasks multiply.

Gotchas (what bit them)

Physical storage models do not match AI consumption needs.

Root cause: Models need task- and modality-specific logical inputs, not generic blob-store representations.

Resolution: Create an explicit logical data model layer owned by hybrid data+AI teams.

No one owns the transformation layer by default.

Root cause: Data teams optimize storage while AI teams build ad hoc fixes for model input needs.

Resolution: Create hybrid data+AI teams to own the mismatch and optimization pathway.

Multiple foundation-model applications can make costs explode.

Root cause: The same feature extraction compute is repeated for each downstream task.

Resolution: Cache embeddings or intermediate feature outputs and reuse them across tasks.

Evaluation

The talk does not describe a formal evaluation harness; it evaluates architecture choices through observed bottlenecks, cost growth, latency reduction, and data-volume scaling examples.

Metrics: latency reduction · cost reduction · compute reuse · data volume by modality · AI pipeline bottleneck removal

Outcomes

• Logical data models are elevated as a production bottleneck for unstructured AI systems.
• Hybrid data+AI teams are proposed as the ownership model for that bottleneck.
• Embeddings are reframed as cached compute, not only semantic representations.
• The architecture targets unstructured text, image, audio, and video workloads.

The case is a design-lessons talk based on multi-year research/building experience, with scale examples showing storage growth from megabytes to gigabytes to terabytes as modalities become richer.