AI-Powered Video Workflow Orchestration Platform for Broadcasting

cires21 · Media & Entertainment · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Broadcast media workflow automation

Cires21 needed to help broadcasters replace fragmented media applications and duplicated work with a unified AI layer for live clipping, subtitling, metadata generation, summaries, highlights, and social-media repurposing.

Constraints:

• broadcast workflows combine live feeds, VOD assets, CMS/MAM systems, clipping services, subtitles, branding, and content protection
• clients integrate through APIs more often than a standalone UI
• live video has different latency and inference requirements from VOD
• content protection and privacy are central requirements
• the platform must orchestrate classic media services, custom AI models, Bedrock metadata generation, and newer agents

Architecture

Cires21 built MediaCopilot as a serverless AWS platform. Lambda and API Gateway expose APIs, Step Functions orchestrate media and AI workflows, Cognito provides authentication, CloudFront/S3 handle delivery and content protection, and AWS Media Services transcode and ingest video. Custom AI pipelines handle ASR, voice activity detection, diarization, and scene detection. Amazon Bedrock creates complex metadata such as subtitles, highlights, and summaries. More recently, MediaCopilot added AgentCore agents and MCP servers so users can ask for multi-step tasks such as finding viral moments, making vertical clips, burning subtitles, and exporting captions/metadata.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  mediacopilot-api-layer["mediacopilot-api-layer"] --> step-functions-orchestrator["step-functions-orchestrator"]
  mediacopilot-api-layer["mediacopilot-api-layer"] --> client-workflow-integrations["client-workflow-integrations"]
  step-functions-orchestrator["step-functions-orchestrator"] --> aws-media-services["aws-media-services"]
  step-functions-orchestrator["step-functions-orchestrator"] --> custom-ai-pipelines["custom-ai-pipelines"]
  step-functions-orchestrator["step-functions-orchestrator"] --> bedrock-metadata-service["bedrock-metadata-service"]
  step-functions-orchestrator["step-functions-orchestrator"] --> video-editor["video-editor"]
  aws-media-services["aws-media-services"] --> custom-ai-pipelines["custom-ai-pipelines"]
  custom-ai-pipelines["custom-ai-pipelines"] --> bedrock-metadata-service["bedrock-metadata-service"]
  custom-ai-pipelines["custom-ai-pipelines"] --> sagemaker-inference["sagemaker-inference"]
  bedrock-metadata-service["bedrock-metadata-service"] --> video-editor["video-editor"]
  video-editor["video-editor"] --> client-workflow-integrations["client-workflow-integrations"]
  agentcore-mcp-server["agentcore-mcp-server"] --> agentcore-agent-runtime["agentcore-agent-runtime"]
  agentcore-mcp-server["agentcore-mcp-server"] --> mediacopilot-api-layer["mediacopilot-api-layer"]
  agentcore-agent-runtime["agentcore-agent-runtime"] --> agentcore-memory["agentcore-memory"]
  agentcore-agent-runtime["agentcore-agent-runtime"] --> agentcore-observability["agentcore-observability"]

Components

Data flow

Uploaded or live video enters MediaConvert, MediaLive, MediaPackage, or harvest jobs, then Step Functions orchestrates downstream AI processing. Custom speech/video models and Bedrock generate transcripts, diarization, scene boundaries, subtitles, highlights, and summaries. Clients call MediaCopilot through APIs from their CMS, MAM, or live clipping systems. AgentCore Gateway exposes API capabilities as MCP tools, and AgentCore Runtime runs stateless per-session agents that call those tools to execute complex media tasks.

Why this architecture

Cires21 chose a serverless, API-first media platform because broadcast clients already have CMS, MAM, and live clipping workflows and need an AI layer that can integrate rather than replace everything. Step Functions provides deterministic workflow orchestration, while AgentCore and MCP add a natural-language agent layer for compound tasks. Serverless services improve time to market, reliability, scalability, and content isolation.

Models used: Amazon Bedrock · AWS AgentCore · MCP · Amazon SageMaker · custom automatic speech recognition · voice activity detection · diarization · scene detection · AWS Elemental MediaConvert · AWS Elemental MediaLive · AWS Elemental MediaPackage · AWS Step Functions

Scale

• Cires21 has been operating for almost 20 years.
  “Cires21 started its business almost 20 years ago.”transcript, stated
• Cires21 began developing its own AI pipelines about two years before the talk.
  “And two years ago, we started to develop our own AI pipelines.”transcript, stated
• The AgentCore work began about two months before the presentation.
  “And two months ago, more or less, we started to develop agents for MediaCopilot.”transcript, stated
• AgentCore Gateway can deploy an MCP server in minutes instead of hours.
  “allows us to deploy an MCP server in minutes instead of hours”transcript, stated

Alternatives considered (roads not taken)

Fragmented application ecosystem

Rejected because: Broadcasters were using many separate tools, causing slow delivery, high costs, and duplicated work.

UI-only product

Rejected because: Most clients need to integrate MediaCopilot directly into their existing broadcast systems.

Synchronous SageMaker endpoint for live workloads

Rejected because: The team found synchronous inference is good for VOD but not enough for live video, where realtime/serverless patterns are needed.

Patterns

tool-use-orchestration (agent architecture)

AgentCore agents call MCP tools generated from MediaCopilot APIs to perform compound media tasks.

durable-execution (workflow)

AWS Step Functions orchestrates the media processing workflows across transcoding, AI models, Bedrock, and exports.

agent-memory (personalization)

AgentCore memory stores session context and user preferences for styling or text generation.

observability-tracing (operations)

AgentCore observability is used to monitor sessions, identify bottlenecks, and optimize workflows.

Tradeoffs

Build an API-first AI layer.

Chose MediaCopilot APIs integrated into client workflows over Standalone UI as the main surface.

Rationale: Broadcast clients already operate CMS, MAM, and live clipping systems, so API integration fits their workflows.

Use serverless architecture.

Chose Lambda, API Gateway, Step Functions, CloudFront, S3, and managed media services over Manually managed infrastructure.

Rationale: Cires21 wanted faster time to market plus reliability and scalability for variable media workloads.

Add AgentCore on top of existing workflows.

Chose MCP/AgentCore layer over MediaCopilot APIs over Only direct API orchestration.

Rationale: Agents let users request multi-step creative tasks in natural language while reusing existing media workflow capabilities.

Gotchas (what bit them)

The same SageMaker endpoint pattern does not fit VOD and live video.

Root cause: Synchronous inference works for VOD but live workflows need lower-latency realtime or serverless inference.

Resolution: Use realtime/serverless deployment models for live and segmented processing for parallelism.

Long media inference needs segmentation to avoid slow end-to-end processing.

Root cause: Processing entire videos serially blocks parallel inference.

Resolution: Segment video and process segments in parallel to reduce inference time.

Evaluation

The source emphasizes operational monitoring and workflow optimization rather than a formal benchmark. Cires21 uses AgentCore observability to monitor sessions and bottlenecks, and uses workflow/inference changes such as segmented processing and deployment model selection to improve performance.

Metrics: workflow bottlenecks · inference time · live versus VOD suitability · content protection · time to deploy MCP server

Outcomes

• MediaCopilot unifies multiple broadcast AI capabilities into one API-accessible platform.
• The platform provides ASR, diarization, scene detection, subtitles, highlights, summaries, live clipping, and branded video editing.
• Most clients consume MediaCopilot through APIs inside their existing workflows.
• AgentCore agents can perform compound tasks like finding viral interview moments, making vertical social clips, burning subtitles, and exporting captions/metadata.
• The public talk claims faster content delivery and lower cost but does not quantify the savings.

Cires21 has nearly 20 years of broadcast live-streaming history, two years of in-house AI pipeline development, and recently added AgentCore/MCP agents.