Building Production AI Agents with API Platform and Multi-Modal Capabilities

manus_ai · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: General-purpose AI agent API platform

Manus needed to expose a production AI agent through many surfaces: web, mobile, Slack, email, Microsoft 365, browser operation, and a programmatic API. The platform had to support long-running asynchronous tasks, multimodal files, sandboxed code execution, connectors, webhooks, multi-turn state, and production integration patterns while hiding much of the infrastructure complexity from developers.

Constraints:

• The same general agent must work across UI, Slack, API, browser, and workflow integrations.
• Agent tasks can run for minutes, so synchronous request/response and polling-only patterns do not scale cleanly.
• Developers need to upload sensitive files, public URLs, screenshots, images, PDFs, and generated artifacts with clear lifecycle control.
• Slack integrations must acknowledge events quickly and preserve thread-to-task state across multi-turn conversations.
• Agents need sandboxed execution broad enough for packages, services, browser work, embeddings, and custom code.
• Browser-control features require careful user permission handling before API exposure.

Architecture

Manus exposes a general AI agent through an API with task IDs, task URLs, asynchronous task states, file uploads, URL attachments, base64 image inputs, connectors, and webhooks. Each Manus chat/task runs in a sandbox with enough capability to install packages and services, build web apps, run scripts, call external APIs, and operate browsers. The API supports model/profile choice between Manus 1.5 and Manus 1.5 Light, plus backend routing when no chat mode is specified. Production integrations use webhooks rather than constant polling, and the Slack bot demo maps Slack thread IDs to Manus task IDs in a key-value store so follow-up messages continue the same agent session. Generated outputs return file metadata for downstream processing, while reports or Slack responses require conversion into the target channel's formatting model.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  developer-application["Developer application"] --> manus-api["Manus API"]
  manus-api --> task-lifecycle-service["Task lifecycle service"]
  manus-api --> files-api["Files API"]
  manus-api --> connector-registry["Connector registry"]
  manus-api --> webhook-dispatcher["Webhook dispatcher"]
  manus-api --> model-router["Model router"]
  model-router --> manus-agent-runtime["Manus agent runtime"]
  task-lifecycle-service --> manus-agent-runtime
  manus-agent-runtime --> sandbox-environment["Sandbox environment"]
  manus-agent-runtime --> connector-registry
  manus-agent-runtime --> remote-browser-operator["Remote browser operator"]
  manus-agent-runtime --> output-artifact-service["Output artifact service"]
  webhook-dispatcher --> webhook-receiver-server["Webhook receiver server"]
  webhook-receiver-server --> slack-event-gateway["Slack event gateway"]
  webhook-receiver-server --> task-state-kv-store["Task state KV store"]
  task-state-kv-store --> manus-api
  slack-event-gateway --> task-state-kv-store
  slack-event-gateway --> slack-formatting-adapter["Slack formatting adapter"]
  output-artifact-service --> slack-formatting-adapter
  sandbox-environment --> output-artifact-service

Components

Data flow

A developer obtains a Manus API key and starts a task. Manus returns a task ID, task title, and task URL. The client can poll task state during prototyping or register a webhook so Manus sends notifications when tasks are created and completed. Context can be attached as uploaded files, public URLs, base64 images, configured connector UIDs, or custom code/toolkits. For Slack, Slack sends an event webhook to a Modal/FastAPI endpoint; the server parses Slack's bot/user markup, creates or resumes a Manus task, stores Slack thread ID to Manus task ID in a key-value map, and posts progress plus final output back to the same thread using Block Kit, Slack mrkdwn conversion, and correct channel/thread file upload parameters.

Why this architecture

Manus chose a general-agent-first architecture so the same underlying capabilities could power web apps, Slack, mail, browser operation, support bots, Zapier/N8N integrations, and custom APIs. A full sandbox lets developers build production-grade applications rather than frontend-only demos. The task/webhook lifecycle exists because agent work is long-running and interactive. The Slack integration shows how a thin channel adapter can preserve Manus task context while handling Slack-specific event, thread, formatting, and file constraints.

Models used: Manus 1.5 · Manus 1.5 Light · LLM agent model · Whisper · OpenAI embeddings · OCR-capable multimodal model · Text-to-speech model · ElevenLabs

Scale

• Manus says it scaled daily conversations to millions and beyond.
  “as we scaled the number of conversations that we have every day to millions and and beyond”transcript, stated
• Uploaded files are automatically deleted after 48 hours.
  “every file that you upload to the manage API, files API, it's basically going to be deleted automatically in 48 hours.”transcript, stated
• A typical Manus task is described as taking roughly three to five minutes.
  “the typical manage task is going to take roughly three to five minutes”transcript, stated
• Slack integrations need to respond in roughly three seconds to avoid retries.
  “you need to send a response in roughly like 3 seconds. Uh if not, they're going to try again.”transcript, stated
• The workshop demonstrates a 100-character fetch for Rick and Morty API data, illustrating programmable file-context ingestion.
  “we can grab the first maybe 100 characters from Rick and Morty and let's get mad to figure out like what exactly Rick and Morty how their characters have been.”transcript, stated
• The conference-site demo references scraping 70+ talks and normalizing event times.
  “There should be like 70 plus talks. I think the time is on like EST.”transcript, stated
• The browser automation Q&A cites spinning up six Selenium instances for a government booking workflow.
  “I wrote like a Python script that spun up like six selenium instances and it scraped like the entire government website”transcript, stated
• The French-learning demo stores a user profile with age and learning weaknesses from ongoing interactions.
  “It seems that you're 28 working at manners where you build agents”transcript, stated

Alternatives considered (roads not taken)

Build vertical products for each channel

Rejected because: Manus argues that a general AI agent first can power many interfaces and applications without rebuilding the core capability each time.

Frontend-only execution environments

Rejected because: The workshop positions full Docker sandboxes as necessary for packages, queues, services, webhooks, Stripe, Redis, and complex production integrations.

Polling every long-running task

Rejected because: Polling works for prototypes, but large numbers of multi-minute tasks make worker-heavy polling inefficient compared with webhooks.

Use the full Manus 1.5 model for every API task

Rejected because: The workshop recommends Manus 1.5 for complex tasks and Manus 1.5 Light when speed and simple queries matter more.

Force all context through file uploads

Rejected because: Manus supports multiple context channels: uploaded files for sensitive/large data, public URL attachments, and base64 images/screenshots.

Expose local browser control through the API immediately

Rejected because: The speaker says browser control requires choosing and approving the browser, so permission systems need to be improved before shipping.

Maintain hidden channel-specific pricing differences

Rejected because: The API is priced the same as equivalent web-app chat usage so developers can choose the interface that fits the user rather than optimizing around channel markup.

Patterns

tool-use-orchestration (agent-runtime)

The Manus agent can run code, call connectors, scrape websites, control browsers, process files, and produce artifacts through a sandboxed general-agent runtime exposed by the API.

model-routing (model-selection)

The API can route simple queries to basic chat or full Manus agent behavior, and lets developers choose Manus 1.5 or 1.5 Light by complexity and speed needs.

agent-memory (state)

The API preserves multi-turn state by reusing task IDs, and the Slack bot maps thread IDs to task IDs so future Slack messages continue the same Manus session.

privacy-preserving-processing (data-handling)

Uploaded files have automatic deletion, manual deletion, and session cleanup paths; the Q&A also states Manus cannot read chats except when users share an issue for support.

rag-retrieval (retrieval)

The event demo uses scraping, JSON, embeddings, Chroma, and semantic search to create a personalized conference-event search and recommendation app.

structured-output-contract (integration)

Manus returns task metadata, file URLs, filenames, MIME types, Slack blocks, markdown outputs, and structured artifacts that integration code can pass to downstream systems.

latency-optimization (operations)

The Slack and webhook paths acknowledge quickly, keep servers warm, and avoid many polling workers for long-running Manus tasks.

Tradeoffs

Expose a general agent through many product surfaces

Chose General AI agent first, then web/API/Slack/browser/email/Microsoft surfaces over Separate verticalized products with different core capabilities.

Rationale: A general-agent core lets Manus reuse the same sandbox, connectors, browser, files, and model capabilities across channels while meeting users where they already work.

Execution environment depth

Chose Full Docker-backed sandbox per chat/task over Limited frontend-only or narrow execution environment.

Rationale: The sandbox lets agents install packages, run Redis queues, handle webhooks, build full apps, and integrate with external services.

Task completion notification

Chose Webhooks for production-scale task completion over Polling every task forever.

Rationale: Polling is simple when prototyping, but webhooks reduce held API loops, worker waste, and cost for many concurrent long-running tasks.

Task model selection

Chose Manus 1.5 for complex tasks and Manus 1.5 Light for faster simple tasks over One model profile for all workloads.

Rationale: Different use cases need different capability/latency tradeoffs, so the workshop selects 1.5 Light for simple fast workshop tasks.

Context ingestion modes

Chose Files API, public URL attachments, and base64 images over A single upload path for every input.

Rationale: Sensitive/large files need temporary upload and deletion controls, public documents can be fetched by URL, and screenshots/generated images can be sent as base64.

Slack state management

Chose Map Slack thread timestamps to Manus task IDs in a KV store over Starting a new agent task for every Slack message.

Rationale: Thread-to-task mapping keeps multi-turn context in the same Manus session while matching Slack's thread model.

Browser automation permissions

Chose Improve permission system before exposing local browser through API over Let API calls freely open user browser tabs.

Rationale: Authenticated browser access is powerful but risky; users must approve which browser is controlled.

Gotchas (what bit them)

Long-running agent tasks do not fit simple synchronous integration.

Root cause: Typical Manus tasks can take minutes and may enter pending state for user clarification or follow-up input.

Resolution: Use task IDs, task states, polling for prototypes, and webhooks for production-scale completion notifications.

Polling becomes inefficient at scale.

Root cause: Many concurrent multi-minute tasks would require clients to hold API loops and keep workers spinning.

Resolution: Register webhooks so the application receives task-created and task-completed notifications.

Slack retries if the bot does not respond quickly.

Root cause: Slack expects event acknowledgement within about three seconds, while LLM work takes much longer.

Resolution: Acknowledge the Slack event immediately, keep the server warm, and run Manus work asynchronously.

Slack thread and file handling have channel-specific traps.

Root cause: Files uploaded to Slack can land in the channel instead of the thread, and raw Slack messages contain user IDs rather than clean user text.

Resolution: Pass both channel and thread parameters for files, parse Slack markup/user IDs, and map thread timestamps to Manus task IDs.

Markdown from an agent does not render correctly in Slack without adaptation.

Root cause: Slack has its own markdown and Block Kit rules, so standard markdown tables, text, images, and files need conversion.

Resolution: Convert Manus markdown into Slack-compatible mrkdwn and blocks, then attach generated files in the correct thread.

A full Docker sandbox creates power and abuse risk.

Root cause: The platform allows custom packages, services, Redis queues, Stripe/webhooks, and broad integration capability.

Resolution: Treat the sandbox as a production-grade execution environment with guardrails and future scaling features such as autoscaling and warm deployments.

Local browser control cannot be exposed casually through the API.

Root cause: Using a user's browser requires selecting and approving a browser rather than randomly opening tabs.

Resolution: Hold the capability behind permission-system improvements before general API exposure.

Persistent cross-conversation memory is not yet available through the API.

Root cause: The current system requires explicit context rather than automatically drawing memory from prior conversations.

Resolution: For now, developers must provide context explicitly; Manus is actively investigating memory.

Evaluation

The workshop demonstrates evaluation mostly through live, end-to-end smoke tests: checking API keys, creating a simple 2+2 task, inspecting task state, uploading files, validating public URLs and base64 screenshots, registering and receiving webhooks, verifying Slack endpoint challenges, posting messages and files back to Slack, exercising multi-turn task continuation, and running an invoice-processing workflow against Notion policy context. It also recommends using the web app as a sandbox to nail down context and request patterns before moving to the API.

Metrics: API key works and files list returns successfully. · Task creation returns task ID/title/URL. · Task states transition through running, pending, completed, or error. · Webhook registration sends task-created/task-completed payloads. · Slack event subscription challenge verifies. · Slack bot responds within roughly 3 seconds. · Thread ID maps to the same Manus task for follow-up messages. · Generated artifacts return URL, filename, and MIME type. · OCR/invoice workflow extracts receipt details and updates the correct Notion page.

Outcomes

• The platform exposes the same underlying agent capability through web, Slack, API, browser, email, Microsoft 365, Zapier, N8N, and internal support-bot integrations.
• The API supports asynchronous task creation, task IDs, pending/resume flows, polling, webhooks, and multi-turn continuation.
• The Files API supports temporary sensitive uploads, public URLs, base64 screenshots, multimodal parsing, OCR, and generated artifact metadata.
• The Slack demo implements mention handling, challenge response, rich Block Kit progress messages, thread-to-task state, markdown conversion, and thread-correct file uploads.
• The invoice demo uses Slack images, OCR, Notion policies, and Notion page updates to automate an expense-review workflow.
• The browser/web-app demos show the agent scraping events, generating JSON, embedding with OpenAI, using Chroma, adding calendar links, and building an interactive site.

The transcript states Manus scaled daily conversations to millions and beyond. It gives operational markers including 48-hour automatic file deletion, 3-5 minute typical tasks, 3-second Slack response windows, 70+ scraped conference talks, six Selenium instances for a booking workflow, and planned autoscaling/warm deployments.