Scaling Image Generation to 100M New Users in One Week

openai · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: consumer AI image generation infrastructure

OpenAI had to keep ChatGPT Images available during a viral launch that created far more user and image-generation load than planned.

Constraints:

Architecture

OpenAI kept the launch online by prioritizing access over latency, adding rate limits and compute, rewriting the product from synchronous rendering to asynchronous queued generation for overloaded free traffic, isolating shared systems, and optimizing database/file-system bottlenecks while the spike was happening.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["ChatGPT Images frontend"] --> B["Synchronous image renderer"]
  A --> C["Asynchronous image queue"]
  C --> D["Temporal workflows"]
  B --> E["Image tokenization"]
  E --> F["Decoder"]
  D --> F
  F --> G["Safety checks"]
  F --> H["File storage"]
  H --> I["Database infrastructure"]
  I --> J["Authentication and onboarding"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
ChatGPT Images frontend User-facing prompt and image iteration interface. Synchronous image renderer, Asynchronous image queue
Synchronous image renderer Original rendering path that completes image generation during the request. ChatGPT Images frontend, Image tokenization, Decoder, Safety checks
Asynchronous image queue Deferred rendering path for overload periods and free-user traffic. ChatGPT Images frontend, Temporal workflows, Decoder
Temporal workflows Reliable workflow substrate used for asynchronous operations. Asynchronous image queue, Decoder
Image tokenization Converts prompts into image-token grids. Synchronous image renderer, Decoder
Decoder Progressively renders image tokens into finished images. Image tokenization, Safety checks, File storage
Safety checks Integrity and content checks that can abort unsafe rendering. Decoder
File storage Stores generated image assets and hit rate limits under launch load. Decoder, Database infrastructure
Database infrastructure Shared persistence layer that required query optimization and isolation work. File storage, Authentication and onboarding
Authentication and onboarding Signup and account systems stressed by viral user growth. Database infrastructure

Data flow

A user prompt becomes image tokens, the decoder renders multiple passes under integrity and safety checks, and completed images return to the user. Under overload, free-user requests can be routed into an asynchronous queue so generation occurs when systems have spare cycles, while surrounding storage, databases, authentication, and isolated API systems absorb or avoid the launch spike.

3 supporting quotes
“1. Image tokens convert the description into a grid of discrete image tokens. These tokens natively encode image content.”article, stated
“2. Decoder progressively renders image tokens into an image.”article, stated
“These requests would get queued up and once our system had extra cycles to spare, we got to generating those images.”article, stated

Why this architecture

The architecture changed to asynchronous deferral because the synchronous design consumed resources until completion and could not exploit non-peak capacity. The operational strategy deliberately sacrificed latency before availability.

3 supporting quotes
“The ChatGPT team designed Images to be a synchronous product: when an image starts rendering, it needs to finish in a synchronous way.”article, stated
“The problem with this setup is that it can’t handle peak load by taking advantage of excess capacity at non-peak times.”article, stated
“A rule of thumb the ChatGPT engineering team uses is to intentionally prioritize access over latency.”article, stated

Models used: GPT-4o image generation model

Scale

Alternatives considered (roads not taken)

Keep image generation purely synchronous

Rejected because: The synchronous path could not defer work and kept consuming GPU and memory if rendering continued.

“If the process is interrupted, there is no way to restart it, and while an image is rendering, it continues using GPU and memory resources.”article, stated

Preserve latency for all users during overload

Rejected because: OpenAI chose delayed generation for some traffic so image generation stayed available.

“This meant that we traded off latency (time to generate free images) for availability (being able to generate images)”article, stated

Let ChatGPT and API systems share launch blast radius

Rejected because: The spike showed shared components could affect independent systems, accelerating isolation work.

“Seeing the sudden surge and impact on otherwise independent systems made the team speed up their isolation work.”article, stated

Patterns

durable-execution (workflow)

OpenAI introduced asynchronous workflows and queues so long image generations could be deferred under peak load.

“> Over a few days and nights, we built an entirely asynchronous Images product. It was a bunch of engineers rolling their sleeves up – and we kicked off a parallel workstream to get this new implementation ready.”article, stated

fallback-degradation (reliability)

The launch strategy degraded latency for some image requests rather than losing availability.

“At peak load, latency did regress, but the team prioritized keeping the service accessible. They kept the site responsive by applying rate limits and increasing compute allocations to stabilize performance.”article, stated

canary-rollout (release)

The initial release plan began with paying subscribers before expanding to free users, although demand still exceeded the staged rollout.

“The initial plan was to release to paying subscribers first, then extend to free users on the same day. This gradual rollout strategy is a common LLMOps pattern, but even with paid users only, demand exceeded expectations so significantly that the free user rollout had to be delayed.”summary, stated

latency-optimization (production-operations)

Engineers optimized database queries and added capacity to stabilize performance under the image-generation spike.

“One of our bottlenecks was our database, so our team started to look for database queries that were taking too many resources. Looking closer, it was clear that some of these were doing unnecessary things.”article, stated

Tradeoffs

Overload behavior

Chose availability with queued asynchronous generation over low latency for every image request.

Rationale: Deferring free-user work kept the product usable during demand spikes.

“This meant that we traded off latency (time to generate free images) for availability (being able to generate images)”article, stated

Runtime architecture during launch

Chose rewrite the image product while keeping the site up over waiting for a calmer migration window.

Rationale: Demand was already beyond expectations, so architecture had to change during live traffic.

“we kicked off a parallel workstream to get this new implementation ready. All the while, a bunch of us worked on keeping the site up”article, stated

Isolation work

Chose speed up decoupling of non-ChatGPT systems over continuing with partially shared compute and database components.

Rationale: The image spike demonstrated that shared systems could transmit load to otherwise independent products.

“They decoupled non-ChatGPT systems from the ChatGPT infrastructure, and most OpenAI API endpoints stayed stable”article, stated

Gotchas (what bit them)

The synchronous renderer could not shed peak load.

Root cause: Rendering had to finish synchronously and kept using GPU and memory while in progress.

Resolution: OpenAI built an asynchronous image product and routed overloaded free-user traffic into deferred queues.

“The problem with this setup is that it can’t handle peak load by taking advantage of excess capacity at non-peak times.”article, stated

The viral feature stressed shared infrastructure outside image generation.

Root cause: Compute clusters and a database instance were still shared with other OpenAI systems.

Resolution: The team accelerated isolation of non-ChatGPT endpoints from ChatGPT infrastructure.

“there were still some shared components for which there was a plan to isolate, which hadn’t happened yet; such as a couple of compute clusters”article, stated

Unexpected database bottlenecks appeared during the launch.

Root cause: Rapid growth exposed inefficient queries and overloaded database infrastructure.

Resolution: A spontaneous workstream optimized code paths and added new filesystem/database capacity.

“Looking closer, it was clear that some of these were doing unnecessary things.”article, stated

Evaluation

OpenAI evaluated the launch operationally through availability, latency regression, rate limits, compute allocation, database load, and whether other systems remained stable.

Metrics: new users · images generated · one-hour signup spike · availability · latency · rate limits · database load

“Despite unexpectedly high traffic, ChatGPT avoided hard outages like the site going down, and maintained availability for existing users.”article, stated
“They kept the site responsive by applying rate limits and increasing compute allocations to stabilize performance.”article, stated

Outcomes

The article emphasizes operational survival and architectural changes during an extreme consumer launch rather than a controlled benchmark.

“This load was far higher than OpenAI had expected and prepared for, but the launch passed with no major outages.”article, stated
“> Over a few days and nights, we built an entirely asynchronous Images product. It was a bunch of engineers rolling their sleeves up – and we kicked off a parallel workstream to get this new implementation ready.”article, stated