Self-Hosting DeepSeek-R1 Models on AWS: A Cost-Benefit Analysis

liftoff · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: self-hosted LLM inference for internal coding assistance

Liftoff evaluated whether self-hosted DeepSeek-R1 models on AWS could replace paid AI coding assistants for internal engineers while maintaining response quality, stability, and cost efficiency.

Constraints:

• The deployment had to support fewer than 100 internal users but still be stable on GPU-backed EC2.
• Model sizes ranged from small efficient variants to a 16B model with higher quality and heavier GPU demands.
• Longer contexts and concurrency exposed memory, latency, and crash risks.
• The economic comparison had to include DevOps and maintenance overhead, not just GPU instance price.

Architecture

Liftoff deployed DeepSeek-R1 model variants on AWS EC2 G5 GPU instances using Docker, NVIDIA Container Toolkit, Ollama, and OpenWeb UI. The team benchmarked 1.5B, 7B, 8B, and 16B models for throughput, quality, stability, cost, context-length behavior, and concurrency before concluding that SaaS LLM offerings still delivered better value for their scale.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Internal engineers"] --> B["OpenWeb UI"]
  B --> C["Ollama runtime"]
  C --> F["DeepSeek-R1 model variants"]
  D["Docker and NVIDIA Container Toolkit"] --> C
  D --> E["AWS EC2 G5 GPU instances"]
  E --> F
  F --> G["Benchmarking harness"]

Components

Data flow

Engineers access OpenWeb UI as a ChatGPT-like frontend; Ollama loads DeepSeek-R1 variants inside Docker containers with NVIDIA GPU access on EC2 G5 instances; benchmarking exercises different model sizes and contexts; tuning changes such as quantization, batch size, sequence length, and context windows attempt to stabilize throughput; the results feed a cost-benefit decision against paid SaaS coding assistants.

Why this architecture

Liftoff chose a lean EC2/Ollama/OpenWeb UI stack because it was fast to iterate and gave control over privacy, model size, and GPU tuning; benchmarking then showed that the operational costs and instability outweighed those benefits at their startup scale.

Models used: DeepSeek-R1 1.5B · DeepSeek-R1 7B · DeepSeek-R1 8B · DeepSeek-R1 16B

Scale

• The test supported fewer than 100 internal users.
  “We were just trying to support fewer than 100 internal users”article, stated
• The experiment benchmarked four model sizes from 1.5B to 16B.
  “Offers multiple model sizes (1.5B, 7B, 8B, 16B) for flexibility.”article, stated
• Long-context 8B and 16B runs dropped below 30 tokens per second.
  “we observed token generation speeds dropping below 30 tokens/sec.”article, stated
• The tested EC2 options included 16 GB GPU instances.
  “g5g.xlarge (4 vCPU + 8 GB + 16 GB GPU)”article, stated

Alternatives considered (roads not taken)

Use smaller 1.5B models

Rejected because: They were efficient but did not meet production-grade response quality.

Use 7B or 8B models

Rejected because: They improved response quality but still did not fully replace commercial LLMs.

Use the 16B model for higher capability

Rejected because: It introduced slower throughput, high operational cost, low concurrency, and memory instability.

Self-host instead of using SaaS LLM offerings

Rejected because: For small-to-medium startup scale, SaaS delivered better value after hidden operating costs.

Patterns

eval-driven-development (evaluation)

The deployment decision was driven by benchmarking model sizes against throughput, quality, cost, concurrency, long-context behavior, and stability.

Tradeoffs

Self-host DeepSeek-R1 for control and privacy.

Chose AWS EC2 GPU instances with Ollama/OpenWeb UI over paid SaaS coding assistants.

Rationale: Self-hosting offered control, privacy, and flexibility, but the cost-benefit did not justify it at the tested scale.

Use mid-sized or larger models for quality.

Chose 7B, 8B, and 16B variants during benchmarking over only the efficient 1.5B model.

Rationale: Smaller models were efficient but quality was insufficient; larger models improved quality but increased cost and stability risks.

Gotchas (what bit them)

The 16B model was unstable under load.

Root cause: GPU memory limits required aggressive quantization and small batch sizes, and stability still was not guaranteed.

Resolution: Treat larger-model self-hosting as requiring meticulous GPU memory management and testing.

Long context degraded generation speed and could crash the service.

Root cause: 8B and 16B models strained the GPU-backed EC2 setup as prompt context increased.

Resolution: Tune batch size, sequence length, and context window, but accept that this added unpredictable latency and runtime risk.

Cost analysis changed when hidden operations work was included.

Root cause: Setup, DevOps overhead, and maintenance widened the gap against SaaS LLM offerings.

Resolution: Treat self-hosting as an infrastructure product, not just a cheaper model endpoint.

Evaluation

Liftoff benchmarked DeepSeek-R1 model sizes for tokens/sec, cost, response quality, concurrency, context-length degradation, and crash behavior against the practical alternative of SaaS LLM coding assistants.

Metrics: tokens per second · response quality · operational cost · concurrency · crash/stability behavior · context-length degradation

Outcomes

• Self-hosting was technically possible but not cost-effective for Liftoff's under-100-user internal workload.
• The 16B model crashed under load until aggressive quantization and batch-size reductions were applied.
• SaaS LLM offerings delivered better value for small-to-medium startup scale at the time of the test.

Even under 100 internal users pushed the limits of reliable GPU-backed EC2 deployment for 8B and 16B reasoning models.