AI-Powered Account Planning System for Sales Process Optimization

aws · Tech · 2025 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Sales account-plan quality evaluation

AWS needed to evaluate and improve account plans consistently as global sales scale made manual quality review slow, subjective, and siloed.

Constraints:

Architecture

Account Plan Pulse is an offline evaluation pipeline: CRM plans are extracted into daily S3 snapshots, preprocessed and validated, evaluated by Bedrock with structured outputs, and surfaced as readiness indices and dashboards.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  crm-scheduled-extraction["CRM scheduled extraction"]
  daily-s3-snapshots["Daily S3 snapshots"]
  etl-preprocessing["ETL preprocessing"]
  bedrock-evaluation-jobs["Bedrock evaluation jobs"]
  structured-output-validation["Structured output validation"]
  readiness-index["Readiness Index"]
  dashboards["Dashboards"]
  crm-scheduled-extraction --> daily-s3-snapshots
  daily-s3-snapshots --> etl-preprocessing
  etl-preprocessing --> bedrock-evaluation-jobs
  structured-output-validation --> bedrock-evaluation-jobs
  bedrock-evaluation-jobs --> readiness-index
  readiness-index --> dashboards
  bedrock-evaluation-jobs --> dashboards
  dashboards --> crm-scheduled-extraction
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
CRM scheduled extraction Pulls account-plan narrative content on a schedule. Daily S3 snapshots
Daily S3 snapshots Stores updated account-plan documents for batch processing. ETL preprocessing
ETL preprocessing Validates and transforms account-plan inputs into Parquet datasets. Bedrock evaluation jobs
Bedrock evaluation jobs Score account plans against structured business questions and summarize gaps. Readiness Index, Dashboards
Structured output validation Constrains model output and checks consistency despite LLM nondeterminism. Bedrock evaluation jobs
Readiness Index Standardizes account-plan quality categories and improvement recommendations. Dashboards
Dashboards Surface results and feedback loops to sales leaders and teams. CRM scheduled extraction

Data flow

Account-plan content is pulled asynchronously from CRM, processed as daily snapshots, validated, and evaluated through structured Bedrock jobs that produce readiness scores and summaries.

“Account plan narrative content is pulled from our CRM system on a scheduled basis through an asynchronous batch processing pipeline.”article, stated
“For this implementation, we use structured output prompting with [schema constraints]”article, stated

Why this architecture

The architecture adds validation and scheduled snapshots because LLM evaluations can vary and account-plan content changes throughout the year.

“First, the non-deterministic nature of LLMs meant identical inputs could produce varying outputs, potentially compromising evaluation consistency.”article, stated
“Second, account plans naturally evolve throughout the year with customer relationships, making static evaluation methods insufficient.”article, stated

Models used: Amazon Bedrock foundation models

Scale

Alternatives considered (roads not taken)

Evaluate and summarize in one coupled job for every document every time

Rejected because: The system runs independent batch workloads and only processes updated plans to reduce overhead and cost.

“We implement these capabilities through asynchronous batch processing, where evaluation and summarization workloads operate independently.”article, implied
“To optimize processing efficiency, the system compares document timestamps to process only recently modified plans, significantly reducing computational overhead and costs.”article, implied

Static criteria for account plans throughout the year

Rejected because: Account plans evolve with customer relationships, so evaluation criteria need flexibility over time.

“Second, account plans naturally evolve throughout the year with customer relationships, making static evaluation methods insufficient.”article, stated

One team-wide definition of plan quality

Rejected because: Different AWS teams prioritize different plan aspects based on industry and business needs.

“Third, different AWS teams prioritize different aspects of account plans based on specific customer industry and business needs, requiring flexible evaluation criteria.”article, stated

Patterns

batch-offline-processing (workflow)

Pulse processes account-plan content through scheduled asynchronous batch jobs rather than interactive generation.

“Account plan narrative content is pulled from our CRM system on a scheduled basis through an asynchronous batch processing pipeline.”article, stated

structured-output-contract (reliability)

The evaluation process uses tailored prompts and schema constraints so outputs integrate consistently with reporting tools.

“For this implementation, we use structured output prompting with [schema constraints]”article, stated

eval-driven-development (evaluation)

The system itself is an automated evaluation pipeline for account plans, producing a readiness index and recommendations.

“Pulse evaluates plans against 10 business-critical categories, creating a standardized Account Plan Readiness Index. This automated evaluation identifies improvement areas with specific improvement recommendations.”article, stated

cost-aware-routing (operations)

The preprocessing pipeline avoids redundant model calls by only processing recently modified plans.

“To optimize processing efficiency, the system compares document timestamps to process only recently modified plans, significantly reducing computational overhead and costs.”article, stated

Tradeoffs

How often to process account-plan documents.

Chose daily snapshots with only updated plans included over rerunning every plan on every batch.

Rationale: The timestamp-based approach keeps results current while reducing repeated LLM work on unchanged content.

“they are processed in daily snapshots, with only updated plans included in each run.”article, stated

Whether evaluation and summarization should share one model job.

Chose independent asynchronous workloads over one combined sequential workload.

Rationale: Separate batch processing lets scoring and knowledge synthesis operate independently.

“We implement these capabilities through asynchronous batch processing, where evaluation and summarization workloads operate independently.”article, stated

Gotchas (what bit them)

LLM evaluations can vary for identical inputs.

Root cause: The model is non-deterministic, which can compromise consistency in account-plan scoring.

Resolution: Add a production engineering framework and statistical checks for reliable AI evaluations.

“First, the non-deterministic nature of LLMs meant identical inputs could produce varying outputs, potentially compromising evaluation consistency.”article, stated

Static evaluation criteria do not fit account plans that evolve during the year.

Root cause: Customer relationships and plans change, so a fixed one-time assessment becomes stale.

Resolution: Run scheduled refreshes on current snapshots and process updated plans only.

“Second, account plans naturally evolve throughout the year with customer relationships, making static evaluation methods insufficient.”article, stated

Different AWS teams value different aspects of account plans.

Root cause: Industry and business-context differences make one rigid scoring rubric insufficient.

Resolution: Support flexible evaluation criteria while keeping output structure consistent.

“Third, different AWS teams prioritize different aspects of account plans based on specific customer industry and business needs, requiring flexible evaluation criteria.”article, stated

Evaluation

Pulse evaluates plans against 10 business-critical categories and 27 tailored questions, then tracks plan-quality and process-time outcomes.

Metrics: 37% improvement in plan quality year-over-year · 52% reduction in time to complete, review, and approve plans · 27 specific questions per account

“The evaluation process runs each account through 27 specific questions with tailored control prompts”article, stated

Outcomes

AWS post about AWS internal sales tooling; outcome metrics are self-reported.

“Implementing Pulse delivered a 37% improvement in plan quality year-over-year, while decreasing the overall time to complete, review, and approve plans by 52%.”article, stated