AI-Powered Shift-Left Testing Platform with Multiple LLM Agents

qyrusai · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Software quality assurance and shift-left testing

QyrusAI aimed to move testing earlier across the SDLC so teams could generate requirements tests, design tests, mock APIs, synthetic data, exploratory tests, and maintenance fixes before late-cycle quality bottlenecks caused delay and defect leakage.

Constraints:

• traditional testing late in development increased delay, cost, and quality risk
• different SDLC phases required different model capabilities including text, vision, embeddings, and structured outputs
• function calling and JSON mode needed consistent behavior across heterogeneous Bedrock models
• frontend teams needed API virtualization before backend completion
• reported impact metrics came from internal testing and select customer pilots, not independent benchmarks

Architecture

QyrusAI built a suite of specialized testing agents on AWS and Amazon Bedrock. TestGenerator uses Llama, Claude, Cohere embeddings, Pinecone, and a ReAct flow to generate and judge tests from requirements. VisionNova and UXtract turn UI/UX designs and Figma prototype graphs into test cases and step-by-step flows. API Builder uses Bedrock models plus ECS, ALB, S3, EFS, and Lambda to virtualize APIs for early frontend testing. Echo, Rover, TestPilot, and Healer extend the suite for synthetic data, exploratory/objective testing, and locator repair. A custom qai package standardizes Bedrock access, model families, function calling, JSON mode, streaming, and updates across agents.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  qai-bedrock-interface["qai-bedrock-interface"] --> testgenerator-agent["testgenerator-agent"]
  qai-bedrock-interface --> visionnova-agent["visionnova-agent"]
  qai-bedrock-interface --> uxtract-agent["uxtract-agent"]
  qai-bedrock-interface --> api-builder-agent["api-builder-agent"]
  qai-bedrock-interface --> echo-rover-testpilot-healer["echo-rover-testpilot-healer"]
  testgenerator-agent --> pinecone-requirements-store["pinecone-requirements-store"]
  visionnova-agent --> s3-test-assets["s3-test-assets"]
  uxtract-agent --> s3-test-assets
  api-builder-agent --> s3-test-assets
  api-builder-agent --> efs-virtual-api-store["efs-virtual-api-store"]
  api-builder-agent --> lambda-api-compute["lambda-api-compute"]
  ecs-agent-runtime["ecs-agent-runtime"] --> testgenerator-agent
  ecs-agent-runtime --> visionnova-agent
  ecs-agent-runtime --> uxtract-agent
  ecs-agent-runtime --> api-builder-agent

Components

Data flow

A user uploads requirements, designs, prototypes, API specs, or an application target. The relevant agent runs as an ECS task behind an Application Load Balancer, stores inputs in S3, and calls Bedrock models through the shared qai package. Requirements are embedded with Cohere and stored in Pinecone for retrieval; TestGenerator creates initial tests with Llama and asks Claude to judge coverage and rank relevance. Design and Figma agents use Claude vision/language models to produce UI flow tests. API Builder interprets API specs and serves virtualized mock responses via EFS/Lambda-backed microservices. Downstream agents explore, validate objectives, heal locator failures, and report shift-left test artifacts.

Why this architecture

QyrusAI uses multiple specialized agents because shift-left testing spans requirements, visual design, prototype flows, API virtualization, exploratory testing, and maintenance. Bedrock gives access to text, vision, and embedding models without managing model infrastructure, while the qai package shields each agent from model-family differences in function calling and JSON mode. ECS/ALB provides scalable service hosting, and the RAG/Pinecone layer lets requirements retrieval guide test generation.

Models used: Amazon Bedrock · Meta Llama 70B · Meta Llama 3.1 · Anthropic Claude 3.5 Sonnet · Anthropic Claude 3 Opus · Cohere English Embed · Mistral 8x7B Instruct · Pinecone · Amazon ECS · Application Load Balancer · Amazon S3 · Amazon EFS · AWS Lambda

Scale

• QyrusAI is positioned across the entire SDLC.
  “enhances the software testing process across the entire software development lifecycle (SDLC).”article, stated
• The suite includes named agents for requirements, design, Figma flows, APIs, synthetic data, exploratory testing, objective testing, and healing.
  “TestGenerator generates initial test cases based on requirements using a suite of advanced models”article, stated

  “QyrusAI offers a range of additional agents that further enhance the testing process”article, stated
• Early adopter data claimed 80% reduction in defect leakage.
  “80% reduction in defect leakage – Finding and fixing defects earlier results in fewer bugs reaching production”article, stated
• Early adopter data claimed 20% reduction in UAT effort.
  “20% reduction in UAT effort – Comprehensive testing early on means a more stable product reaching the user acceptance testing (UAT) phase”article, stated
• Early adopter data claimed 36% faster time to market.
  “36% faster time to market – Early defect detection, reduced rework, and more efficient testing leads to faster delivery”article, stated

Alternatives considered (roads not taken)

traditional late-cycle testing

Rejected because: Late testing can delay releases, raise costs, and compromise quality.

one generic model interface per agent

Rejected because: Different model providers and families have different APIs, prompt formats, function calling, and JSON-mode behavior.

waiting for backend implementation before frontend testing

Rejected because: API Builder virtualizes APIs so frontend testing can begin before the backend is complete.

Patterns

multi-agent-orchestration (agent-architecture)

QyrusAI splits shift-left testing work across specialized agents for requirements, UI design, Figma flows, API mocks, test data, exploration, objectives, and healing.

rag-retrieval (knowledge-grounding)

TestGenerator embeds requirements documents and retrieves them from Pinecone as part of a ReAct generation loop.

llm-as-judge (quality)

Claude evaluates generated test scenarios for comprehensiveness, accuracy, missing cases, failure points, and relevance ranking.

structured-output-contract (reliability)

The custom qai layer normalizes function calling and JSON mode across Bedrock model families.

synthetic-data-generation (data)

Echo generates synthetic test data using Claude, Mistral, and Llama to improve coverage without real data dependencies.

Tradeoffs

Use a shared Bedrock abstraction package.

Chose centralized qai model access over duplicating Bedrock/model-family handling in every agent.

Rationale: Centralization reduced duplication, improved maintainability, and let new Bedrock models/features be integrated once.

Use specialized models per test phase.

Chose LLMs, VLMs, embeddings, and judges matched to requirements, design, API, and maintenance tasks over one model for all testing capabilities.

Rationale: Different phases require natural-language reasoning, visual understanding, retrieval, judging, and structured output.

Virtualize APIs before backend completion.

Chose AI-generated mock responses over waiting for full backend implementation.

Rationale: Frontend teams can begin testing UI behavior earlier in the SDLC.

Gotchas (what bit them)

Model heterogeneity can leak into every agent.

Root cause: LLMs, VLMs, embeddings, function calling, and JSON mode differ across Bedrock model providers.

Resolution: Centralize model access and specialized model-family behavior in the qai package.

Initial generated test scenarios may miss edge cases or high-risk paths.

Root cause: Requirement documents and first-pass model output may not surface all failure modes.

Resolution: Use Claude as an evaluator to highlight missing scenarios, potential failure points, and relevance priority.

Visual prototypes require flow-level understanding, not only static screen reading.

Root cause: Figma prototypes encode transitions between screens and UI elements that test cases must validate.

Resolution: UXtract analyzes prototype graphs and uses layered Claude models to produce step-by-step tests.

Evaluation

The article reports QyrusAI's internal and select-customer pilot metrics, plus agent-level quality checks such as Claude judging generated scenarios for coverage and relevance. Because the source is vendor co-authored, outcome claims should be treated as promotional.

Metrics: 80% reduction in defect leakage · 20% reduction in UAT effort · 36% faster time to market · Claude judging coverage, accuracy, missing scenarios, failure points, and relevance

Outcomes

• QyrusAI claims fewer defects reaching production through earlier testing
• QyrusAI claims lower UAT effort and faster delivery from early defect detection
• the platform covers requirements, design, pre-implementation, implementation, testing, and maintenance phases
• the technical foundation provides reusable model access and structured output across multiple testing agents

The reported business metrics are 80% defect-leakage reduction, 20% UAT-effort reduction, and 36% faster time to market, all from QyrusAI internal/pilot data.