Production Skills Framework for Agentic LLM Workflows

workos · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Reusable skills for production agentic LLM workflows

WorkOS needed a way to make LLM workflows repeatable across teams and tools instead of reloading context, conventions, and domain knowledge from scratch in every conversation.

Constraints:

• Every LLM conversation starts from zero context unless project or workflow knowledge is reintroduced.
• Uncodified prompts cause inconsistent outputs and siloed know-how across teammates.
• Repository-level instruction files can bloat context windows with irrelevant information.
• Production workflows need deterministic command outputs and evaluation rather than purely free-form model behavior.

Architecture

WorkOS packages reusable LLM workflows as skills: folders containing skill.md with YAML front matter, routing descriptions, constraints, progressive-disclosure references, and optional scripts/resources. Skills can be project-specific, global, marketplace-distributed, or packaged as .skill files. WorkOS uses them in production through its CLI installation tool, where Claude Agent SDK executes framework-specific AuthKit setup using public WorkOS skills. Skills are evaluated against baselines, iterated from conversation transcripts, and shared across technical and non-technical workflows.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  user-task["User task"] --> skill-router["Skill router"]
  skill-router["Skill router"] --> skill-package["Skill package"]
  skill-package["Skill package"] --> progressive-disclosure-references["Progressive disclosure references"]
  skill-package["Skill package"] --> deterministic-scripts["Deterministic scripts"]
  skill-package["Skill package"] --> llm-agent-runtime["LLM agent runtime"]
  progressive-disclosure-references["Progressive disclosure references"] --> llm-agent-runtime["LLM agent runtime"]
  deterministic-scripts["Deterministic scripts"] --> llm-agent-runtime["LLM agent runtime"]
  llm-agent-runtime["LLM agent runtime"] --> workos-cli-installer["WorkOS CLI installer"]
  workos-cli-installer["WorkOS CLI installer"] --> workos-api-proxy["WorkOS API proxy"]
  workos-cli-installer["WorkOS CLI installer"] --> application-codebase["Application codebase"]
  skill-package["Skill package"] --> skill-repositories["Skill repositories"]
  skill-repositories["Skill repositories"] --> evaluation-harness["Evaluation harness"]
  llm-agent-runtime["LLM agent runtime"] --> evaluation-harness["Evaluation harness"]

Components

Data flow

A user task enters an LLM environment with available skills. The model reads skill descriptions to route the task, loads only the relevant skill and any conditionally referenced files, and follows constraints such as command choices or citation rules. Optional scripts run deterministic commands and interpolate results back into context. In WorkOS CLI, npx workos install detects the application framework, loads framework-specific skills, calls WorkOS APIs through a proxied token, modifies authentication setup, and benefits from the same skill improvements as manual developer workflows.

Why this architecture

Skills turn implicit prompting know-how into versioned, portable context artifacts. Descriptions make routing semantic, progressive disclosure controls context bloat, scripts inject deterministic facts, and evaluation catches skills that no longer help as models improve. WorkOS also uses the same skills in production CLI flows and manual developer workflows, so improvements compound across both interfaces.

Models used: Claude Agent SDK · Claude Code · Claude Desktop · Codex · Cursor · Imagen 3 · video generation APIs

Scale

• The presenters had not written code without AI assistance in six to eight months.
  “The presenters noted they hadn't written code without AI assistance in six to eight months”summary, stated
• The ideation skill uses a 100-point confidence rubric.
  “The ideation skill demonstrated in the workshop uses a 100-point rubric across dimensions like problem clarity, goal definition, success criteria”summary, stated
• The confidence threshold is typically 90-95%.
  “The LLM only proceeds when confidence exceeds a threshold (typically 90-95%), otherwise it asks clarifying questions.”summary, stated
• A video skill returns a video with sound in approximately 30 seconds.
  “Returns a video with sound in approximately 30 seconds”summary, stated
• The video workflow generated all interstitial scenes for a 32-minute film.
  “This same workflow was used to generate all interstitial scenes for a 32-minute film”summary, stated
• The image-generation wrapper reduces generation time to under 7 seconds.
  “reducing generation time to under 7 seconds and enabling consistent style application across a project.”summary, stated
• Organizational governance questions arise when 60 engineers can create skills.
  “With 60 engineers potentially creating skills, questions arise about:”summary, stated

Alternatives considered (roads not taken)

Repeated ad hoc prompting in every session

Rejected because: It wastes time, creates inconsistent outputs, and keeps useful workflows siloed with individual users.

Load all repository context into claude.md or agents.md

Rejected because: Repository memory can bloat context windows with information that is not relevant to the current task.

Ask the LLM to discover command outputs itself every time

Rejected because: Scripts produce deterministic, consistent data and avoid multiple attempts at command formulation.

Keep skills forever once written

Rejected because: Model capabilities evolve, and old skill instructions can become counterproductive.

Patterns

context-engineering (context)

Skills codify reusable instructions, constraints, routing descriptions, and progressive context into portable artifacts.

tool-use-orchestration (architecture)

Skills can invoke deterministic scripts or API chains and feed their outputs back into the model context.

eval-driven-development (quality)

WorkOS evaluates skills against baseline model performance and success thresholds before treating them as production improvements.

human-in-the-loop-gate (workflow)

Confidence scoring forces the LLM to ask clarifying questions rather than proceed when a task is under-specified.

structured-output-contract (interface)

Skills package markdown, front matter, referenced files, scripts, and resources into a predictable folder contract that can be versioned and shared.

Tradeoffs

Use skills rather than plain conversation memory

Chose Portable markdown skill packages over Repeated prompt setup or repository-local instruction files only.

Rationale: Skills are shareable, versionable, routeable, and scoped to the task rather than permanently loaded into context.

Embed deterministic scripts in skills

Chose Script interpolation for facts and command outputs over Letting the LLM formulate every command ad hoc.

Rationale: Scripts reduce token waste and make repeated workflows more consistent.

Use skills in both CLI and manual workflows

Chose One skill repository powering WorkOS CLI and developer usage over Separate implementation knowledge for product automation and human prompting.

Rationale: Improvements to the skill improve both programmatic and manual workflows simultaneously.

Evaluate skills against model baselines

Chose Baseline and rubric-based tests over Assuming every skill improves performance.

Rationale: Some skills become harmful as models improve, so measured improvement is necessary.

Gotchas (what bit them)

Skill libraries can create context bloat.

Root cause: Loading all available skills or repository instructions adds irrelevant context as libraries grow.

Resolution: Use progressive disclosure and scoped skill repositories so only relevant context is loaded.

Skill descriptions determine routing quality.

Root cause: LLMs use description fields as semantic routing metadata, so vague or overlapping descriptions cause wrong skill activation.

Resolution: Treat descriptions as routing rules and govern overlapping skills.

Skills can become stale as models improve.

Root cause: Instructions that once helped may overconstrain newer models with stronger baseline capabilities.

Resolution: Re-evaluate skills after model releases and remove or reduce obsolete instructions.

Organizational skill proliferation creates governance problems.

Root cause: Many engineers can create overlapping skills without clear ownership, review, or canonical repositories.

Resolution: Use scoped public, internal, team, and individual repositories with plugin-style versioning and review.

Evaluation

WorkOS evaluates skills by comparing task performance with and without the skill, scoring multiple runs against rubrics and confidence thresholds, re-running as models evolve, and analyzing local JSONL conversation transcripts after production use.

Metrics: baseline comparison · 80-90% success threshold · 90-95% confidence threshold · multiple test runs · transcript-derived failures · model-evolution regression checks

Outcomes

• WorkOS ships a production CLI installer that uses skills and Claude Agent SDK to automate AuthKit setup.
• Skills support developer workflows, code review, image/video generation, recruiting reports, Slack-to-Linear automation, and repo analysis.
• Skills become organizational knowledge artifacts that can work across Claude, Cursor, Codex, desktop, web, and programmatic interfaces.
• Evaluation caught a Next.js skill that degraded performance after the base model improved.

Concrete scale anchors include 6-8 months of AI-assisted coding, 100-point rubrics, 90-95% confidence thresholds, 80-90% evaluation success thresholds, 30-second video generation, a 32-minute film, under-7-second image generation, and governance questions for 60 engineers.