AI Strategy and LLM Application Development in Swedish Public Sector

swedish_tax_authority · Government · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Public-sector AI strategy and LLM applications

The Swedish Tax Authority needed to scale AI and LLM applications across public-sector domains while operating under sensitive-data, silo, governance, and regulatory constraints.

Constraints:

• public-sector silos are often deliberate for data security and purpose limitation
• sensitive data may require private cloud or on-premise deployment
• AI projects need risk assessment and documentation under regulation
• text and unstructured data dominate many citizen-agency interactions
• AI strategy must bridge technical development, product ownership, and management steering

Architecture

The described operating model combines an AI strategy role, flexible development teams, reusable AI/text-processing capabilities, open-source model/runtime evaluation, on-prem or private-cloud deployment for sensitive data, and risk/governance workflows. The authority’s earlier text categorization, transcription, OCR, risk evaluation, search, and translation capabilities become reusable layers that can be adapted across business silos, while compliance discussions are pushed left into early project ideation rather than left to a pre-deployment review.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  citizen-interaction-channels["citizen-interaction-channels"] --> text-and-data-processing-pipelines["text-and-data-processing-pipelines"]
  text-and-data-processing-pipelines["text-and-data-processing-pipelines"] --> domain-business-silos["domain-business-silos"]
  flexible-development-teams["flexible-development-teams"] --> text-and-data-processing-pipelines["text-and-data-processing-pipelines"]
  flexible-development-teams["flexible-development-teams"] --> domain-business-silos["domain-business-silos"]
  ai-strategy-role["ai-strategy-role"] --> flexible-development-teams["flexible-development-teams"]
  ai-strategy-role["ai-strategy-role"] --> risk-governance-workflow["risk-governance-workflow"]
  ai-strategy-role["ai-strategy-role"] --> open-source-model-stack["open-source-model-stack"]
  open-source-model-stack["open-source-model-stack"] --> private-cloud-runtime["private-cloud-runtime"]
  open-source-model-stack["open-source-model-stack"] --> benchmark-evaluation-layer["benchmark-evaluation-layer"]
  benchmark-evaluation-layer["benchmark-evaluation-layer"] --> risk-governance-workflow["risk-governance-workflow"]
  risk-governance-workflow["risk-governance-workflow"] --> domain-business-silos["domain-business-silos"]

Components

Data flow

Citizen and agency interactions generate large amounts of structured numeric data and unstructured text through websites, personal pages, calls, and email. Development teams build reusable AI/text-processing solutions that can be adapted for different domain silos while respecting purpose-specific data boundaries. Open-source models and benchmarks support model selection, with sensitive workloads packaged for private cloud or on-prem environments. AI project ideas move through risk assessment, documentation, and legal/engineering discussion early enough to shape MVP design.

Why this architecture

The architecture is organizational as much as technical: public-sector data boundaries force silo-aware reuse, sensitive data pushes the stack toward private/on-prem deployments and open-source tooling, and regulation requires compliance to enter the development loop early. The AI strategy role sits outside pure IT so it can bridge product delivery, management, and technical reality.

Models used: open-source large language models · Llama 3.1 · Mixtral 7B · Cohere · GPT-3.5 · OCR · transcription · text categorization · risk evaluation models

Scale

• The digitalization effort spans several decades.
  “it's a ongoing effort that has been that that is several decades long at least”transcript, stated
• AI/text-processing work started around 2018 from the speaker’s perspective.
  “when I uh started working at Swedish tax Authority that was around 2018”transcript, stated
• Public-facing systems may need to support potentially millions of users.
  “robust enough to go into production and service with potentially millions of users”transcript, stated
• Internal systems may serve hundreds of thousands of users.
  “maybe hundreds and thousands of uses if it's inside scatter market”transcript, stated
• A referenced proof-of-concept process took approximately three months for one case.
  “today the proof of concept takes approximately three months to go through a single case”transcript, stated

Alternatives considered (roads not taken)

Treat AI strategy as a purely technical IT role

Rejected because: The role needs to support organizational decision making, prioritization, product delivery, and management steering.

Break public-sector silos to maximize sharing

Rejected because: Some silos exist by design for data security and purpose-specific data use, so reuse must happen through shared development layers rather than unrestricted data sharing.

Use cloud-hosted commercial models for sensitive public-sector data by default

Rejected because: Sensitive data can force private cloud/on-prem operation, which makes open-source packaging and local infrastructure capacity important.

Handle AI Act/GDPR/security review after product development

Rejected because: The discussion argues for shift-left compliance and security by design before or during the first MVP.

Patterns

privacy-preserving-processing (privacy)

Sensitive public-sector data pushes some LLM workloads to private cloud or on-prem deployment instead of external hosted APIs.

eval-driven-development (evaluation)

Benchmarks and evaluation frameworks help public-sector teams choose among open-source and commercial model options.

data-quality-loop (data-foundation)

The strategy treats text, structured numeric data, and reusable data-processing pipelines as the foundation for AI applications.

synthetic-data-generation (evaluation)

The discussion cites synthetic data as a way to create benchmarks where the correct answer is known even when human agreement is imperfect.

Tradeoffs

Favor open-source options in public sector

Chose open-source model and tooling ecosystem over exclusive reliance on commercial closed services.

Rationale: Open source helps with financial constraints, community trust, evaluation assets, and packaging for private/on-prem deployments.

Reuse general text-processing solutions across domains

Chose domain-adaptable reusable capabilities over fully bespoke AI applications for every silo.

Rationale: Search, translation, OCR, transcription, and text categorization can be adapted with slight modification to multiple public-sector domains.

Use shift-left compliance

Chose documentation and risk assessment at the repo/MVP starting point over late compliance review just before deployment.

Rationale: Early security, GDPR, and AI Act artifacts make compliance more efficient and reduce rework.

Gotchas (what bit them)

Public-sector silos can block AI reuse unless the reusable layer respects legal data boundaries.

Root cause: Silos may be intentional for data security and purpose limitation, not just organizational dysfunction.

Resolution: Build flexible development teams and reusable solution components that can adapt across domains without collapsing the silos themselves.

Regulatory review can become waterfall-shaped if lawyers interpret the AI Act before engineers shape a viable design.

Root cause: AI compliance is cross-disciplinary, but legal-first workflows may not understand practical engineering tradeoffs.

Resolution: Shift compliance, security, GDPR, and AI Act documentation left into early project and MVP setup with engineers involved.

Model choice is a small part of production usefulness.

Root cause: The surrounding system, data, UX, fine-tuning, compound-system design, and infrastructure carry most production complexity.

Resolution: Treat LLMs as one component of a larger production system rather than the central architecture.

Evaluation

The case discusses benchmark-driven model evaluation, open-source benchmark ecosystems, broader frameworks for truthfulness and model selection, and synthetic-data benchmarks where ground truth can be controlled despite imperfect human agreement.

Metrics: benchmark performance · truthfulness · risk assessment completion · human expert consistency · synthetic benchmark accuracy · production robustness

Outcomes

• the authority developed a strategy function connecting AI engineering with management and product steering
• text categorization, transcription, OCR, and risk-evaluation pipelines created a foundation for later AI applications
• reusable solutions can be adapted across public-sector business silos
• sensitive-data constraints drove open-source, private-cloud, and on-premise considerations
• the governance discussion moved toward shift-left compliance and risk documentation

This is a strategy/operating-model case rather than one deployed agent. The transcript grounds scale in decades of digitalization, potentially millions of external users, hundreds of thousands of internal users, and public-sector interactions dominated by text channels.