Deploying Secure AI Agents in Highly Regulated Financial and Gaming Environments

sicoob_/_holland_casino · Finance · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Regulated financial and gaming AI deployment

Sicoob and Holland Casino needed to deploy generative AI and agent systems inside highly regulated environments where cost, security, governance, compliance, and management visibility were as important as model capability.

Constraints:

Sicoob operates at national financial scale under central-bank and data-protection regulation.
GPU inference is expensive and has to scale with demand instead of sitting idle.
Multiple LLMs are needed because one model does not fit all tasks.
Holland Casino operates under gaming oversight focused on player safety, fraud, and money laundering detection.
Management stakeholders need cost, security, and operations answers without using AWS consoles or waiting for ad hoc scripts.

Architecture

The presentation contrasts two regulated architectures. Sicoob runs open-source LLMs such as Llama, Mistral, Phi, DeepSeek, and Granite on Amazon EKS with GPU EC2 instances, Karpenter, KEDA, and vLLM so it can scale inference and manage GPU cost. Holland Casino uses Anthropic models through Amazon Bedrock, Strands Agents, and Bedrock AgentCore to provide management self-service agents for cost, security, and operations reporting, backed by Bedrock guardrails and compliance posture.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  sicoob-users-and-workloads["Sicoob users and workloads"] --> amazon-eks-gpu-inference["Amazon EKS GPU inference"]
  amazon-eks-gpu-inference --> open-source-llm-model-pool["Open-source LLM model pool"]
  amazon-eks-gpu-inference --> karpenter-gpu-provisioning["Karpenter GPU provisioning"]
  amazon-eks-gpu-inference --> keda-autoscaling["KEDA autoscaling"]
  amazon-eks-gpu-inference --> vllm-serving["vLLM serving"]
  holland-casino-stakeholders["Holland Casino stakeholders"] --> strands-agents["Strands Agents"]
  strands-agents --> amazon-bedrock-agentcore["Amazon Bedrock AgentCore"]
  strands-agents --> bedrock-models-and-guardrails["Bedrock models and guardrails"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Sicoob users and workloads`	Financial use cases such as code generation, RPA, investment advisory, and document interaction.	`Amazon EKS GPU inference`
`Amazon EKS GPU inference`	Kubernetes cluster running open-source LLM inference on GPU EC2 instances.	`Open-source LLM model pool`, `Karpenter GPU provisioning`, `KEDA autoscaling`, `vLLM serving`
`Open-source LLM model pool`	Task-specific models including Llama, Mistral, Phi, DeepSeek R1, and Granite.	`Amazon EKS GPU inference`
`Karpenter GPU provisioning`	Scales GPU instances, including spot instances, to reduce idle GPU cost.	`Amazon EKS GPU inference`
`KEDA autoscaling`	Scales deployments so GPU-backed inference capacity responds to workload demand.	`Amazon EKS GPU inference`
`vLLM serving`	Optimized inference serving layer for faster and more compute-efficient model execution.	`Amazon EKS GPU inference`
`Holland Casino stakeholders`	Management, security, and operations users needing self-service AI reports.	`Strands Agents`
`Strands Agents`	Lightweight agents answering cost, security, and operations questions.	`Amazon Bedrock AgentCore`, `Bedrock models and guardrails`
`Amazon Bedrock AgentCore`	Managed hosting/runtime for Holland Casino's agents.	`Strands Agents`
`Bedrock models and guardrails`	Anthropic/Bedrock model access plus managed guardrails and compliance features.	`Strands Agents`

Data flow

Sicoob model requests run through an EKS GPU inference stack where Karpenter provisions GPU instances, KEDA scales deployments, models are staged for faster pod startup, and vLLM improves compute utilization. Holland Casino stakeholder requests go to lightweight Strands agents hosted through Bedrock AgentCore, which call tools or report scripts to answer management questions about cost, security, and operations.

“they are using Karpenter to do this using spotting instances on EKS, on Kubernetes.”transcript, stated

“we give management and stakeholder agents that give them information they need on a self-service basis.”transcript, stated

Why this architecture

Sicoob chose open-source LLMs on EKS because it needed cost-managed GPU infrastructure and the right model for each task. Holland Casino chose Bedrock/Strands/AgentCore because management needed conversational self-service access to operational information without ad hoc reporting scripts or AWS-console expertise.

“We wanted to use the right model to the right task. We really believe that only one model doesn't fit all needs.”transcript, stated

“management is always a need for cost, security and compliance oversight. They will not log into the Amazon console to get the data. It is not the right tool for them, right?”transcript, stated

Models used: Llama · Mistral · Phi 4 · DeepSeek R1 · IBM Granite · Anthropic models in Amazon Bedrock · Strands Agents · Amazon Bedrock AgentCore · vLLM

Scale

Sicoob is present in nearly 2,500 Brazilian cities.
“Sicoob is present in almost 2,500 Brazilian cities.”transcript, stated
Sicoob serves more than 9 million people.
“All this structure serving more than 9 million people in our country.”transcript, stated
Sicoob has more than 300 credit unions and more than 60,000 employees.
“We have more than 300 credit unions integrated with our cooperative system. We have 14 central co-ops and we have 4,700 brands in all over the country with more than 60,000 employees.”transcript, stated
Holland Casino runs 13 physical casinos and one online casino.
“Currently we run 13 casinos in Holland and one online casino.”transcript, stated

Alternatives considered (roads not taken)

Use one LLM for all financial tasks

Rejected because: Sicoob explicitly wanted the right model for each task and ran multiple models side by side.

“Sometimes we are using multiple models side by side in the same use case”transcript, stated

Keep GPU instances always running

Rejected because: GPUs are expensive, so Sicoob used Karpenter, spot instances, and demand-based scaling.

“So we know GPUs are expensive. They are not cheapest infrastructure. We know this.”transcript, stated

Continue writing ad hoc Python scripts for management reports

Rejected because: The scripts created dependency on the internal team and did not scale.

“we created many Python script to provide these reports and this creates an unnecessary dependency and it does not scale.”transcript, stated

Require management stakeholders to use AWS console directly

Rejected because: The presenter says the console is not the right tool for management cost/security/operations questions.

“They will not log into the Amazon console to get the data. It is not the right tool for them, right?”transcript, stated

Patterns

`model-routing` (model-ops)

Sicoob used multiple open-source models and selected models by task rather than forcing a single model across workloads.

“We wanted to use the right model to the right task. We really believe that only one model doesn't fit all needs.”transcript, stated

`cost-aware-routing` (operations)

Karpenter, KEDA, spot instances, and pay-as-you-go GPU scaling controlled inference cost.

“using spotting instances on EKS, on Kubernetes. So instead of just leaving the GPUs over there without utilization, they pay as you go”transcript, stated

`tool-use-orchestration` (agent)

Holland Casino used Strands Agents as tool-using self-service interfaces for operational information.

“these are single agents not yet busy with orchestration, yeah. These agents centers around costs, security and operations and they can have a dialogue with them.”transcript, stated

`safety-policy-enforcement` (governance)

The regulated environments are framed around Bedrock compliance, guardrails, ISO 42001, and gaming oversight.

“We operate under a very strict gaming oversight through the Dutch Gaming Authorities that centers around player safety and detection of fraud and money laundering.”transcript, stated

Tradeoffs

Run open-source models on EKS for Sicoob

Chose Amazon EKS, GPU EC2 instances, Karpenter, KEDA, and vLLM over Only using managed model APIs.

Rationale: The architecture gave Sicoob model choice, cost control, and infrastructure-level tuning for regulated financial workloads.

“It was very important to us run the large models considering the strategy pay as you go and considering the strategy related of open source LLM adoption.”transcript, stated

Use managed Bedrock agent services for Holland Casino

Chose Anthropic models in Bedrock, Strands Agents, and Bedrock AgentCore over Building full custom agent infrastructure.

Rationale: The casino use case needed a simple management-facing agent interface for cost/security/operations rather than custom GPU inference infrastructure.

“About a year ago we started using code generation with the Anthropic models in Bedrock. Then we looked at Bedrock agents. In Q2, Q3, we started looking at Strands Agents”transcript, stated

Expose management data through agents

Chose Self-service conversational agents over Manual ad hoc reports.

Rationale: Ad hoc scripts created an unnecessary dependency on the internal team and did not scale.

“we created many Python script to provide these reports and this creates an unnecessary dependency and it does not scale.”transcript, stated

Gotchas (what bit them)

GPU-backed LLM inference can waste expensive capacity.

Root cause: Leaving GPU instances provisioned without utilization burns money.

Resolution: Use Karpenter and KEDA to provision/scale GPU-backed Kubernetes workloads according to need.

“So instead of just leaving the GPUs over there without utilization, they pay as you go, as they go, as you go as Edson said and use these GPU instances based on their needs.”transcript, stated

Kubernetes GPU inference requires specialized deployment hygiene.

Root cause: GPU drivers, AMIs, model downloads, and pod startup paths are operationally fragile.

Resolution: Use generative-AI/GPU-specific AMIs and pre-stage models for faster EKS deployment.

“you guys should think about using specific AMIs for generative AI workloads, for GPU workloads. So there is, for example, drivers from Nvidia that you don't want to deploy it by your own.”transcript, stated

Management-reporting scripts do not scale as AI oversight demand grows.

Root cause: Stakeholders need repeated cost, security, and compliance answers but cannot or should not use cloud consoles.

Resolution: Wrap reporting tools in self-service agents for management stakeholders.

“management, stakeholders, especially the security department came to us, the internal team, my team, to provide 'em with ad hoc reports.”transcript, stated

Evaluation

The source emphasizes operational and governance metrics: cost visibility, GPU utilization, model-fit per task, regulatory compliance, and self-service stakeholder reporting. It does not provide a model-quality benchmark.

Metrics: GPU cost visibility · GPU autoscaling/utilization · Model selected per task · Management access to cost/security/operations reporting · Compliance posture such as ISO 42001 and regulated-industry oversight

“Both organizations emphasized the importance of security, governance, compliance frameworks (including ISO 42001 for AI), and responsible AI practices”summary, implied

“management is always a need for cost, security and compliance oversight.”transcript, stated

Outcomes

Sicoob deployed open-source LLM inference on Amazon EKS with GPU instances, Karpenter, KEDA, and vLLM.
Sicoob used multiple open-source models for different tasks and cost-managed GPU workloads.
Holland Casino built Bedrock/Strands/AgentCore agents for self-service cost, security, and operations insights.
Both organizations showed regulated AI deployment paths without treating regulation as a blocker.

Sicoob serves more than 9 million people across nearly 2,500 Brazilian cities; Holland Casino operates 13 casinos and one online casino.

3 supporting quotes

“All this structure serving more than 9 million people in our country.”transcript, stated

“Currently we run 13 casinos in Holland and one online casino.”transcript, stated