AI Agents for ML Experiment Orchestration: Reducing Friction in Machine Learning Workflows

teads · Media & Entertainment · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: ML experimentation orchestration

Teads had a mature experiment orchestrator, but configuring features, hyperparameters, and runs still created friction and human errors.

Constraints:

• cloud GPU cost
• dataset context
• experiment failures
• API integration

Architecture

Teads wrapped its existing Datakinator ML-experiment orchestrator with MCP routes for coding agents, then learned that API exposure alone was insufficient. The second iteration added dataset probing, run-error retrieval, feature-context tools, failed-run archiving/correction/restart loops, and a cost-estimation permission gate. Agents can now configure and launch experiments over Datakinator’s cloud-backed processing/training/evaluation pipeline while cost guardrails prevent unconstrained GPU spending.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  data-scientist-request["data-scientist-request"]
  coding-agent["coding-agent"]
  mcp-routes["mcp-routes"]
  context-tools["context-tools"]
  datakinator-orchestrator["datakinator-orchestrator"]
  cloud-gpu-backbone["cloud-gpu-backbone"]
  cost-guardrail["cost-guardrail"]
  experiment-results["experiment-results"]
  data-scientist-request --> coding-agent
  coding-agent --> mcp-routes
  coding-agent --> context-tools
  mcp-routes --> datakinator-orchestrator
  context-tools --> datakinator-orchestrator
  datakinator-orchestrator --> cloud-gpu-backbone
  cloud-gpu-backbone --> cost-guardrail
  datakinator-orchestrator --> experiment-results

Components

Data flow

A data scientist asks a coding agent to configure or run an experiment. The agent calls MCP routes into Datakinator, probes datasets/features, starts a processing/training/evaluation workflow, retrieves exact run errors if it fails, archives failed runs, corrects the setup, and restarts. Before expensive work, it estimates price and asks permission or forbids the run if it is too costly.

Why this architecture

Teads already had a mature experiment orchestrator, so the AI layer’s job was not to invent ML infrastructure but to remove configuration friction. The first MCP wrapper proved that agents need more context than humans do: wrong dates and nonexistent features showed that raw API routes were underspecified for autonomous use. Context tools plus feedback loops made the agent self-correcting, while cost permissioning handled the predictable GPU-spend spike from lower experimentation friction.

Models used: AI coding agents

Scale

• Datakinator can orchestrate simple through complex experiments, including forward feature selection, grid search, and successive halving.
  “Those experiments can be very simple (one training, one evaluation), a bit more complicated (multiple trainings and evaluations), or quite complex for hyperparameter tuning (forward feature selection, grid search, successive halving…).”article, stated
• The platform can start hundreds of deep-learning training sessions on hundreds of GPUs.
  “It can start up to hundreds of deep learning training sessions using hundreds of GPUs without breaking a sweat.”article, stated
• Before agentification, Datakinator powered more than 3,000 experiments per year.
  “Datakinator a super important tool, used to start 3,000+ experiments per year and enabling substantial R&D breakthroughs”article, stated
• After release, more than 200 experiments were started in 48 hours and cloud costs spiked.
  “In just 48 hours, more than 200 experiments were started. And as you can guess, cloud costs spiked.”article, stated
• Teads reports 5-10% offline metric uplift on multiple models and nearly $1M direct margin gain.
  “We found up to a 5–10% uplift on our offline metrics on multiple models, which translates to a total of nearly a million in direct margin gain”article, stated

Alternatives considered (roads not taken)

Expose existing Datakinator API routes only

Rejected because: Agents lacked human context and made mistakes with dates/features, so additional context and error-retrieval tools were required.

Let agents launch arbitrary cloud experiments without cost approval

Rejected because: Lowered friction immediately spiked cloud costs, so a price-estimation permission gate was added.

Patterns

tool-use-orchestration (mlops)

Agents call Datakinator APIs through MCP to configure and launch ML experiments.

cost-aware-routing (operations)

The team added cost estimation and guardrails after cloud spend spiked from increased experimentation.

Tradeoffs

How much experimentation freedom to give agents.

Chose autonomous experiment launch with guardrails over manual configuration only.

Rationale: The agents increased throughput and ROI but temporarily increased cloud cost.

How to expose experiment-launching power to agents without runaway GPU spend.

Chose Add price estimation and permission/forbid controls before expensive experiments. over Letting the agent freely launch every experiment made easy by the MCP wrapper..

Rationale: The release drove more than 200 experiments in 48 hours and spiked cloud cost, so cost estimation became part of the agent tool layer.

Gotchas (what bit them)

A bare API exposure worked for humans but not for agents.

Root cause: Agents lacked dataset context and made errors such as wrong dates or nonexistent features.

Resolution: Add context-probing and error-retrieval tools so agents can correct failed experiments.

Evaluation

Teads measured launched experiments, offline metric uplift, and business margin.

Metrics: 200+ experiments in 48 hours · 5-10% offline metric uplift · $1M direct margin gains

Outcomes

• The MCP agent interface made Datakinator easier to use and increased experiment throughput.

Medium engineering post; ROI attribution should be treated cautiously.