Building a Rust-Based AI Agentic Framework for Multimodal Data Quality Monitoring

zectonal · Tech · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multimodal data quality monitoring with AI agents

Zectonal needed to scale multimodal data inspection beyond coded rules while preserving speed, auditability, security, and deployment flexibility across online and on-premise environments.

Constraints:

The system inspects multimodal datasets for defects, errors, poisoned data, and anomalous conditions
Rules-based analysis became harder to scale as file codecs, data sources, volume, and frequency increased
Python-based agent tools were seen as complex and hard to audit when wrong tools were called
The product needed visibility into agent behavior at scale
The roadmap includes remote online and on-premise agent execution

Architecture

Zectonal built a custom Rust agentic framework as a single compiled binary for autonomous AI function-tool calling. The framework supports OpenAI, Anthropic, and Ollama APIs, can spawn agents and tools on demand, wraps Zectonal's high-speed diagnostic algorithms, records Agent Provenance for auditability, tracks intent/confidence-style metrics for future utility functions, and is intended to support best-of-breed agents across remote and on-premise execution modes.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  data-store[Data Store] --> deep-data-inspection[Deep Data Inspection]
  deep-data-inspection[Deep Data Inspection] --> rust-agentic-framework[Rust Agentic Framework]
  rust-agentic-framework[Rust Agentic Framework] --> tool-calling-layer[Tool Calling Layer]
  tool-calling-layer[Tool Calling Layer] --> ai-created-agents[AI Created Agents]
  ai-created-agents[AI Created Agents] --> agent-provenance[Agent Provenance]
  rust-agentic-framework[Rust Agentic Framework] --> agent-provenance[Agent Provenance]
  rust-agentic-framework[Rust Agentic Framework] --> model-providers[Model Providers]
  rust-agentic-framework[Rust Agentic Framework] --> on-premise-agents[On Premise Agents]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Data Store`	Large stores, pipelines, or lakes containing multimodal files to inspect.	`Deep Data Inspection`
`Deep Data Inspection`	Zectonal diagnostic algorithms for detecting anomalous file and data characteristics.	`Data Store`, `Rust Agentic Framework`
`Rust Agentic Framework`	Single-binary AI agent function-tool calling runtime.	`Deep Data Inspection`, `Tool Calling Layer`, `Agent Provenance`, `Model Providers`, `On Premise Agents`
`Tool Calling Layer`	Function tools and meta-programming path for creating and assigning new tools to agents.	`Rust Agentic Framework`, `AI Created Agents`
`AI Created Agents`	Agents spawned on demand based on immediate needs.	`Tool Calling Layer`, `Agent Provenance`
`Agent Provenance`	Audit trail recording agent behavior, intent, confidence, and tool interactions.	`Rust Agentic Framework`, `AI Created Agents`
`Model Providers`	OpenAI, Anthropic, and Ollama-compatible model/API endpoints.	`Rust Agentic Framework`
`On Premise Agents`	Local agent/model deployments for secure or cost-sensitive use cases.	`Rust Agentic Framework`

Data flow

Multimodal files in data stores, data pipelines, or lakes are inspected by Rust diagnostic algorithms. The Rust agentic framework lets LLM agents decide which function tools to call, records those actions in Agent Provenance, and exposes provider-specific agents through OpenAI, Anthropic, or Ollama APIs. Provenance then feeds prompt, tool-description, and model-tuning improvements.

3 supporting quotes

“We developed our own Rust AI Agentic framework in order to facilitate efficient and auditable agent communications using OpenAI, Anthropic, and Ollama API’s.”article, stated

“AI Research Lab](https://bair.berkeley.edu/) (“BAIR”), we immediately saw the value of having LLM’s decide what function tools to call instead of our pre-defined rules-based approach. While function tool calling is still in its infancy, it has provided us with a unique way to scale our”article, stated

“Although we could not always predict the behavior of our agents, viewing the Agent Provenance provided visibility to understand what motivated agent behavior, and thereby allowed us to develop better agent prompts, better function tool descriptions, and allowed us to fine tune models more”article, stated

Why this architecture

Zectonal chose Rust to keep the agent runtime fast, secure, auditable, and deployable as one binary. The team rejected Python agent stacks because they had prior Python performance problems and found Python tool-calling frameworks too unreliable to diagnose wrong tool calls without human intuition.

3 supporting quotes

“We get all of this LLM function tool calling capability inside a single Rust compiled binary application that can be deployed almost anywhere”article, stated

“Maybe we were too early or asking too much, but Python just could not keep up. We had a similar experience with “Global Interpreter Lock” (“GIL”) a few years prior when capturing network packet data where we had to re-write significant multi-threaded Python code.”article, stated

“in Python — the language of data science. Experimenting with these early Python-based tools, we found them overly complex, and subject to the same unreliabilities as the underlying models. For example, when enabling an LLM to call function tools with a Python based framework, we found it was very”article, stated

Models used: OpenAI Assistants API · Anthropic APIs · Ollama · llama.cpp · Gpt-4o · Gpt-4o-mini

Scale

Zectonal had been building Rust diagnostic capabilities over the prior 24 months.
“Over the past 24 months, we leveraged Rust to develop a large number of efficient query and file diagnostic capabilities”article, stated
The data inspection example targets files with a million rows and million columns at sub-second speed.
“detecting malformed text inside a single cell in a spreadsheet with a million rows and million columns at sub-second speed.”article, stated
The pipeline scenario extends to similar files flowing every few seconds on a 24/7 basis.
“As an example, think of detecting malformed text inside a single cell in a spreadsheet with a million rows and million columns at sub-second speed. Extend that to analyzing similar files flowing through a data pipeline every few seconds on a 24/7 basis.”article, stated
The framework supports both remote online service and on-premise agent combinations.
“Our framework is extensible, and our future roadmap includes the possibility of best-of-breed agents running in any combination of remote online service and/or on-premise in order to maximize capabilities, ensure a more secure AI experience, and minimize costs.”article, stated

Alternatives considered (roads not taken)

Stay with pre-defined coded rules

Rejected because: The number and frequency of data sources and codecs made rules harder to scale, while LLM function calling could choose diagnostic tools dynamically.

“there was still a scalability issue as the number of data sources and file codecs we supported increased in volume and frequency.”article, stated

Use Python-based agent frameworks

Rejected because: The team had Python performance concerns and found early Python agent tools complex and hard to audit for wrong tool calls.

“As we started to explore AI agents and function tool calling, we observed firms like LangChain and LlamaIndex which were developed in Python”article, stated

“For example, when enabling an LLM to call function tools with a Python based framework, we found it was very difficult to detect when the wrong tool was called by the LLM. While not the tool’s fault, we felt it was still the implied responsibility of the tool to alert the user that something”article, stated

Rely only on cloud model APIs

Rejected because: Zectonal wanted best-of-breed agents across vendors and on-premise options to maximize capability, security, and cost control.

“leveraging best-of-breed agents across multiple vendors and models in order to maximize functionality, avoid lock-in, and minimize costs.”article, stated

“running LLM’s on-premise to support a wider array of use cases outside of the common Chat and Code Completion use cases.”article, stated

Patterns

`tool-use-orchestration` (agent-architecture)

LLM agents choose and call Rust diagnostic function tools instead of following fixed rules.

“having LLM’s decide what function tools to call instead of our pre-defined rules-based approach.”article, stated

`multi-agent-orchestration` (agent-architecture)

The framework experiments with autonomous agents that spawn new agents and tools on demand.

“autonomous agents that can determine when to create and spawn new agents on-demand based on immediate needs and without human interaction.”article, stated

`observability-tracing` (observability)

Agent Provenance records behavior and motivation so the team can debug prompts, tools, and models.

“Agent Provenance provided visibility to understand what motivated agent behavior, and thereby allowed us to develop better agent prompts”article, stated

`structured-output-contract` (tooling)

The framework incorporated structured LLM tool outputs before OpenAI released structured outputs.

“Structured LLM tool outputs was a concept we incorporated into our Rust framework out of necessity before it was released by OpenAI.”article, stated

`model-routing` (model-operations)

The roadmap combines agents across OpenAI, Anthropic, Ollama, remote services, and on-premise models.

“leveraging best-of-breed agents across multiple vendors and models in order to maximize functionality, avoid lock-in, and minimize costs.”article, stated

Tradeoffs

Build an agentic framework in Rust

Chose A custom Rust framework over Python agent frameworks.

Rationale: Rust gave speed, type safety, auditability, and single-binary deployment, while Python had prior performance and reliability issues for Zectonal.

“Using Rust, we have become very accustomed to just having everything blazingly fast, being fearless about concurrency,”article, stated

“we found them overly complex, and subject to the same unreliabilities as the underlying models.”article, stated

Use LLM function tool calling for data inspection decisions

Chose LLM-decided diagnostic tool calls over Pre-defined rules alone.

Rationale: Function calling scaled internal decision-making over growing file codecs and sources, but required provenance to debug wrong calls.

“it has provided us with a unique way to scale our internal software decision making capabilities that leverage our Rust-based algorithms.”article, stated

“viewing the Agent Provenance provided visibility to understand what motivated agent behavior”article, stated

Support multiple model providers and on-premise execution

Chose Best-of-breed remote and on-premise agents over Single-provider dependency.

Rationale: The team wanted more functionality, security, and cost flexibility without vendor lock-in.

“maximize functionality, avoid lock-in, and minimize costs.”article, stated

“ensure a more secure AI experience, and minimize costs.”article, stated

Gotchas (what bit them)

Wrong tool calls were hard to detect in early Python frameworks.

Root cause: The framework could let an LLM call a wrong function tool without surfacing that something was off.

Resolution: Zectonal built its own auditable Rust framework and Agent Provenance visibility.

“we found it was very difficult to detect when the wrong tool was called by the LLM.”article, stated

“It still required a human to detect and diagnose by intuition if the wrong function tool was called.”article, stated

Rules-based analysis did not scale with data-source and codec growth.

Root cause: More sources and file codecs increased the volume and frequency of cases rules had to cover.

Resolution: The team moved toward LLM-selected function tools backed by Rust diagnostic algorithms.

“there was still a scalability issue as the number of data sources and file codecs we supported increased in volume and frequency.”article, stated

Agent behavior became difficult to predict at scale.

Root cause: LLM agents can choose tools and actions dynamically.

Resolution: Agent Provenance exposed behavior and motivated prompt, tool-description, and model improvements.

“Although we could not always predict the behavior of our agents, viewing the Agent Provenance provided visibility”article, stated

Evaluation

Zectonal evaluates and improves agents through Agent Provenance, intent and confidence metrics, prompt and tool-description refinements, and future model fine-tuning where needed.

Metrics: Agent Provenance · intent · confidence · useful information reward signals

“We started building more and more subjective and objective metrics into each interaction that became part of the overall provenance, things like intent and confidence.”article, stated

“providing feedback to reward providing useful information, and penalize otherwise.”article, stated

Outcomes

Zectonal built an auditable Rust AI agentic framework
The framework supports OpenAI, Anthropic, and Ollama APIs
The system packages LLM function calling into a single compiled binary

The article describes sub-second inspection examples over million-row/million-column spreadsheets and 24/7 pipeline files every few seconds, but does not provide customer adoption metrics.

“Both OpenAI and Anthropic API’s are supported by our extensible Rust Agentic framework”article, stated

“We sometimes refer to this Rust LLM function calling capability as our “genie-in-a-binary””article, stated