Agentic Search and Context Engineering for Production LLM Systems

elastic · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production search and retrieval for LLM applications

Elastic needed search systems that could assemble context dynamically instead of relying on a fixed one-shot RAG pipeline.

Constraints:

multiple context sources
tool-selection reliability
latency-sensitive retrieval choices

Architecture

Elastic frames context engineering as a tool-mediated search layer between many context sources and the LLM context window. The demos move from fixed RAG to an agent that decides when to call a semantic Elasticsearch tool, then to a general ESQL query-execution tool plus an agent skill for syntax guidance and error correction, and finally to shell/file-system search with exact grep and semantic CLI search. The production lesson is to curate a low-floor/high-ceiling tool stack, log agent behavior and breakages, then add purpose-built tools where repeated failures or excessive tool calls appear.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  llm-agent[LLM agent] --> semantic-search-tool[Semantic search tool]
  semantic-search-tool --> elasticsearch-vector-store[Elasticsearch vector store]
  llm-agent --> agent-skill-loader[Agent skill loader]
  agent-skill-loader --> esql-query-tool[ESQL query tool]
  esql-query-tool --> elasticsearch-vector-store
  llm-agent --> shell-tool[Shell tool]
  shell-tool --> local-file-system[Local file system]
  shell-tool --> semantic-cli[Semantic CLI]
  llm-agent --> logging-feedback-loop[Logging feedback loop]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`llm-agent`	Chooses whether and how to retrieve context, invokes tools, reads tool responses, and produces the final answer.	`semantic-search-tool`, `esql-query-tool`, `shell-tool`, `agent-skill-loader`, `logging-feedback-loop`
`semantic-search-tool`	Simple low-floor retrieval tool over an Elasticsearch vector store using an embedding model and top-k similarity search.	`elasticsearch-vector-store`
`elasticsearch-vector-store`	Database holding chunked conference-session text and metadata fields for semantic search and ESQL demos.	`semantic-search-tool`, `esql-query-tool`
`esql-query-tool`	General-purpose database query tool that lets the agent execute full Elasticsearch query-language statements and self-correct errors.	`elasticsearch-vector-store`, `agent-skill-loader`
`agent-skill-loader`	Progressively loads ESQL syntax guidance into context before the agent calls the general-purpose query tool.	`llm-agent`, `esql-query-tool`
`shell-tool`	General-purpose terminal tool for navigating local files, running grep, invoking semantic CLI search, and using database or web CLIs under sandboxing.	`local-file-system`, `semantic-cli`
`local-file-system`	File-based context source containing session data for shell-search demonstrations.	`shell-tool`
`semantic-cli`	A semantic-search CLI paired with the shell tool for fuzzy retrieval over files when grep-style exact matching is insufficient.	`shell-tool`
`logging-feedback-loop`	Observed agent behavior, breaks, and repeated tool-call patterns used to decide when to add more purpose-built search interfaces.	—

Data flow

A user asks a question. The agent decides whether it needs retrieval, chooses among available search tools, and either calls a semantic search tool against Elasticsearch, loads an ESQL skill before generating a database query for the general-purpose query tool, or uses a shell tool to explore local files. Tool responses are returned to the LLM context window and the agent answers. In production, teams log tool behavior and use observed breakages or repeated tool calls to decide where to add more specialized interfaces.

4 supporting quotes

“now the agent can decide by himself by itself uh whether or not to call the search tool and retrieve some information.”transcript, stated

“We have elastic search database and we're going to be writing a semantic search um search tool. And for in the database I have the conference session um already chunked.”transcript, stated

“always use elastic the elastic search ESQL skill to generate the ESQL query before using this tool because otherwise if the agent then still uses this tool first without without um using the agent skill then that would be a shame. Right, I'm doing the exact same.”transcript, stated

“Generally speaking, logging your agents behavior recommended. Um, but if you notice, maybe your agent is taking four or five tool calls per question.”transcript, implied

Why this architecture

Elastic argues that no single search tool is sufficient. Specialized tools give the agent a reliable low floor for common operations, while general-purpose query and shell tools provide a high ceiling for unexpected or complex questions. Skills, system prompts, error handling, sandboxing, and behavior logging are the controls that keep flexible tools usable in production.

4 supporting quotes

“if you think about it, doing good search is incredibly difficult. So ideally you want to have or curate the right set of search tools”transcript, stated

“So ideally you want to have or curate the right set of search tools for your agents search behaviors and you want to have a combination of specialized tools and a combination and uh general purpose tools. So specialized tools are something that the agent can use out of the box.”transcript, stated

“If you have the perfect tool description and your agent still doesn't call the right tool, then reinforce it in the agent system prompt.”transcript, stated

“since it's so general purpose, the agent sometimes might need more iteration to iterations to actually get to the right answer.”transcript, stated

Models used: GPT 5.4 Nano · GPT 5.4 Mini · new genome embeddings v5

Scale

The demo uses top-k three semantic search to intentionally show how a narrow search tool can be brittle.
“I'm setting the limit or the top K to three. This is a little bit of foreshadowing because I'm limiting the capabilities of this tool”transcript, stated
The ESQL example uses database-side aggregation to count 27 sessions rather than asking the LLM to count items in context.
“it's filtering for April 8th. And then it also does an aggregation, some some counting and tells me today there are 27 sessions.”transcript, stated
Elastic's practical threshold for adding specialized tools includes seeing four or five tool calls per question.
“Um, but if you notice, maybe your agent is taking four or five tool calls per question. That's too many tool calls. Then you probably that's probably an indicator that the tool your agent has is too difficult for it to use.”transcript, stated

Alternatives considered (roads not taken)

Fixed RAG retrieval pipeline

Rejected because: It retrieves whether or not context is needed and cannot naturally perform multi-hop retrieval when the first result reveals another query.

“since this is a fixed pipeline whether or not you actually need any context, you're still retrieving additional information”transcript, stated

“if you're only retrieving once, maybe the retrieved chunks um reveal some information about another search query you need”transcript, stated

A single simple semantic-search tool

Rejected because: It is useful only for a narrow scope and fails on keyword/filtering needs such as the GDPA session demo.

“we can see the search tool we just created. It's not very useful or at least useful only for a very narrow scope of use cases.”transcript, stated

A one-sentence or underspecified tool description

Rejected because: As parameters and tool counts grow, the agent needs trigger conditions, relationships, and sometimes system-prompt reinforcement.

“if you add more parameters or more tools and your agent is starting to struggle with calling the right uh tool, then maybe add some trigger condition.”transcript, stated

Only a shell tool for all search

Rejected because: Shell tools are versatile but risky, require sandboxing, and exact grep can be inefficient for semantic search unless extended with specialized CLIs.

“Using the shell tool can be risky since giving your agent access to a terminal can make it delete files or do other things you don't want it to do.”transcript, stated

“Is it the best? I let you decide. So at the moment there's many different semantic search alternatives to grab.”transcript, stated

Patterns

`tool-use-orchestration` (retrieval)

The agent decides whether to call retrieval tools and can make multiple calls when the first result reveals a need for additional search.

“So that's why we then moved on to aentic rag. So we replace the fixed pipeline with now a search tool. So now the agent can decide by himself by itself uh whether or not to call the”transcript, stated

`hybrid-search` (retrieval)

Elastic frames production search as a curated stack of vector, keyword, sparse, dense, and multi-vector approaches selected by need.

“So depending on what kind of search requirements and latency requirements you have,”transcript, implied

Tradeoffs

How much general-purpose capability to expose to the agent.

Chose shell/database query tools for flexible search over only a narrow semantic-search tool.

Rationale: General tools unlock more search strategies but require sandboxing and safeguards.

“Using the shell tool can be risky since giving your agent access to a terminal”transcript, implied

How broad the agent search-tool set should be.

Chose a curated mix of specialized low-floor tools and general-purpose high-ceiling tools over one silver-bullet search tool.

Rationale: Common queries benefit from simple reliable tools, while unexpected or complex queries need flexible query/shell interfaces.

“If you're looking for just one silver bullet tool, that's probably not the right way to go.”transcript, stated

“you want to have a combination of specialized tools and a combination and uh general purpose tools.”transcript, stated

How to choose new purpose-built search interfaces over time.

Chose start general, log agent behavior, then add specialized tools where repeated breaks appear over designing all specialized retrieval tools before observing real agent use.

Rationale: The presenter recommends behavior logging as the signal for tool difficulty, repeated calls, and useful future interfaces.

“if you don't know your agents query behavior yet, then I would recommend to start with a general purpose tool. Then log your agent's behavior.”transcript, stated

“after three days I was asking it what kind of interesting patterns do you see and was recommending me actually to um implement some specific uh search tools”transcript, stated

How to make general database query tools reliable enough for agents.

Chose agent skills plus error-returning query tools over letting the agent generate complex ESQL with only a terse tool description.

Rationale: Complex query syntax is a failure mode; skill guidance and tool-level errors let the agent self-correct instead of crashing the workflow.

“letting the agent write an entire SQL query from scratch can be quite challenging.”transcript, stated

“I return the error response to the agent so it can kind of self-correct, rewrite the query.”transcript, stated

Gotchas (what bit them)

Agents can skip retrieval or pick the wrong search tool.

Root cause: Tool descriptions and system prompts may not give enough situational guidance when several tools are available.

Resolution: Add trigger conditions, relationship guidance, and system-prompt reinforcement for tool choice.

“At Elastic, we help a lot of internal and external teams build uh agents based to interact with elastic search data. And the reality is that this can break in many different ways. I'm just going to show you three um today. So the first is the agent doesn't call any tool. So this means the agent d”transcript, stated

Evaluation

The presentation uses live demos that deliberately break simple search setups and then tests richer retrieval tools.

Metrics: tool call correctness · retrieval relevance · latency fit

“got the very similar search results back. So after that it decided that sorry it decided that it's now able to um respond with the right talks. This is where most agentic search demos fail. But this is very brittle. Does anyone have an idea how we can break this? Yes, that's a good idea. Anything”transcript, implied

Outcomes

The case yields reusable design guidance for moving from fixed RAG to tool-mediated context engineering.

Presented as a production pattern from Elastic work with internal and external teams rather than a single deployment metric.

“Um, the retrieval tool gives you the tool response and then your agent uh responds to you with the correct answer. At Elastic, we help a lot of internal and external teams build uh agents based to interact with elastic search data. And the reality is that this can break in many different ways. I'”transcript, implied