The gap between what AI agents can do autonomously and what enterprises can verify they did correctly is driving a new infrastructure category: eval engineering. A SiliconANGLE analysis published May 17 surveyed the emerging vendor landscape, identifying a common bottleneck across the field: leveraging validator agents to govern agentic orchestrations is too slow and expensive to support modern automation requirements at production scale.

The timing tracks with a major acquisition. Cisco announced its intent to acquire Galileo Technologies on April 28, planning to integrate the startup’s eval platform into Splunk Observability Cloud. Kamal Hathi, SVP and GM of Cisco’s Splunk Business Unit, described Galileo as “a complete solution that enables deeper insights from the earliest stages of prompt optimization and model selection, through evaluations, all the way to production monitoring, observability and enforcing guardrails,” according to Security Buzz. The deal is expected to close in Q4 of Cisco’s fiscal year 2026.

The Cost Problem

The central challenge is straightforward: running an LLM to evaluate another LLM’s output doubles inference cost and latency at every step. For agentic workflows where an agent takes dozens or hundreds of autonomous actions per task, that overhead makes production-grade evaluation prohibitively expensive.

Most vendors are converging on the same workarounds. Maxim AI combines offline evals during development with online evals during production, using a sampling-based approach that focuses evaluation on high-risk interactions rather than every action, according to co-founder and CEO Vaibhavi Gangwar, as reported by SiliconANGLE. Arize AI offers continuous lightweight monitoring in production, reserving full LLM-as-a-judge evaluations for high-risk situations, according to SiliconANGLE. Confident AI moves most evaluations to asynchronous observability pipelines, combining traffic sampling with targeted metric collection to reduce compute overhead.

The pattern is consistent: sample in production, evaluate fully only on the most consequential decisions, and run comprehensive evaluation during development.

ChainPoll and Luna

Galileo’s approach differs from the sampling consensus. ChainPoll, the company’s hallucination detection methodology, combines chain-of-thought reasoning with polling: evaluator models must explain their reasoning step by step, and the system runs evaluations multiple times across potentially different models, then aggregates the results, according to SiliconANGLE. The methodology provides a framework for coordinating multiple evaluations while reducing cost.

Building on ChainPoll, Galileo developed Luna, a purpose-built evaluation model designed specifically to detect hallucinations in LLM outputs, including retrieval-augmented generation queries. Where ChainPoll provides the methodology for pass/fail results, Luna delivers a specialized model that operates at a substantially smaller token consumption footprint than general-purpose LLMs used as judges, according to Galileo.

The practical difference: Galileo claims it can offer agentic observability with 100% sampling in production, without requiring asynchronous out-of-band evaluations or evaluations that use only a subset of available telemetry. Competitors achieve cost efficiency by looking at less data. Galileo achieves it by using a cheaper, specialized model to look at all of it.

The Vendor Landscape

Beyond the companies Cisco and the sampling-based startups, several other vendors are building eval infrastructure. Comet ML, Evidently AI, GoodEye Labs, and the open-source MLflow Project under the Linux Foundation offer eval engineering for pre-deployment testing, according to SiliconANGLE. Conscium delivers controlled virtual simulations to identify unsafe agentic behavior, goal drift, and policy violations before agents reach production. Klover AI uses eval engineering for decision support rather than automation, extracting and evaluating each fact within input data for accuracy across opposing viewpoints.

Google, Microsoft, and IBM are also active in eval engineering tooling, though the startup ecosystem is moving faster on purpose-built agentic evaluation, per SiliconANGLE.

Why Cisco Paid Up

Cisco acquiring Galileo signals that AI observability has crossed from engineering tooling to enterprise governance infrastructure, according to Security Buzz. Multi-agent systems introduce compounding failure risks that single-model evaluation frameworks were not designed to handle. Security, compliance, and cost exposure all increase as agents operate with greater autonomy.

For teams building or deploying agent workflows, the practical takeaway is that eval engineering is becoming table stakes. The question is no longer whether to evaluate agent behavior in production, but whether to sample a fraction of actions cheaply or evaluate all of them with a specialized model. Cisco is betting the specialized model approach wins at enterprise scale.