Testing Agentic Systems: New Quality Standards for Enterprise AI

When asked, the DPD chatbot wrote a haiku about how useless DPD was. Air Canada's chatbot falsely told a passenger that he could book a flight at the regular fare and apply for a bereavement discount retroactively within 90 days – a misleading statement for which the airline was later held liable. A Chevrolet dealership's chatbot was tricked into agreeing to sell a $50,000 SUV for just $1, even describing the offer as legally binding. Stories like these are entertaining – at least as long as you’re not the one responsible for the system.

Anyone developing agentic systems for insurance companies, banks, or the automotive industry today is operating in a completely different environment. Here, errors aren’t just embarrassing. They can be costly or even create liability. Sooner or later, therefore, a key question arises: How do you actually test such systems?

While testing is well-established, agentic testing is uncharted territory. Agentic systems place new demands on quality assurance that many traditional CI/CD pipeline approaches and tools can no longer meet. It’s no coincidence that agentic projects tend to fail during the transition from a working prototype to a production-ready solution. In the first part of this series, I described why 80% of AI projects fail at this hurdle and why a platform strategy is the structural solution. This article takes it a step further: How can you test agentic systems that are developed and operated on such a platform?

Why 'assertEquals' is no longer enough

A traditional unit test checks whether a function produces exactly the specified output for a given input. This works because the function is deterministic. An LLM is not. It generates a probability distribution of possible answers. Therefore, even with an identical prompt, a different answer may be produced the next time it is called. A test that insists on exact string equality is thus, by its very nature, a flaky test. That’s the first hurdle.

The second hurdle is subjectivity: Depending on the context and the user’s preferences, the same response could be correct, incomplete, or phrased in the wrong tone. The third hurdle is latency. A test that waits 20 seconds for an agent’s response is no longer a unit test. The fourth hurdle, which is often underestimated, is scope: What exactly is being tested?

Agentic systems are not single functions. They consist of a system prompt, an LLM, a conversation state, tool calls via MCP servers, guardrails and an agent loop. If any of these components changes - and change is inevitable in software development - the entire system must be re-evaluated.

Asking the Right Questions Before Establishing Metrics

Before test cases are defined, it must be clear exactly what is to be measured. For agentic systems, the questions go beyond checking whether a return value is correct.

Is the retrieved information actually relevant to the request? Is the agent hallucinating, or more precisely, is it confabulating by inventing facts that are not contained within the available information? Is the response complete? Does the tone match the intended communication? For example, a chatbot should not shout at a corporate customer or address them informally without being asked to do so. Does the agent respond in the user’s language? Does it use the technical terminology required by the organisation or the customer?

None of these questions can be answered with a simple ‘yes’ or ‘no’. Furthermore, not all quality criteria are equally relevant to every use case. Deciding which to prioritise is part of the development process.

This raises the crucial question: What does 'good' actually mean? Is an 80% resolution rate acceptable for a customer support agent? Or does it have to be 90%? For many use cases, such figures would be acceptable. For others, however, they are completely inadequate.

A real-world example makes this clear: If software is used to control train routes in a rail network, it must guarantee absolute reliability and prevent conflicting routes from being cleared simultaneously. In this case, a success rate of 95 per cent is by no means sufficient, since even a single incorrect decision can have direct, safety-critical consequences.

Defining what constitutes 'good' is therefore not a technical detail. It is a business decision that must be made before the first test case is written or the first metric is defined.

Frameworks: Which Tool for Which Task?

The market for frameworks built for various aspects of AI testing is growing.

DeepEval is a pytest-native Python testing framework that integrates seamlessly into existing CI/CD pipelines. It enables developers and SDETs to verify LLM outputs just like traditional unit tests, using automated pass/fail thresholds. With over 50 built-in metrics - including G-Eval for qualitative rubrics, hallucination and bias checks, and specific metrics for agents and RAG pipelines - it is ideal for automated regression testing and quality gate checks before every deployment.

Ragas (Retrieval Augmented Generation Assessment) is a specialised, reference-free Python framework designed to accurately evaluate Retrieval-Augmented Generation (RAG) pipelines. It breaks down the evaluation into the core components of retrieval and generation and measures standardised metrics such as faithfulness (how well the answers align with the context), Answer Relevancy (how relevant the answer is to the user’s question), context precision, and context recall. This enables teams to specifically diagnose whether weaknesses stem from flawed retrieval logic or from the generation process.

Promptfoo is an open-source, CLI- and YAML-driven framework specialising in prompt testing, direct comparison of different model providers (multi-model A/B testing), automated red teaming and security testing. It systematically scans AI applications for vulnerabilities such as prompt injections, jailbreaks and other security risks, thereby serving as a reliable safeguard before applications move to production.

In practice, these tools complement each other perfectly. Promptfoo is used for security testing and prompt engineering, while DeepEval handles automated CI/CD quality assurance. Ragas, meanwhile, provides the necessary depth for evaluating RAG architectures.

Continuous Testing: The agent runs on the platform, and so does the test.

The real challenge begins with the continuous and automated evaluation of the system, not the first test run. After all, changes are an everyday occurrence: a new model, an updated tool version or a modified system prompt, for example. Each of these changes can affect the agent’s behaviour.

The main problem with integrating AI tests into traditional CI/CD pipelines is the resulting bottleneck. While traditional unit tests run in milliseconds, AI evaluations, particularly LLM-based assessments ('LLM-as-a-Judge') and complex RAG or agent test runs, require much longer execution times. If these long-running AI evaluations are integrated directly into the standard CI/CD pipeline as blocking steps, feedback loops and deployments will be significantly delayed. To avoid this, it is necessary to decouple AI tests from the primary build pipeline and implement asynchronous, scalable test execution on dedicated infrastructure.

This is where TestKube comes in. TestKube is a cloud-native testing framework which integrates test execution directly into Kubernetes. The key advantage is that tests run on the same infrastructure as the agent. There is no need to expose APIs that should remain private. There is no test client hammering away at a staging system from the outside. The Ragas test runs in the same cluster as the agent.

TestKube responds to Kubernetes events by automatically running the appropriate tests as soon as there is a change to the agent deployment, ConfigMap or MCP service. This creates a fully automated quality loop: code is pushed; the build runs; and GitOps tools, such as Argo CD or Flux, handle the deployment. TestKube then detects the change and runs the defined tests. The results are then fed into a central quality dashboard.

More Than Just Testing: The Agentic Control Plane

Testing is necessary, but not sufficient, especially in regulated industries. Those developing systems for insurance companies, banks and other highly regulated sectors require more than just test reports. What is needed are audit trails. Which agent modified which backend data? Who authorised an action? A test framework alone cannot meet these requirements.

Our open-source project, Agentic Layer, addresses this issue. It is a Kubernetes-native reference implementation of an Agentic Control Plane. In the second part of this series, I described the three gateways (Agent Gateway, AI Gateway and Tool Gateway) and the interaction of the architectural components in detail. 

From a quality assurance perspective, the test bench lies at the heart of the system. It is by no means an afterthought, but rather a universal, cross-cutting concern of the entire control plane. All evaluations take place continuously and directly on the live system - event-driven and on the same infrastructure - and are centrally aggregated in the observability backend.

The project is available under the Apache 2.0 licence and is freely available at GitHub. It is not a finished product, but rather a set of conceptual ideas and reference implementations that teams looking to build an AI platform for an enterprise context can use.

What This Means for Teams

Agentic systems do not require fundamentally new quality standards. The established principles of software quality still apply. However, they need to be supplemented with additional metrics and testing methods to ensure quality in non-deterministic, autonomous systems.

The good news: Much of what has become established in the cloud-native environment in recent years remains valid: CI/CD, GitOps and continuous testing continue to form the foundation. The metrics, evaluation methods and understanding of what 'correct' means are new.

Teams that integrate a testbench into their platform from the outset lay the groundwork for operating agentic systems reliably, securely, and with full traceability in production.