# Asserts and Errors
Test asserts are the part of a test step that define the state to wait for Kubernetes to reach. It is possible to match specific objects by name as well as match any object that matches a defined state. Test errors define states that should not be reached.
Table of Contents
The test assert file for a test step is found at
$index-assert.yaml. So, if the test step index is
00, the assert should be called
00-assert.yaml. This file can contain any number of objects to match on. If the objects have a namespace set, it will be respected, but if a namespace is not set, then the test harness will look for the objects in the test case's namespace.
The test error file for a test step is found at
$index-errors.yaml and works similar to the test assert file.
By default, a test step will wait for up to 30 seconds for the defined state to be reached. See the configuration reference for documentation on configuring test asserts.
Note that an assertion or errors file is optional. If absent, the test step will be considered successful immediately once the object(s) in the test step have been created. It is also valid to create a test step that does not create any objects, but only has an assertion or errors file.
# Getting a Resource from the Cluster
If an object has a name set, then the harness will look specifically for that object to exist and then verify that its state matches what is defined in the assert file. For example, if the assert file has:
apiVersion: v1 kind: Pod metadata: name: my-pod status: phase: Successful
Then the test harness will wait for the
my-pod pod in the test namespace to have
status.phase=Successful. Note that any fields not specified in the assert file will be ignored, making it possible to specify only the important fields for the test step.
If this object is in the errors file, the test harness will report an error if that object exists and its state matches what is defined in the errors file.
# Listing Resources in the Cluster
If an object in the assert file has no name set, then the harness will list objects of that kind and expect there to be one that matches. For example, an assert:
apiVersion: v1 kind: Pod status: phase: Successful
This example would wait for any pod to exist in the test namespace with the
If this is defined in the errors file instead, the test harness will report an error if any pod exists in the test namespace with
When a failure occurs in either an
errors step, kuttl will print a difference (diff) in the test output showing the reason why the step was deemed to fail. While this may be helpful in most cases, it may still be insufficient to determine the exact cause of a failure. Some additional information may be required to fully explain why a step failed which provides fuller context. When the diff is not adequate to explain a failure, a
collectors object may optionally be used to gather further troubleshooting information in the form of pod logs, namespace events, or output of a command.
For example, consider a simple test case in which a pod is created as the initial step followed by an assertion that the pod is present and in a state of
ready=true. If the pod is observed to contain the state
ready=false the step, and test, will fail. With a
collectors object present in the
TestAssert, it may provide logs for the pod to help explain why this state was not reached.
apiVersion: v1 kind: Pod metadata: labels: run: hello-world name: hello-world namespace: default spec: containers: - image: docker.io/hello-world name: hello-world
apiVersion: kuttl.dev/v1beta1 kind: TestAssert timeout: 5 collectors: - type: pod pod: hello-world namespace: default --- apiVersion: v1 kind: Pod metadata: labels: run: hello-world name: hello-world namespace: default status: running: true
In this example, the
hello-world container was not started with any arguments resulting in its running followed by termination as expected. Therefore, the status of
running=true was not asserted.
In the command output, prior to the full diff kuttl displays will be shown the pod's logs.
logger.go:42: 20:06:29 | collectors/1-pod | starting test step 1-pod logger.go:42: 20:06:30 | collectors/1-pod | Pod:default/hello-world created logger.go:42: 20:06:35 | collectors/1-pod | test step failed 1-pod logger.go:42: 20:06:35 | collectors/1-pod | collecting log output for [type==pod,pod==hello-world,namespace: default] logger.go:42: 20:06:35 | collectors/1-pod | running command: [kubectl logs --prefix hello-world -n default --all-containers --tail=-1] logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] Hello from Docker! logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] This message shows that your installation appears to be working correctly. logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] To generate this message, Docker took the following steps: logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] 1. The Docker client contacted the Docker daemon. logger.go:42: 20:06:35 | collectors/1-pod | [pod/hello-world/hello-world] 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. <snip> case.go:362: failed in step 1-pod case.go:364: --- Pod:default/hello-world
See the reference page for more configuration options available with the