The validation of your LoadTest ongoing results is a key aspect of ensuring your target services are performing as expected, for example you might want to ensure that the average response time of your web pages never goes below a certain value, or that the number of failures in a soapUI Runner doesn't exceed a predefined limit. loadUI provides the Assertion component for this purpose; you connect it to whatever components' output you want to validate, configure it accordingly and run your tests.

1. The Assertion Component

When initially created the Assertion looks as follows;

Configuring the component is straight forward; set the "Value" combobox to the incoming message property you want to validate and set the desired Min and Max values. When running your test each incoming message will be analyzed in regard to these values; if the specified property fails the validation the "Failed" counter will increase and an error message will be written to the output terminal at the bottom of the Assertion component (which you could connect to a TableLog for logging). If you want to monitor more than one property you need to create an assertion component for each of them, alternatively if you want to validate the same property from different components you can connect all those to one assertion component and choose desired variable. If one fails it will be detected.

Most commonly you will want to validate the output of Runner components and the Statistics component (although you can of course validate any message coming from any component). Let's have a look at these two in turn.

2. Validating Runner Components

Runners are the "workhorse" components of loadUI; the run requests or tests can collect the results for you to visualize and analyze. All runners output the following basic values:

• Execution and status times for each request - from the left-most terminal
• Test Statistics for the execution of the component in total - from the right-most terminal.

(Individual runner components might have additional outputs, check out their corresponding documentation)

For example connecting the leftmost output of a Web Page Runner to your Assertion will give the following properties to choose from:

For example selecting the TimeTaken value and setting a max value of 1000 will cause the assertion to fail for every request taking more than one second.

Alternatively you can connect the right-most output of the Web Page Runner, and then get the following properties to choose from:

or example if you wanted to check that the number of running web pages is never more than 10 you would select the "Running" property and set Min and Max to 0 and 10 respectively.

3. Validating Statistics

The Statistics component is primarily used for analyzing and visualizing results from Runner components and since it outputs its calculated statistics continuously its values can be easily asserted; connects the output of the Statistics component to an Assertion component and you get the following:

(The properties available in the Assertion component are those that have been enabled in the Statistics components settings)

Here we can easily assert values like the average response time ("Avg" ) and the Transactions per Second (Tps), for example selecting "Avg" in the combobox and setting min and max to 0 and 100 you will test for average response times between 50 and 100 ms during the execution of your LoadTest.

4. Setting the Assertion Limit for your Test

Each time the assertion fails the "Failed" counter increases as does the global Assertion counter in the Run Toolbar:

If you want your LoadTest to fail when a certain number of assertions have occurred, set the limit in the Limits dialog as desired:

(for example setting this to 1 will fail the LoadTest as soon as any assertion in your setup fails)

After set the current progress in regard to the set limit will be visible in the run toolbar:

5. Advanced settings

By default the Assertion component fails for each failed property validation. This may be a bit too harsh; you might want to tolerate individual errors but not bursts of them, which is possible using the Tolerance and Period settings in the Configuration dialog:

For example, if a tolerance is 10 and period is 20 seconds then failure will be raised for every 10th invalid value which was received in last 20 seconds. Basically this means that you must have at least 10 invalid values and a period between the first and the last which must be less than or equal to 20 seconds to raise a failure, a setup that can be useful to catch bursts of errors but ignore individual ones. If a period is set to 0 (zero), then a failure is raised for every invalid value specified by a tolerance parameter disregarding the period e.g. when tolerance is 2 and period is 0, failure is raised for every second invalid value and when tolerance is 1 and period is 0 it is raised for every invalid value.

The "Sample ID" setting lets you specify if a specific source message should be filtered out for validation, for example the soapUI Runner can be configured to output statistics for each contained soapUI TestStep, in which case you could enter the name of a specific TestStep to assert only its properties (provided that the soapUI Runner is connected to the Assertion of course).

6. Logging Assertion Failures

Let's finish off by looking at how failures can be logged to an external file; connect the output of your Assertion component(s) to a TableLog as follows:

Each time the assertion fails it will output a failure message containing information on the actual failure, for example in the screenshot above you can see that the asserted value ("Value") of the TimeTaken property ("Assert") is below the Min value.

Using the advanced settings of the TableLog you can configure this to be written to an external file for later analysis, etc.