Assertions

Getting Started with Assertions

User Rating: / 4  |  Rate this article: PoorBest 

{ArticleIndex}

This tutorial assumes that you are familiar with the basics of LoadUI and Statistics.

Video Tutorial



Preparations

asserionsStarded1

The starting-point is a simple load test containing one Generator and one Runner. Start the project and then go straight to the Statistics Tab and drag the Runner from the toolbar to create a chart.

Creating Assertions

Let's pretend that we considered a TimeTaken of more than 100 milliseconds unacceptable. To create an assertion for this, go back to the main window of LoadUI and open the Assertions bottom panel. Now drag the icon representing your Runner to the empty assertion list.

asserionsDrag1

A dialog that lets you configure your assertion will appear. All statistics that are available in the Statistics Tab are also assertable. Additionally, you can assert real-time values, meaning the actual value (not a per-second calculated Min, Max, Average, etc.). Chose Time Taken and then Real-time value. In the Constraint section, set Min to 0 and Max to whatever value makes sense for you. In this tutorial we'll set Max to 100, since we don't want requests to take more than 100ms.

asserionsCreationDialog1

Click Create and your new assertion will show up 1_snag_evi in the Assertions Bottom Panel. From now on, each time the constraint is violated, the assertion's Failures counter 2_snag_evi will increase.

asserionsRename2

To rename your assertion click in the assertions menu, and then click Rename.

asserionsRename1

Viewing Assertion Failures

Your assertion will now be available in the toolbar in the Statistics Tab. Drag it to a new or existing chart to see its failures.

asserionsDragToChart1

Failures are represented as vertical lines in the graph. In some cases (when zooming out or when asserting real-time values), multiple failures may be grouped into a thicker vertical line.

asserionsInChart

Open the Event Log bottom panel and you will find that all assertion failures are logged in text format as well, with a bonus: It contains the actual value that caused the failure. The Event Log is part of the result and will be auto-saved together with the statistics.

asserionsEventLog1

Setting Failure Limits

Suppose that you would like the project to stop when a certain number of failures have occured. This is especially useful when running unattended tests, e.g. long running test or tests running from the command line. Such limits are configured by clicking the Set Limit button, residing in the playback panel.

asserionsSetLimits1

Just set Failure limit to a reasonable number (in this tutorial, we wanted no more than 30 failures).

assertions_gettingStarted_setRunLimits

Immediately when the limit is reached, LoadUI will stop all VU Generators and the test will soon complete.

assertions_gettingStarted_projectStopped

 

What are Assertions?

User Rating: / 5  |  Rate this article: PoorBest 

{ArticleIndex}

What it is

An assertion is a logic constraint on a value, e.g. that the time a request takes may not exceed 100ms. Any time that constraint doesn't hold, an Assertion Failure will be created. This means that you can:

  • See Assertion Failures in the Statistic Tab. Thus, they can also be compared, reported and styled and are auto-saved together with the test run.
  • See Assertion Failures in the Event Log, together with information such as the Assertion's name and the value that failed. All this is auto-saved with the rest of the test run.
  • Set a Failure Limit for your project which, when reached, will stop the project.

assertionsInChart

Why you need it

One great thing about LoadUI is that it records and saves all statistic data from all VU Scenarios, Components and Server Monitors. However, you shouldn't have to spend your time browsing through hours of chart lines for extraordinary values — instead let Assertions handle that for you!

Assertions are perfect for verifying that your service/application passes the SLA (Service Level Agreement). For example, you can assert that:

createNewAssertion

The difference to SoapUI assertions

As you may know, SoapUI also has the concept of assertions, and those assertions can be evaluated by the SoapUI Runner. However, they will not appear in the LoadUI user interface. Here's how we see it:

  • SoapUI assertions
    • Functional assertions - used to assert that a request is functionally correct. E.g:
      • Response is valid SOAP
      • Response contains an element called customerId.
  • LoadUI assertions
    • Performance assertions - used to assert key performance values from the user or server point of view. E.g:
      • A request never takes more than 2 seconds.
      • The database cache miss rate may not exceed 25% for more than 10 seconds.

 

A different approach

User Rating: / 3  |  Rate this article: PoorBest 

A different approach for logging extraordinary events in LoadUI is to use the Condition component, combined with a Table Log. In case you are not familiar with the Condition component, it's a component that sends incoming Virtual Users to either its left output or its right output, depending on whether a condition holds.

condition_withBalloons

In the example setup below, all Virtual Users that experienced a request time (TimeTaken) of less than 1000ms will end up in the Table Log. As you may already know, TableLogs can be configured to automatically save its content to a CSV-file for later analysis.

condition_basic

Using the Advanced Mode of the Condition component, you can type any condition in the Groovy scripting language. Using this mode, the only limit to the conditions is your mind!

condition_advanced