Sample Project

Load Testing Introduction

In order to help you getting started with loadUI we have created a sample project that helps you understand how loadUI works and how you set up loadUI tests. The project consists of four TestCases, we’ll walk through them here. A lot of the information you can find here, can also be found inside the TestCases where we use the Note Component to document the TestCases.

Preparations

Before you can run the Sample Project you need to install loadUI. Download it here.

About the Project

We would like to point out that the Sample Project will not bog down your network or in any other way make external calls. It will only use a local simulation of a Web Page and a Web Service.

The Project also simulates tests that you have created in soapUI, loadUI’s sibling.

Let’s get going!

You can start The Sample Project two ways;

1. In the Getting Started Wizard
2. By Double Clicking the Sample Project in the Projects List
   

When you have opened the Sample Project, you will see the TestCases in the Project. In The Sample Project you have four TestCases of increasing complexity and showing different feature sets in loadUI.

1. Getting Started
   in the TestCase 1. Simple Web LoadTest
2. Assertions and Logging
   in the TestCase 2. Assertions and Logging
3. Schedulers and Generators
   in the TestCase 3. Complex Load Generation
4. Controlling Flow
   in the TestCase 4. Complex Load Scenario

Let’s move on and look at one of them. Double Click on the Getting Started TestCase.

1. Simple Web LoadTest

Now, before we move on further, let’s explain the setup of a loadUI Test, or of any Load Test.

1. First we need some kind of load generation, something that sets the request rate and how it develops. In order to generate Load in loadUI you have to add an aptly named Load Generator. The Load Generator is what in loadUI is called a component. Components are the building blocks of a loadUI test.
2. Now, when we have the pulse of the test, the rate at which we’re generating load, we need something to actually test. In loadUI we have two kinds of tests we can run, Web Load Tests, that runs requests to one URL in order to expose weaknesses and the more sophisticated soapUI Test, where take a functional test created in the worlds most used functional test tool, soapUI, and run it under load. What you add in loadUI if you want to run a test of a certain type? A runner of course.
3. Then, when you have a test running with a certain generated load, you would like to see how it performs, right? We would like to analyze the test. In order to do that you need to add an Analysis component.

That’s it! Basically these three components is all you need, but you can build out the test to much more complex setups than that. We’ll look at some tests later that illustrates just how powerful loadUI can be.

Double-click the TestCase to open it!

The Setup

Let’s look at the TestCase and see what the components are based on the trifecta we described in the introduction.

1. Generator:
   For this test we’re using the Fixed Rate Generator. This is a very simple Generator with no special pattern for load distribution; it just sends requests at a fixed rate. The rate can be per Second, per Minute or per Hour. You can change it by grabbing the switch and moving it. Changing the rate is the same; grab the Rate Knob and Pump up the Volume! The generator is set to run 10 requests per second.
2. Runner:
   The Runner used is also very simple. We’re using the Web Page Runner which calls one URL over and over again. As you can see the URL is http://localhost:8081, we’ll explain a bit more about it later. Also, on the right side of the component you can see a lot of useful statistics.
3. Analysis:
   The Statistics Component draws a graph based on the results it’s getting from the Web Page Runner. How does it get the results? By connecting the two with a connector of course!

The Extras

Remember we said we talk about the URL in the Web Page Runner being http://localhost:8081?

Now’s the time to explain why; in order for you to actually run the test without crashing a server somewhere. We have a simulated server running on your local machine from inside loadUI.
This is a great tool during the development period when you’re creating your tests, it might not be entirely popular if you slow down the development, or even worse live, server for the rest of the team.
Creating and modifying the tests using a simulated server is the solution. An attentive reader might say “Now, that sounds a lot like a soapUI MockService…” and you’re right, it is. Meet the soapUI Mockservice Component.

The MockService is used in this tutorial to have something to test against . It is started when the LoadTest starts and the number of Requests will be the same as the Requests sent by the WebPage Runner

The LoadTest

1. First, Make sure you are in "Local Mode" in the System toolbar at the bottom of this window
2. Then run the LoadTest by pressing the Run button on the top left toolbar (Next to the TestCase menu).
3. You will see the Web Page Runner starting to send requests and statistics being displayed in the Statistics Component.
4. This LoadTest has no configured limit; it will run until you stop it manually with the Stop button. Once you have stopped it, be sure to generate a report with the report-button at the top right.

Note

Some things to try:

• Try increasing or decreasing the load generated by the Fixed Rate Generator (by turning the knob) to see how the statistics change
  
• Open the settings dialog and select which values you want to show in the Statistics Component
  
• Try minimizing Components (with the top-right [-] button
  

That’s it! Let’s move to the next test; 2. Assertions and Logging

2. Web Load Test With Assertions

Intro

Let’s extend the Simple Web LoadTest TestCase. The base trifecta is the same, but we’ll add some verification, assertions as well as a TableLog that shows the output from the assertions.

Preparations

Make sure you are in Local Mode in the System toolbar at the bottom of this window, and then run the LoadTest by pressing the Run button on the top left toolbar

The Limit

If the last TestCase didn’t have a limit, the only way to stop it was to manually stop it, this TestCase will be different. This LoadTest will run until either 100000 requests have been sent or there have been 100 assertion failures (whichever comes first).

This is set in the TestCase Toolbar where you can see;

1. How long time a TestCase has run and what the limit is
2. How many Requests a TestCase has run and what the limit is
3. How many Failures s a TestCase has had and what the limit is

Please note that Time only has one value in our sample, this is because we have not set a time limit for the test.

The Setup

As we mentioned earlier the basic test is the same, so let’s look at what we’ve changed.

1. Assertion
   This Assertion is connected to the request output of the Web Page Runner, allowing it to assert (validate) the contents of each requests. We will set it to check that the TimeTaken is never over 100ms
2. Assertion (1)
   This assertion is connected to the second output of the Web Page Runner, which outputs the stats you can see in the display of the runner. We will use the assertion to make sure there are no Discarded messages.
3. Assertion (2)
   This last assertion is used to assert the Avg response time as calculated and output by the statistics component. We will set the max value to 50 ms
4. Table Log
   The TableLog is connected to the output of all assertions; it will contain all failed assertions which we can export or log continuously while running

Note

Note a few things:

• What Value you can use in the Assertion depends on what output terminal (a terminal is that round little thing used for connecting components) you choose. For example the Web Page Runner delivers different Values in the Results (connected to Assertion) and in the Runner Statistics (connected to Assertion (1)).
  
• Table logs can be saved to file for further analysis, just click the save button
  

3. Complex Load Generation

Intro

Let’s not extend the test we’ve been looking at previously, but create a completely new test that shows how easy it is to create more a more complex, but still totally realistic scenario. We’re going to look into the Scheduling Components, which are created to control how tests run connected to time.

Preparations

As before: make sure you are in Local Mode in the System toolbar at the bottom of this window, and then run the LoadTest by pressing the Run button on the top left toolbar

The Limit

This TestCase will run for 10 minutes. When those ten minutes are up the TestCase will stop.

The Setup

Here’s a bit of complexity but the test we’re creating is very sophisticated and very hard to do with commercial software.

The Scheduling

To set up how the test will run, we have three different Schedulers, which is what’s used in loadUI to manage how the test runs over time they are connected to three different Load Generators that loads a Web Page Runner with different strategies.

1. The first two minutes
   First we set up an Interval Scheduler that runs for two minutes. This scheduler controls a Ramp Generator that starts by sending 0 requests per second and ramps up to 100 requests per second.
2. The following 8 minutes
   For the final 8 minutes of the test we run at the rate of 100 requests per second but use the Random Generator to make the load more unpredictable. The randomness is Gaussian and has a random factor of 50%. Let’s walk through all controls in the component.
   
   1. Base Rate is just that, the Load Rate we will base our randomness from. If we didn’t random it at all this is what we would have all the time.
   2. Unit is how often the Rate should change, in our case every minute.
   3. Distribution is how the randomness work. Not all randomness is like flipping a coin, or Uniform randomness as we call it. For example in the randomness we use, Gaussian, we are more likely to get a value close to the base rate. Gaussian Distribution is also called normal distribution or a Bell Curve. The random factor works differently depending on which distribution you use, but in our case, the Gaussian distribution, this means the higher the factor, the flatter the curve is and the more likely we are to have a high spread. That is, the test is more random.
3. During the 8 minutes
   During the last 8 minutes we’ve added an extra spice. We’ve added a Scheduler to the 8 minute Interval which is constantly running. And on every day at every hour every minute at the 00 mark it runs for 10 seconds. A normal use case for The Scheduler is running it at a certain time every day or something similar.

The Scheduler controls a Fixed rate Generator that send 100 Requests per second.

Note

Note a few things:

• How do you know when an Interval is running?
  When any component is running the little diode in the top left corners is blinking, but on an Interval you can also see a small meter in the Test Duration Progress bar.
  
• If you look at the Generators, when a Scheduler starts or stops a Generator it also turns it on or off as you can see on the ON/OFF button. The Generators will also blink.
  

4. Complex Load Scenario

Intro

Last TestCase we looked at the components of the Scheduler type, there is another very useful type of component; the Flow Component. If the Schedulers control how the test is related to time, the flow components control how the project flows, hence the name.
If you think of the test data flowing from the components in the connector lines, the flow components are able to control which direction the data flows and even stop the flow.

Let’s look at the flow components and see how they can be used.

Preparations

As before: make sure you are in Local Mode in the System toolbar at the bottom of this window, and then run the LoadTest by pressing the Run button on the top left toolbar.

The Limit

This TestCase will run for 10 minutes or for a 1000 requests. LoadUI will stop whenever of these two conditions is met.

The Setup

Let’s look at the how the final TestCase in the sample project is set up.

1. The Generator
   We are using a fixed rate generator to create a steady base load in the TestCase. Running at 96 requests per second, the generator will create a decent load.
2. The Splitter
   Central to the TestCase is the splitter. The splitter will split up the load according to the Type we’ve set. For this test we’ve chosen round robin which passes messages on in a steady round robin order.
   
   We also have the random type which splits the requests randomly. You can adjust the number of outputs you have simply by moving the Number of Outputs slider.
3. The Runners
   The first execution path running from the splitter goes to three Web Page Runners going to different web pages (http://localhost:8081/index.html, http://localhost:8081/userguide/index.html, and http://localhost:8081/search.html). Between the second and third Web Page Runner we will add a Delay Component.
   
   A delay component is a component that delays sending the message on in the flow. The messages can be set to run with a certain amount randomness as well. The purpose of the delay component is to add a certain amount of unpredictability, making the test more lifelike. The output from the third component will also be shown in a graph component.
   
   The second execution path is pretty similar although using a a soapUI Runner, but a bit more simple except for one small connection. This connection comes from the output of soapUI Runner in the third path. This means that the number of requests from the second runner actually is double.

Note:

• A Delay Component can be used to create more lifelike behavior in the tests.
• The Splitter Component can be used to create parallel execution paths, adding a new path is done by using the slider in the component.

Conclusion

That's it! You've now seen how tests can be created in loadUI, you've seen that loadUI supports creating the most simple quick test possible to the most sophisticated user behavior test possible.
Now, go forth and test! Use your imagination! Find those bottlenecks!